Because the people who set up the Fed made sure that it was unelected - and unaccountable.
Now, we might as well complain about the wind as complain about the Fed.
Congress and the president can hold them accountable. Congress has the power to require testimony. It can also change any laws regarding the Fed. And the president has the power of appointment. It would also help if the press didn’t grant infallibility to central bankers.