That would be like saying the President has no right to fire the Secretary of Defense or anyone else he appoints. I remember when Nixon got in trouble politically for firing the Attorney General but he could still legally fire the man!
The President appoints the Chairman of the Fed, but cannot remove him. Whether this is the case here I do not know.