That's not what the Constitution says, it says
The President, Vice President and all civil Officers of the United States, shall be removed from Office on Impeachment for, and Conviction of, Treason, Bribery, or other high Crimes and Misdemeanors.
Art. II, Section 4.
It also says:
No Person except a natural born Citizen, or a Citizen of the United States, at the time of the Adoption of this Constitution, shall be eligible to the Office of President;
So how can someone not eligible to the office "hold" it? By that argument, someone could walk in off the street and "hold" the office. I mean if you're willing to disregard one requirement, why not a couple more of them? Maybe only one requirement. The guy could sign a copy of the oath, be over 35, 14 years a resident, and be a natural born citizen.
Ridiculous of course, but it does follow from your position.