The reason that the military has standing to remove him is as follows: Ever single order given to a military person is given, in effect, by the Commander in Chief. If that Commander is found to be not qualified to give that order, then no order given by any general to any subordinate is legitimate, and every action performed in this chain of command is therefore illegal. There is no pretext, no validation for their actions. They could be liable for God knows what, especially if it could be proven that they knew about the fraud and failed to act. It would be a horrible situation to be in--one I wouldn't wish on anyone.
You win the prize.
That is EXACTLY the scenario.
They have no such authority. Precisely, they cannot issue binding orders to combat commands to protect or enforce such an action.
And, presuming that they did, seemingly valid orders from the chain of command would follow immediately to effect their arrest and imprisonment.
This is a much bigger problem than you appreciate.
Under such a scenario, EVERY SINGLE COMMISSIONED OFFICER would have to take sides. Most of them, thankfully, score high on loyalty and respect for civilian authority, and know perfectly well that JCS is not in their chain of command.