You've left out Justice Roberts in this whole debacle. Does he have to recuse himself because he actually swore the Barry into office? It's tough to listen to a case that says the President isn't really the President, when you're one of the key figures involved in his installment in the office.
"The problem is of course that the Constitution doesn't contemplate an ineligible person becoming de facto President."
This above all others, is the primary problem, at least legally. Wherever it is we are today, no one - included the Framers - ever anticipated that we would be here.
Well Sotomeyer would have to recuse herself as she was appointed by the usurper.
The framers did anticipate it, that is why we have the second amendment.
That's a misnomer. The President is not "sworn into office". Swearing the oath is just another requirement to hold the office. The Constitution only says it must be done "efore he enter on the Execution of his Office, he shall take the following Oath or Affirmation:". What makes someone President is meeting all the requirements, including swearing the oath, and then the clock. The oath need not be given by the Chief Justice, or any judge or justice. For example LBJ was first sworn by a local Federal Judge in Dallas. Assuming the requirements have been met, the person becomes President at 1200 EST, January 20th following the election. Arguably we have several short periods with no President, because the oath was not sworn before that time.
You might think that a technical distinction, but then as you say, that's what the law is about. We've made a coronation out of the swearing, but it's not a coronation, it's a ceremonial version of a Constitutional requirement, that could just as well be done by some clerk. Better if it was, IMHO.