Probably, but since there is no requirement for anyone to "handle the swearing" in, it's hard to say that if the one who does fails to validate the eligibilty of the person being sworn, it has any legal or moral implication at all.
But I think you miss my point as well. Suppose the Chief Justice refused to swear him in without proof of eligiblity? I'm sure one of the 4 liberal Justices, or some friendly Chicago area federal judge would do the job instead. Or he's just sign the oath and be done with it.
Would you not like to see that happen? Especially the public explanation of his refusal to administer the oath.