“OK I thought traditionally it was the chief justice”
It usually is, but doesn’t have to be. Lyndon Johnson was sworn in aboard AF1 by a federal district court judge, Sarah Hughes.
I seem to recall Coolidge was up in Vermont or NH and the local county judge or someone like that swore him in when he found out about Harding dying.