[Albion Wilde]

This was before he was inaugurated.

Nope, it was after, when they went back into the Capitol for lunch.

[stylin19a]

ThankQ.

Here is the CSPAN video of the entire day:

https://youtu.be/ZwvbQR887W0

The Marine non-salute (and if he was President, they would have saluted) event happens at about 3 hours and 30 min mark.
Joe swears in to become President at about 4 hours 47min into CSPAN’s coverage