It’s worse than that. It wasn’t a threat to convict in the post inauguration impeachment. At that point all of his cabinet had either resigned or made statements against him. Lead by Pence, they could have removed him with a simple cabinet vote on the 7th or any day before the 20th. Trump could have responded that he was not disabled and was in command. That afternoon, Congress could override Trump reasserting office with a 2/3 vote, no trial needed.
The process could have taken a day. And then he and his kids would face immediate arrest right there at the White House.
It was a very real threat and was immediate. Effectively, Pence and Pelosi and Milley were the Ruling junta.
Trump should manned up and pardoned them anyway. A general stands by his troops.