Not hard at all. Just need an actual CONVICTION for the crime of Insurrection.
Which could land you in jail. Congress has to pass a law providing for removal from office.