The act of secession itself may have been invalid; withdrawing from Congress was not.
What is withdrawing from Congress if not an act of secession?