Some of them came from the Essex Junto, an even earlier New England group of prominent politicians advocating secession.
And if they had attempted to secede the current President at the time would have had constitutional authority to suppress such rebellion and insurrection by armed force if necessary.
Secession was never seriously discussed during the Hartford Convention. There were those there that promoted it, true. But they were a small minority and never came close to having their proposals made part of the convention final report.