Yep. There were a few states that were considering secession in New England and wanted to rejoin Britain.
That was during the war of 1812. Not at all true during 1860s