Then why deny people the right to leave your plantation? It's not free will if they have to do what YOU want.
Of course anyone in the south was free to leave. They were not free to start a war against the United States, or to pretend that their home owner’s association could secede from the US.
That they committed their crimes in an attempt to extend, and protect the institution of human slavery and their organized crime syndicate is pretty damning.
People were free to leave anytime they wanted to leave. Many Southerners moved to Brazil and became Confederados. Many of their descendants are still down there waving Confederate flags. Many of them still speak some English.
Everyone who lived in the South was an American citizen. As American citizens, each of them had rights under the Constitution. The secessionists tried to deprive those American citizens of their rights under the U.S. Constitution and tried to deprive them of their American citizenship.
Like many of the bigwigs of his day, George Washington was very dissatisfied with the Articles of Confederation because it left the nation weak and indefensible. He pushed for the creation of an indissoluble Union of the States. Our Constitution makes it clear that it was a bond by and between "We the People of the United States" and not just another league of states. In fact, it was for that very important reason that Patrick Henry was opposed to ratifying the Constitution - "Who authorized them to speak the language of, We, the people, instead of, We, the states?"
The secessionists had the right to leave the plantation. They had no right to try to break the constitutional bonds between the American people or to deprive any Americans of their rights under the U.S. Constitution.