“If the people of those states chose to leave the union, then they had that right.”
But, you can’t put the rights guaranteed in the constitution up to a vote. That is the most fundamental and important element of the constitution, protecting the minority (even of one) from the majority. Essentially, the states voted to remove the protection of the constitution from their citizens and they didn’t have that right.
What rights guaranteed do you assume that I believe were put up to a vote? The people of the states voted to leave the Union. There are no rights or powers enumerated in the Constitution that they voted for or against. The people and the states did have the right to govern themselves on any and all issues that were not enumerated in the Constitution.