According to Madison, they did.
"It was generally agreed that the objects of the Union could not be secured by any system founded on the principle of a confederation of Sovereign States." -- Madison Explains the Constitution to Jefferson
Their sovereignty was transferred to the people, to be expressed via the popular vote, which elected Congress, and in turn elected the President.
The Constitution was an agreed upon mode of government which gave equal representation to all States in the decision-making process of the central (Federal) government, and laid out a prescribed method of arriving at decisions as a result of the majority will of a Constitutional body, so then, to secede as a result of a disagreement on the decisions arrived at in a Constitutional method by the majority of that Constitutional body, is not to adhere to the State's rights under the Constitution, but a unilateral rejection of the Constitution as a form of government, as well as the Nation it created. A Constitution and government that in the words and minds of they who both wrote it and by signature agreed to have it ordained, were created for the sake of securing "the Blessings of Liberty to ourselves and our Posterity."
A perpetual Union.
Secession was not, IS NOT, a Constitutional right, secession was a unilateral rejection of the Constitution, and the principles their own antecedents died for.
First, it is always asserted by the 'let them go' crowd, that slavery would have ended.
Since the South was leaving to keep slavery, it is highly unlikely that it would have died a peaceful death.
Second, no where does the Constitution give the right to secede.
Not even the Confederate states wrote it in their Constitution.
The Union was not a coalition of semi-nations.
It was a Federal system that carefully balanced sovereignity between local and federal government.
The right of secession is simply the right of revolution, the right to dissolve a government.
But to invoke that right there must be some real abuses that cannot be resolved peacefully.
Losing an election is not a valid reason to dissolve a government.
If that is true then should not the reverse be true as well? Since it requires a majority of states agreeing to admit a new state, can not a majority of states also expel another state?