James Madison, the father of the constitution, thought differently.
The Constitution requires an adoption in toto, and for ever.” — James Madison to Alexander Hamilton, July 20, 1788
Madison’s views expressed decades after the constitution was written and ratified are irrelevant. What is relevant is what the parties ie the states agreed to at the time of ratification. Given 3 states made express reservations of their right to secede at the time of ratification, given nobody said this was inconsistent with the constitution, given Madison’s writings in the federalist papers trying to convince states to ratify the constitution - writings in which he went to great pains to say the federal government’s powers were specifically listed and few while the state’s powers were unlimited, the evidence all points to the conclusion that states are sovereign and are only bound by their voluntary act in ratifying the constitution. In fact, Madison used those words in the federalist papers.