Ever heard of the Tenth Amendment?
“The powers not granted to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.”
It’s part of the Bill of Rights. Long before the War Between the States.
Too bad the southern secession didn’t bother to use that as the reason for their action.