So if the larger power says you don’t have rights, then you don’t have rights?
All of the states fought for the land in various wars. You just can't bypass Congress and steal half the country because you love slavocracy. They found out the hard way.