It was far more than that, as I've pointed out. Southern repudiation for debt. Southern theft of federal property. Southern abandonment of national commitments. Southern threat to cut off the midwest from access to the sea.
I asked this question before and you never answered it. But what the hell, I'll try again. If states can leave unilaterally then why can't they be expelled unilaterally? I know that you'd be thrilled, but why couldn't the U.S. expell Virginia? Turn it out, remove all federal troops, workers, property, revenues, what-have-you and cast you adrift? Wouldn't you agree that if Virginia can leave the rest of the states unilaterally, the rest of the states could do the same?
There is this little song in my head I think it goes "I'm in heaven, I'm in heaven da da da da da da...." Huge smile.......Let me savour that thought for a while.
The answer to this question is in the structure of the Union. The States were first, and the states together created the Federal government. This government is of limited powers, while the several States retained the balance of the power; another way of putting this is that the federal government has only the powers delegated to it by the states, while the States exercised plenary authority in all matters not delegated.
Keeping this in mind, the answer to your question is simple: the mere fact that the states chose not to delegate expulsion authority to the federal government (and could certainly act to expel a state without its consent) does not effect the State's authority (not delegated to the federal government) to leave the Union.