Your "reasoning" is completely baseless. What you are saying is that, because the Southern states were prevented for staying withdrawn (which they had already done), a right that they had was taken away and a new law created through simple military force with no constitutional precedent or Federal law being cited. This type of "reasoning" is completely asinine. Your argument would be laughed out of a freshman-level survey course!
How long did the rebels hold Nashville? New Orleans?
How long was South Carolina without federal troops within her borders?
No state has ever been out of the Union.
Walt