"An inference from the doctrine that a single State has the right to secede at will from the rest is that the rest would have an equal right to secede from it; in other words, to turn it, against its will, out of its union with them."
Is he right?
Yes, he is. The rest of the states can expell another if they so desire. It's happened with territories. Why not with states?