Madison is refering to a state being turned out of the union by the others.
The fallacy which draws a different conclusion from them lies in confounding a single party, with the parties to the Constitutional compact of the United States. The latter having made the compact may do what they will with it. The former as one only of the parties, owes fidelity to it, till released by consent, or absolved by an intolerable abuse of the power created.
The underlined clause is important. Madison states that a single party (a state) is released from it's bound of fidelity to the union when an intolerable abuse exists.
I have seen written that this means the state must revolt - I don't agree that the state must declare war upon the union in order to assert it's sovereignty - secession is, in effect, the same statement without hostilities.
That "war" is the likely outcome of such a declaration is obvious. It is also obvious that the founders abhored the idea.