I don't believe that Madison's statement allows for the unilateral secession practised in 1860-61. The context was a discussion of the ratification of the Constitution. Later in #39, Madison states that the states are bound by their voluntary act of ratification. An entity that is bound does not possess absolute sovreignty, regardless whether the binding was involuntary or voluntary. Of course, the right of revolution is still available but that line of action is beyond the Constitution.
Here's Madison's statement from Federalist 39: "Each State, in ratifying the Constitution, is considered as a sovereign body, independent of all others, and only to be bound by its own voluntary act."
If the state is "only to be bound by its own voluntary act," i.e., nothing else binds it, then the state can just as easily unbind itself. As long as it has not unbound itself, some aspects of sovereignty have been delegated to the Federal government under the Constitution. But once a state unbinds itself (i.e., secedes, for those of you in East Tennessee), it resumes the powers it delegated to the Federal government.
Consistent with that thought, here is Madison from June 24, 1788 in the Virginia Ratification Convention [my bold below]:
That resolution declares that the powers granted by the proposed Constitution are the gift of the people, and may be resumed by them when perverted to their oppression, and every power not granted thereby remains with the people, and at their will. It adds, likewise, that no right, of any denomination, can be cancelled, abridged, restrained, or modified, by the general government, or any of its officers, except in those instances in which power is given by the Constitution for these purposes. There cannot be a more positive and unequivocal declaration of the principle of the adoption — that every thing not granted is reserved. This is obviously and self-evidently the case, without the declaration.
As you know if you've read the ratification documents, Madison voted for such a resumption of governance statement in the Virginia ratification document. So did Hamilton and Jay, the other authors of the Federalist Papers, for a similar statement in the New York ratification document. The New York document said they could resume their governance if it made them happier.
Where in your opinion does sovereignty ultimately reside? The king, the federal government, the state, the people of the state, the lumpen mass of the people of the whole country, individuals?