Replies

Madison would not have disagreed with Patrick Henry's claim that our legal rights were granted by the creator. They disagreed on collecting religous taxes.
Read Madison's words in the document you sited.

1. Because we hold it for a fundamental and undeniable truth, "that religion or the duty which we owe to our Creator and the manner of discharging it, can be directed only by reason and conviction, not by force or violence." The Religion then of every man must be left to the conviction and conscience of every man; and it is the right of every man to exercise it as these may dictate. This right is in its nature an unalienable right. It is unalienable, because the opinions of men, depending only on the evidence contemplated by their own minds cannot follow the dictates of other men: It is unalienable also, because what is here a right towards men, is a duty towards the Creator. It is the duty of every man to render to the Creator such homage and such only as he believes to be acceptable to him. This duty is precedent, both in order of time and in degree of obligation, to the claims of Civil Society. Before any man can be considerd as a member of Civil Society, he must be considered as a subject of the Governour of the Universe: And if a member of Civil Society, do it with a saving of his allegiance to the Universal Sovereign. We maintain therefore that in matters of Religion, no man's right is abridged by the institution of Civil Society and that Religion is wholly exempt from its cognizance. True it is, that no other rule exists, by which any question which may divide a Society, can be ultimately determined, but the will of the majority; but it is also true that the majority may trespass on the rights of the minority.

2. Because Religion be exempt from the authority of the Society at large, still less can it be subject to that of the Legislative Body. The latter are but the creatures and vicegerents of the former. Their jurisdiction is both derivative and limited: it is limited with regard to the co-ordinate departments, more necessarily is it limited with regard to the constituents. The preservation of a free Government requires not merely, that the metes and bounds which separate each department of power be invariably maintained; but more especially that neither of them be suffered to overleap the great Barrier which defends the rights of the people. The Rulers who are guilty of such an encroachment, exceed the commission from which they derive their authority, and are Tyrants. The People who submit to it are governed by laws made neither by themselves nor by an authority derived from them, and are slaves.