Rawle was deliberately lying and Madison was shaving the truth because he wanted William Smith seated. Madison clearly did not believe his own rhetoric as demonstrated by his treatment of James McClure being held in French Custody in the run up to the War of 1812.
Makes you wonder what else Rawle was lying about in his book, doesn't it?