I respect Col. Madison's opinion, but the Supreme Court can take a flying leap, if you'll pardon the expression :-).
Just because you don't agree with them does not alter the validity of their decisions. They are tasked with ruling on the Constitutionality of an issue, not you or me.