Nonsense. By this standard, John Paul II and Bill Clinton would both deserve the exact same respectful hearing if they argued in favor of honesty and chastity.
People who say they are arguing against Buchanan's position, but really only argue against Buchanan, thereby confess that they are unable to argue against Buchanan's position. Which is why the argumentum ad hominem is generally frowned upon, at least by those seeking truth.