Thomas Jefferson, for one, was no libertarian. Here is what the author of the Declaration of Independence has to say about all of this:
See the following in a criminal bill that Jefferson drafted. Sources: Thomas Jefferson, The Writings of Thomas Jefferson, Andrew A. Lipscomb, ed. (Washington, Thomas Jefferson Memorial Association, 1904) Vol. I, pp.226-27, from Jefferson's "For Proportioning Crimes and Punishments."
"Whosoever shall be guilty of rape, polygamy, or sodomy with a man or woman, shall be punished; if a man, by castration, a woman, by boring through the cartilage of her nose a hole of one half inch in diameter at the least." (Virginia Bill number 64; authored by Jefferson; 18 June 1779)