Libertarianism would work if everyone was born 30 years old with a fully formed brain and was a functioning adult. Unfortunately, to raise a child in a libertarian society is is hell on earth.
You do realize that if any of the founding fathers came back today, they would probably be considered small-l libertarians. Not as far out there are the official Libertarian party, buy way, way, way (way) to the right of today's typical conservative.
The Federalist Papers (and anti-federalist papers, written by people like Patrick Henry who opposed ratifying the Constitution) really sheds a lot of light on how the founders thought. If you think raising a child in a "libertarian society" would be "hell," then you think raising a child in the society envisioned by our founders would be "hell" as well. Because, the society envisioned by the founding fathers (when it comes to the role of government) is a libertarian society.
Yeah, people might actually have to do a little hands-on parenting. They might not be able to raise them free-range with a bunch of government school sports and clubs funded by property taxes extracted at the barrel of a gun. Ye gods...they would actually have to spend some time with the little bastards, and we can't have that!