Precisely. It's the free-rider problem that, IMO, is the most serious defect of a fully libertarian society, and the easiest way to see it is when considering national defense.
Presumably, since there is no coercive mechanism for funding defense, defense is a fully voluntary, cooperative effort. But the free-rider performs a simple sort of mental calculus - I can, he realizes, get the benefits of defense without actually paying for it. After all, society can hardly leave my house selectively undefended when it is planted right among a whole load of other folks who are paying for defense. So I don't pay, and yet I receive the benefits anyway.
Of course, the system breaks down when enough people do the same calculation and thereby refuse to fund defense. In a sense, the flaw of a hard-core libertarian society is that it relies on people to rationally act in their individual self-interest, except when it comes to something like national defense, where the game changes and people are suddenly supposed to think in terms of the common good. But then again, if people acted consistently acted altruistically, for the common good, communism would work just fine. But they don't, and it doesn't.