Obviously the existence of utilitarianism and pragmatism, as well as the individual "pursuit of happiness" so long as you do not materially harm anyone else has never crossed your radar.
Oh but it has, many times. I used to think that way myself.
But it is becoming increasingly difficult for me to believe that there is some kind of hermetic seal between a mythical "self-sufficient individual" and the rest of society. And it is becoming increasingly clear to me that lines of thinking which declare "society does not exist" are a kind of cultural Bolshevism that are every bit as destructive of society as the Marxist variety. If the libertine line of reasoning were truly valid, we should not have seen so many disasterous results from the destruction of traditional Western values and norms.
Of course I am sympathetic to small, limited government. But small, limited governments work best with a moral society of individuals who do not think of themselves as social atoms who can do whatever they want with impunity because they do not believe their actions have any effect on others. Small government works best with a moral society which is not afraid to enforce its values, with laws as well as with customs.
It is no accident that moral corruption and Big Government go hand-in-hand; the two feed off each other. This is the one fact the libertines fail to grasp. They also fail to demonstrate how a society is supposed to reform itself without resort to the State or its laws.
The best mere "voluntary action" can hope for in a libertine notion of politics is to create pockets of resistance, powerless to affect or reform an increasingly corrupt State, and thus ultimately doomed to be swept aside by those who are not afraid to use the State for their own purposes. This is an inevitable result of the rejection of politics, and is indicative of why conservatives and libertarians so consistently lose ground politically, in spite of many aparent political victories (that ultimately prove to be illusory).
Unfortunately the social and political thinking and writings that are at the core of Western values and traditions are a closed book to most who take the libertine line of reasoning at face value. They simply are not aware that these things exist. And thus Western Civilization passes away, for wont of enough people who are consciously aware of the need to perpetuate it, and who are aware of the means of doing so.