Tell me Duck, when we start "removing people from civilized society" who gets to decide on what the list of "right choices" is.
"The free man owns himself. He can damage himself with either eating or drinking; he can ruin himself with gambling. If he does he is certainly a damn fool, and he might possibly be a damned soul; but if he may not, he is not a free man any more than a dog is."
--G. K. Chesterton
The majority, working within the framework of constitutional laws.
We, as a society, decide which rights we will protect, bearing in mind that protecting a right places a burden on society -- everything from enforcement and incarceration of those who violate your right to the consequences of your freedom to exercise that right.
We choose not to protect your right to do drugs. If and when a majority of the people decide that we should, then we will. Given that we're a self-governing nation, there's nothing to stop the majority from deciding this.
"Every society has a right to fix the fundamental principles of its association, and to say to all individuals, that if they contemplate pursuits beyond the limits of these principles and involving dangers which the society chooses to avoid, they must go somewhere else for their exercise; that we want no citizens, and still less ephemeral and pseudo-citizens, on such terms. We may exclude them from our territory, as we do persons infected with disease."
-- Thomas Jefferson to William H. Crawford, 1816
The elected legislature. We are a representative democracy and a nation of laws. Our representatives decide issues like speed limits, levels of flouride in our water, and many other issues. We do have a vested interest in assuring a civilized society that we can raise the next generation in
If someone wants to drop out of society for ever then he is free to do so. If someone chooses a life of self destruction within society, then society has a right to remove such a person once they start becoming a cancer on society. Skid row wasn't built by those who follow the rules, but we all have to suffer for the actions of those who choose to live in ways we deem inappropriate for a polite and civilized society.
As for your quote about the free man owning himself, well that's the position a secular progressive would take. In fact, I believe I've heard NARAL's Kate Michaelman use some form of that same quote.