Radical Libertarianism is all about strawmen. It is a lot like the “zero tolerance” mentality that has become so common in our schools. No room for the exercise of judgment, by society, as to what behaviors create too great a danger to others. If it doesn’t directly harm another person, every time, radical Libertarians have zero tolerance for laws against it.
In principle I largely agree with sensible Libertarians, that generally laws should only be in place that have a legitimate purpose in protecting people from the actions of other people (not merely protecting people from themselves). However, society needs to be able to regulate behaviors that indirectly cause great harm to others.
We need to balance the benefit of the behaviors against the risk to innocent lives. Illegal drugs have no benefit to individuals and cause great harm to innocent people.
In other words, no room for Nanny Staters to exert unwarranted power over their neighbors.
Don't be afraid of it. Say it. "I want more government control over everyone's life."
You'll feel better once you admit it openly.