Let's be clear about something, if we may...
It is *not* a matter of being pro-drug. It is a matter of being anti-government.
It is none of the govenment's business what people do in their homes.
If they try to operate heavy equipment on an interstate highway while they're stoned, then yes. Not otherwise.
What libertarians ALWAYS miss is that when you allow some freedoms, and a great many people hurt others by abusing those freedoms, punishment after the fact does nothing to help those injured. In our town, we used to allow people to have pit bulls, until several of those who did caused others (including a child) to be severely injured. So the honest question is: well - continue to allow pit bulls in the town, knowing that some will abuse the freedom and that some kid will be maimed or killed OR ban pit bulls. The vast, vast majority in this town opted for banning pit bulls. Why? Because the likely injury to others is far more abhorrent to them than the taken-away freedom of wanna-be pitbull owners. Governments (and citizens) make these kinds of (real world) decisions all the time.