Do you have any alcohol in your house? Any mind-altering drugs in your medicine cabinet? Pain-relievers? Sleeping pills? Anti-depressants? Stimulants? Muscle-relaxants? Cold-medicine? Would your neighbors be morally justified in dragging you out of your house and kicking your ass because you consumed an intoxicant?
Just as leftist liberals like to blame guns for crime you like to blame drugs. It isn't the use of something that is a crime. It is the misuse.