I don’t think the libertarians cared about the taxes. They just didn’t want to spend money going after all those people.
Go back to the 1990s and read the old commentary that people used to write about why it ought to be legalized. Generally, everyone wanted to make it look like cigarette-like taxes and how the money would help someone (like in the south when we tell folks that casino-gambling or lotto profits within a state will help fund schools....it’s always a bogus statement but people like to hear it).
It's more than not wanting to spend the money. It's not wanting to go after these people at all. Not wanting vicious armies of ultra-armed police crashing down doors (sometimes the wrong door, oops) and confiscating property without convictions (sometimes the wrong property, oops).
I'm no libertarian and I don't like the idea of more pot or any drug being more available but I'm so sick of these gestapo (nazi reference on the internet, oops) tactics in our United States that I'm willing to see all drugs legalized, just to get rid of property forfeitures and ballooning SWAT mobilizations.