A good reason to keep pot illegal is America is a messed up welfare state, and have thrown away moral absolutes. Universal legalization would have such destructive impact, that not even the Liber(al)tarians would like it. They can make believe that current usage would stay the same, but they would be wrong.
You make a good point. Our relatively recently established history of NOT holding individuals responsible for the end results of their life-choices does make it difficult to change to a policy of “freedom with responsibility”.
By not holding up the responsibility side of the equation, we make freedom almost meaningless.
However, this does not mean that we should not work our society closer to “freedom with responsibility”.