I'm more anti-pot than pro-pot, but let's wait five years and see what happens in Colorado and Washington. More states will probably follow. I never believed pot was totally harmless like the enthusiasts claimed, but it's just too early to make any hard and fast pronouncements about the effects on the general population.
We already did this experiment. Pot criminalization phased in pretty slowly, with Alaska keeping it legal until the 90s. There were no effects on the general populace.