Personally, I think pot can mess you up worse.
I have been free from alcohol for thirty years, and just stay away from any kind of drug.
As a younger man, I was a heavy party child and didn't know how to do anything in moderation. So... I just avoid anything that might tempt me to do that again.