I am certain that my experiences are not unique. People I know who were active and then started smoking pot, typically become much more sedentary. It goes hand in hand.
The liberty-respecting answer is not to ban pot - no more than any of the many other entertainments that dispose one to being sedentary - but to require consumer warnings.