No, there are other grounds for laws. Nuisance laws ban loud noise, bright or strobe lights at night, nudity, lewd behaviour, etc.
Now you can make a case there there should be no nuisance laws of any kind. But what about a community’s right to make it’s own laws?
Community and local laws have always been around. Sure, some of them are silly ... like no vegetable gardens in front yards. But people have the opportunity to change those at the local level.
From the nuisance point of view, I can say that as a non-smoker, I find ANY smoking nearby to be a nuisance.
If I am outdoors and somebody is smoking 30 feet away, it is an instant irritation. At that distance it’s not as bad as some of the idiots who play pounding music in traffic, but it is still a significant irritation and annoyance.