Because THAT logic would fail the Bennett morals test.
What if you're a coke or heroin addict? If you're still able to support your family, and it's not causing you any financial trouble, what's the fuss? Well, Bennett would find quite a fuss in that.
Your desire to stay out of Bennett's personal business is anathema to Bennett's raison d'etre, which is to chide people based on their MORAL decisions.
And if gambling weren't 'immoral', why is it illegal in so many states? Because of the 'addictive' nature of gambling.
For pure moralists like Bennett, it doesn't matter whether or not you're CURRENTLY hurting those around you. The 'evil' or immorality will catch up with you in the long run, so you have to avoid those moral pitfalls ON PRINCIPLE, not based upon your ability to weather the immoral storm because you've got lots of moral bucks in your pocket.
Nice straw man argument. You must be a libertarian. You compare a legal pasttime, that has no health implications, to an activity that is illegal and can cause many health problems, including death. Using this false analogy, you try to justify the activity with real legal and moral implications. But it doesn't wash.