Because THAT logic would fail the Bennett morals test.
What if you're a coke or heroin addict? If you're still able to support your family, and it's not causing you any financial trouble, what's the fuss? Well, Bennett would find quite a fuss in that.
Your desire to stay out of Bennett's personal business is anathema to Bennett's raison d'etre, which is to chide people based on their MORAL decisions.
And if gambling weren't 'immoral', why is it illegal in so many states? Because of the 'addictive' nature of gambling.
For pure moralists like Bennett, it doesn't matter whether or not you're CURRENTLY hurting those around you. The 'evil' or immorality will catch up with you in the long run, so you have to avoid those moral pitfalls ON PRINCIPLE, not based upon your ability to weather the immoral storm because you've got lots of moral bucks in your pocket.