I share your sentiments, but you shouldn't be surprised if many here do not. For many nowadays, dressing like a slob, and/or declining to "dress up," no matter the occasion, is viewed somehow as integral to their "personal freedom."
I'm not too impressed with "conservative" men who won't take off their hats indoors, even when sitting down to eat, but maybe that's just me being an old fogey.