I’m assuming this was some sort of dress code violation (”he looked him up and down”), and not a racial thing, which probably would have been illegal. Many years ago, I was asked to leave a hoity-toity bar because I was wearing a hat, which I was told violated their dress code. I think they associated the wearing of hats indoors with gang culture or pimping.
If you ever served, you would know that anyone wearing a hat in an Officer’s Club had to buy the house a round.
No hat! Would they have turned away Frank Sinatra?
Or, you could have removed your hat. A friend was told to remove his hat or leave a bar on Worth Street in Palm Beach, Ta-Boo.