Of course a restaurant owner should be free to serve whites only. And you and I are perfectly free to not patronize such a restaurant. If all the meddlesome laws on that subject went away, there'd be precious few establishments actually operating that way, and the few that did would be small and financially struggling.
Nope. If you're in business, you serve who walks in the door, unless they violate your dress code.