Sure it is...
Every membership entity means that it's open to potential members...and potential members should know what why exclude them...if for no other reason than to never plan on joining.
I would say if you're a business and you supply products and services to such a golf course...and they have racist policies...then it literally IS your business...and such a business may desire to simply not sell its products or offer its services to such racists.
Who a company boycotts IS their business!
Likewise, a member may join a golf course and not realize it has racist policies. Such a criteria could (1) be offensive to such a member -- and they may wish to recant such a membership; (2) Even if they weren't personally offended, if they were someone with, say, political machinations who doesn't want their reputational name linked with such a course, they, too, have a right to know what the membership criteria is.
Otherwise you get into these occultic (Latin for "hidden") 30-odd degrees like the Masons...where they don't reveal to you what it takes to climb their ladder...and you get people in a sudden situation where they are making oaths that contain throat-slashings if you reveal certain things.
Agreed. Freedom of association. Walter Williams is all for it.
Government schools are like prisons, and the inmates of both behave the same.