That of course flies in the face of the 1st Amendment guarantee of freedom of association.
Or it protects it. I remember Jim Crow. Some of my friends still have Jim Crow-era bathrooms.
If the Civil Rights Act was unconstitutional, it would be gone by now.