The right to discriminate. Or, if you prefer, the right of free association.
I'm not absolutely sure of your meaning but I will say the following. Many words in our language have been perverted in our culture of PC correctness and doublespeak. I used to be a student of the English language many years ago. Discrimination implies the right to choose..nothing more. We should all have the right to choose our associations in our private lives.