That problem has always been, when someone else says, "I do not wish to associate with X." X being the pot-head, the drunk, the sexual deviant, etc. It is one thing for you to sit in your home and ruin your life, it is quite another for you to enter another man's neighborhood, home, or business.
It is then that the argument goes from, "You can't tell me what I can or cannot do," to, "I'm telling you what you can or cannot do," bringing the litany of "discrimination" laws. You must accept potheads, you must accept drunks, you must accept perverts, and if you don't, you will be in trouble.
Certainly, you have a valid argument. But, that argument is not against me or what I believe.
We SHARE this point of view...that NO!...our children should not be forced to accept homos...and NO, we should not be forced to pay for the medical treatment of drug addicts.
However, NO, we do not have the right to tell someone what they can, or cannot, do when there's no harm to another.
Does your religion support that point of view? Does your politics?