I’ve known a number of trans-people since the early 80s. None of them ever had an issue with which bathroom to use. If they “looked like a man” that day they used the men’s room. If they “looked like a woman”....
And they didn’t get harassed that I’m aware of. It wasn’t a frequent conversation but it has been brought up a number of times when I was present over the decades.
As a rule, if they don’t make an issue out of it, most other people don’t - and those who do will no matter what so don’t bother fighting it. Once they make an issue of it everyone picks a side, even those who didn’t have a problem before but do now.
After gay marriage became the law and generally accepted, LGBT advocate groups needed a new issue - can’t get donations or attention if there isn’t a “problem”. Suing a few bakers and photographers out of business wasn’t enough. So they jumped on this to polarize everyone again.
Part of the problem is the rise of “transgenders” who are biologically intact men living and identifying as women, while sexually attracted to women.
Thus you have demands that mentally ill biologically intact men (3/4 of trans are men, less than 1/4 have the male plumbing removed today) often sexually attracted to women be allowed in the bathroom, the changing room, the locker room, the girls’ dorm.
I don’t care if someone used to be a man and had the plumbing removed. That person can use the women’s bathroom. The biological man should not be in the women’s spaces segregated for their safety. The violation of women’s privacy is unacceptable, as is the risk that voyeurs and rapists will claim to be women to gain access is too high.
The danger goes up even more with the whole “gender fluid” - that you can be a man when in the hallway but be a woman when you want to shower next to girls.