That is ultimately how it will work out and what the left is aiming for. But it remains to be seen that that kind of invasion of privacy will be tolerated. If it were not a serious privacy issue then the transgenders would not be so determined to use the ladies’ room in the first place.
There is also the fact that, on average, ladies inappropriately approached are less likely to separate the perv’s teeth from their gums than are similarly outraged men.