Then restrooms that are for one person, gender neutral is a solution. Public restrooms present a social situation where personal privacy is violated to varying degrees. Many people, even straight people feel a certain degree of discomfort using them.
That is ultimately how it will work out and what the left is aiming for. But it remains to be seen that that kind of invasion of privacy will be tolerated. If it were not a serious privacy issue then the transgenders would not be so determined to use the ladies’ room in the first place.