I work with risk averse companies where they control access to the ladies' room in particular by a combination lock on the door or by key. It's an issue with visitors at meetings but it works.
Some of these companies are mandated to lock the door to the ladies' room by the landlord who also doesn't want to get dragged into a lawsuit by an assault victim.
These companies aren't even dealing with public-facing bathrooms. They're in buildings where you already have to get past a security desk and the bathrooms are separated by tenant, not communal.
Don't be so sure. Courts, the law, and corporations are siding with trannies. If you're against trannies, you're a "hater," and courts generally rule against "haters."