“what could you even charge them on?”
Criminal trespass.
Owner (or agent thereof) tells you to leave the property, you leave the property.
That it escalated to police detaining them for removal shows absolute intent to violate trespass laws.
If you pulled out criminal trespass (legit reason), how many folks would you have to call the cops on (per day)? Each person that came in and asked to use toilet, without buying a cup of coffee?
If anything, the lesson learned here is to shut down all public toilets, and make folks pay a buck each time (like they do in Europe now at train stations).