Actually, it is well within questioning.
Your hypothetical brothel and/or biker bar bring up important points.
What is objectionable about a biker bar? Is it the noise? If so, there is no right to disturb the peace.
Is it drunken bikers wandering around on private property? They're not allowed on your property. The only property they're allowed on is the bar's and on the government's property (i.e. the street). In a free society, the government would not own the street - local property owners would and could decide not to allow bar patrons to cross it in order to get to this establishment.
Is it public drunkenness? Again, if you don't want drunks on your property (including now the road) they're trespassing if they do enter it.
Every single objectionable bit of behavior by the bar's patrons is actionable in a free society - i.e. it is a violation of property rights. The patrons would be quiet as churchmice and sober as judges.
Read that slowly. Consider and inwardly digest every word. Think very hard about all of the implications.
"The peace" is a community interest. If my activities on my private property "disturb the peace," then the community has a right to curtail my activities.
Case closed.