If the good people of Lincoln considered his behavior disruptive then they have every right to escort him to the door. And if the good people of Lincoln considered that a Black man should not sit in the front of the bus...? Or that the few Jews in town should move somewhere else...? They have the right to escort them out?
Get back to me when blacks or Jews are equated to 'behavioral minorities' such as perverts.