I was attacked in High school several times by Latinos and blacks. They could not fight one on one or mano y mano and they always fought in gangs. I never backed down and got hurt several times but I gave many as good as I got. Ironically, many of the ones that jumped me were 18-19 year olds still in high school and I was just 14 at the time. We moved to a less diverse school and it all stopped.
Unlike you, I didn't fight back physically. Being a girl, small in stature and the studious quiet sort, I'd never been taught to fight and didn't readily go at it. I got several of the girls suspended from school by taking my complaints to school officials. They didn't retaliate, and mostly left me alone after that. I don't think a guy could have done that - it would have been an invitation to further problems. Like you, I found some peace when I left the setting. My college classmates were completely unaware of these things, having lived in communities where these things just didn't happen.
Our sons are growing up in a community where things are more peaceful as well. We've taught them not to instigate any violence, but they know they'll never be punished at home for defending themselves - no matter what school officials say.