This does sound fishy. If he really said those things he would be dead within minutes.
I can actually imagine him saying exactly that.
Suppose she actually was ranting at the woman in front of her, not just asking if she was waiting. Eventually, the woman gets upset and tells her to shut up, and the black woman calls her racist names, and the white woman responds.
The manager realizes the black woman is the one causing the problem, and asks her to leave.
Then, the black woman shouts at him, and at some point says “you’re just throwing me out because I am black, aren’t you”.
So he responds “Right, I’m throwing you out because you are black”. In other words, a sarcastic response to a woman who knew exactly why she was being thrown out.
This is of course all speculation. Maybe he was a racist, and maybe the other customer was racist.
I note that there is no quotes from other customers who were there at the time, and also no quotes from the white woman.