Just a bit of humble not an attorney advice. Defend yourself! I do not know what your contract read but the school must go by the written terms of your contract. Now even if it states that you can be dismissed without cause blah, blah.. I believe it is essential that you fight to keep your reputation.
The place to start would be to consult an attorney but that gets $$$ and may not be practical. At the very least write an affidavit recounting exactly what happened.
And no your are not a racist.
Perhaps the true solution is to shrug your shoulders and not care if you’re called a racist. If more people did that the “race card” would disappear. Frauds like Zero, Sharpton, Abrams, Gillum, Jackson (Is he even still alive? He has been very quite!), etc. would have no leverage over you. One could even practice MLKs dream of using individual character as your guide.