It's a negotiation ploy, that (thankfully) Pres. Trump has made OK to do.
Think about it, these racebaiters only have the power that they're given. If they scream "RACIST!", and your response is, "I'm so sorry, what can I do to make this right?", you've ceded all of the power in the negotiation process to them.
Whereas, like Trump, if people scream "RACIST!", and your response is, "I don't care", then you've taken all of their power away from them. All that's left is for them to yell louder, and hope to goad you into a response, or a mistake.
BTW, that's where we're at on a national level, approximately, right now. Lots of sound and fury, signifying nothing.
“Hell, you’ll call me a racist for thinking algebra should be taught in school, and gifted programs should be kept even if they “lack diversity.” And so now, if I can be so bold, here’s my response: “So what? I get it. I’m a racist. Do you have anything else to say? Now that you’ve defined me as a racist, should I just disappear? Should I just admit that you are right, and come around to your way of thinking? What, exactly, do you want me to do? Because I still think about the issues affecting this country in the exact same way.”
http://www.theamericanconservative.com/dreher/he-doesnt-care-if-you-call-him-racist/
“I have been saying for some time now that if the alt-right grows in power and influence, it will be because ordinary people get tired of being bullied by these kinds of accusations, and choose to ally with people who might actually be bona fide racists, but who aren’t bothered by the attacks from the left.”