You mean blacks have turned the term into a negative racial stereotype.
No, I meant Yankees! I've heard it used by as many White persons as Black and a Black person from the South knows that calling me a Cracker isn't as bad as calling me late for dinner.
It doesn't raise hackles, stir cockles or get any dander up for me at all.