Only if you let them. The "race" card, "hate" card or any other "victim" card only have power over those who buy into the concept of group guilt or group victimhood.
Coulter speaks for Coulter. Who cares if some Leftist demagogue wants to try to make you feel guilty for Coulter's words? You didn't say them. You either agree with them, in which case you should proudly throw it into the demagogue's face, or you don't, in which case you bear no association with them and no guilt.
I do care what she says..I was cheering her own a few months ago.