No - if a conservative called a bunch of people (of various races) a bunch of neanderthals, and a liberal claimed that he just called the blacks and hispanics neanderthals, I would then conclude that the liberal was the one making a racist statement.
I am consistent.