"Racist" and "racial" are not synonymous. Fox made a racist remark because he made a disparaging assumption based on race - namely, that if you're black, you have such low self-respect that you'll take any stinky old job that someone will wave in front of you.
"you have such low self-respect that you'll take any stinky old job that someone will wave in front of you."
The wrong assumption MANY PEOPLE make is that "any old job" is beneath them and hold those who work in such fields(pun) in contempt. "Any old honest work" is better than pimping and whoring porn and drugs.