FWIW, he said nothing about anyone going back to Africa.
Which is to say that's something you read into it.
For all I care, he might as well have said a one-way ticket to Antarctica.
But, hey. Don't let that stop you from playing the less-racist-than-thou card. I can imagine it makes you feel real good about yourself.
Does it make me feel better to call something wrong when it is? You're damn right it does.
Someday, your way of thinking will be extinguished. How does that make YOU feel?