You’re absolutely correct. His comment was stereotyping folks, but there was no racism indicated. The word racism is thrown about so much these days that the meaning is actually changing before our very eyes.
There is a difference between a vote and an explanation.