How is it racist? A ban on all immigration would include Europeans too.
It would be considered racist by most people since we know that most immigrants who would be affected are non-Europeans.