You're dead wrong!
For the most part the concern is principled and it's not about immigrants, not even about immigration.
The concern is about unregulated immigration and the ability of our society to absorb the masses being encouraged to emmigrate.
It's about the concentration of immigrants in areas which have neither the resources nor infrastructure to absorb these masses. It's about the consequent multicutural nightmares that arise when absorbtion is not possible.
Both the society and the immigrant are harmed. The American culture is not preserved and the immigrant is denied the American dream because his presence has erased the culture and thereby the possiblity of achieveing the dream. The immigrant has not escaped but simply drug his cultural limitations with him.
Now I understand. It's not about immigration.