Yeah, but the millions who come here are, generally, planning to assimilate not to conquer and throw out the original inhabitants (i.e. us).
What's that Hispanic group in California that wants to do exactly that? There was a poll taken in Mexico and it turns out most Mexicans think the half the US rightfully belongs to Mexico.
I think Western Civilization has backed out of Africa because we realized their is no way we could coexist with the indigenous population and maintain civilized Western ideals. We simply don't have the stomach to do what we would need to do.