The problem is the availability of welfare.
And other government money, which can be controlled once you have a "critical mass" of your own people.
If welfare, and other government money, was unavailable, then people would have to work, and in so doing would have an incentive to get along with all the people you need to get along with in order to work real jobs.