There is a white black and hispanic working class. They're being displaced, moving away.
My point about the localities is the "cost" of getting workers is cheaper if you can pick them up off the corner rather than go to another city or state and advertise. I think things will be fine. I do think southwestern farming needs foreigners proven by the need to use them over many decades.
Here is another little angle. Is it possible that some of the folks in these localities prefer an Hispanic working class to a black one? That is a sleeper issue. Working class blacks are being forced out of due to economics, and moving away from, heavily Hispanic areas. In LA county, the most heavily black census tracks are now middle to upper middle class. Who would have imagined that 30 years ago?