My guess is that the white SAs are working circles around the black folks, and not smoking dope behind the haystack, not stealing anything that is not tied down, not constantly carping and complaining, not constantly asking for an advance because they done spent their paycheck on a good time instead of the rent, not skipping out on work because they are “sick” - in other words, the white folks just plain work harder, thus the farmers are willing to pay more.
You left out "black time", i.e. showing up for a work day whenever.