In a reading of the book THE WASHING OF THE SPEARS it appears South Africa only had a few bushmen and Hottentots when the Dutch and English settled there. All the blacks were still hundreds of miles to the North East.
The land was virtually barren when the Dutch settlers arrived.