Would like to see some documentation for this claim. Quite possibly accurate for the state of VA, but not for the South as a whole.
Statistics. Half of all slave owners held fewer than 5 slaves which tends to indicate that the average slave owner was not a plantation owner.