What about the thousands of BLACK slave owners?
In the early days, white folks feared a "negro uprising" and sought to repatriate blacks back to Africa. A lot of black willingly sold themselves into slavery to avoid going back to the motherland.
And, let us not forget that the first order of business for former slaves repatriated to Africa (Liberia) was to enslave the indigenous blacks.
Again, not an endorsement of slavery. Not a justification for slavery, because blacks did it too. But, if we're going to discuss historical events, let's discuss them in context.