If genetics is a factor, it may go back to natural selection operating in African environments centuries or thousands of years ago, long before the trans-Atlantic slave trade.
natural selection?....well, we must include the fact that the strongest male bred with how many females he could gather....the selection part was more of a forced relationship....