There should be a credit for white genes. The new genetic material diluted the all black African pool.
Those enslaved were the bottom of the African gene pool, being conquered and enslaved by stronger more vigorous people.
There are two schools of thought on the whole “race” thing.
One is that some races are differently (ie, more/less highly) evolved than others. This was the assertion in the book that the ACLU was trying to keep included in the classroom during the Scopes trial - that “Caucasians” were the most highly evolved, and blacks and aborigines less so.
The other school of thought is that all humans are from the same original gene pool (created, perfect Man and Woman), and that separation and natural selection gradually lost information until only a certain phenotype human was present in an area.
The first position I view as abhorrent, but either position would be supported by your claim that mixing in other gene pools was beneficial to genetic diversity.