well, technically they were whites captured as slaves by whites and for whites.
The north African emirates had earlier been part of the Phoenician, Greek and Roman world for millenia. They were and are genetically very similar to southern Europeans.
And many of the slave traders were Europeans who went to the Moorish side.
well, technically they were whites captured as slaves by whites and for whites.
—
Technically, they were Christians captured by Muslims who continue the slave trade to this day.
Keep in mind that they didn't do much chattel slavery like was done here. When a Muslim man raped his sex slaves and had babies, the babies were typically treated like his kids and, therefore, as Muslims they were free. When those babies grew up and became men, they too had sex slaves and repeated the process.
Another feature is that many slave owners were required to set their slaves free if the slave announced he/she was Muslim. That too lightened up the population (former Christian white slaves became average Berbers if they denounced Christ and embraced Islam).
So maybe towards the end of the Barbary slave trade, the Berbers where white as you say. But did they start out quite a bit darker? Perhaps not as dark as Sub-Saharan Africa, but probably darker than the blondes that Muhammad said made the best sex slaves.