Actually slavery in the ancient world was highly correlated with ethnic differences, because the largest source of slaves was captives taken in war and warfare was waged (setting aside the Roman empire) by largely homogenous ethnicities or tribes against neighboring peoples of a different ethnicity or tribe (albeit they may be of similar shades of skin color).
Tribalism describes the ancient world (again setting aside Roman civilization, which was a big step forward in human societies) better than racism, but one shouldn't pretend that tribalism was any better than racism or differed much from it in any essential way. To take a more modern example, the hatred between Hutus and Tutsis might not be considered racism by many people because both peoples happen to be black-skinned, but the tribalism at work there is essentially no different from racism.
It was correlated with ethnicity as you note, but it was not ethnically driven as it was in the Americas.
There were African Roman Emperors and African slaves and Germanic slaves and Germanic soldiery in the Empire.
Greek city states might be just as likely to enslave other Greeks as they would enslave non-Greeks.
In the world of the western 1700’s and early 1800’s, slavery was essentially race related. No?