Hair color and eye color are also genetic. But we don't separate brunettes, blondes and red-heads into separate "races". But we do separate on the basis of how flat noses are, round eyes are and by skin color.
What would a sloth know about race?
That's because skin color breeds true. Eye color doesn't.