I believe we once had a more diverse gene pool and we would have children that often didn’t look like us. Similar to cat or puppy litters. In time, those who looked similar were more familiar or attracted to each other and those traits became dominate and repeated, distinctive groups formed.
There is some circumstantial evidence in the Bible where parents would have several children and one would be in the same image.
What would have happened to the descendants of Ham had they moved north?
Natural selection of genetic variation would favor his lighter skin descendants - of course.