Have you considered that traditionally, Uslam emerged with Ishmael and Esau, and this inbreeding has highly concentrated the bloodline if these two?
...and genetic drift has obviously had a bad effect. My guess is marriage was probably closer than cousins for much of the Ishmael/Esau history. Their family trees probably only have a trunk with no branches.