There were OTHER groups, though, and during interglacials they also moved across Eur-Asia.
It is difficult to imagine that they couldn't move to Africa, and back into Eur-Asia.
The groups in Africa may well be "ancestral" at some level, but at the same time they've been on the receiving end of a lot of East-West and North-South gene flow ~ just like happens today.
They've developed their own optimal types for living in the Tropics.