Cross-breeding of what??
If you cross a horse and a donkey you get a sterile mule. But some lion-tiger crosses are fertile. So are crosses between different "species" of snakes and fish.
What is a "species"? Are Neaderthals and Homo erectus and modern man really different species?
100,000 years seems like a long time, but from an evolutionary perspective, its not.
I agree that "modern man" whatever that is, originated in Africa. But I don't think you can pinpoint migrations from Africa to the rest of the world in one or two or three exclusive events. Successful dispersal of species does not occur that way. In all probability the three waves concept is a conservative one. My guess is there were numerous waves of migrants and that they were close enough to the Neanderthals to have interbred with them. I'm not so sure about the Homo erectus, but anything is possible.
Looking at reconstructions of Neaderthals and "modern man" as well as their skeletal remains, especially those of later Neanderthals makes them appear very similar.
Put a Neanderthal in a business suit, give him a shave, haircut, nail trim and bath, and he probably would like a very strong modern man.
Just put him in a tank shirt and sweat shorts and he could be a personal trainer at the gym.