Good point, I was taught in biology long ago that only members of the same species can produce fertile offspring. Two species may be able to produce offspring called hybrids but there is no third generation, the mule is the classic example, there are stories of mules giving birth to live offspring but if it happened there apparently was never a fourth generation. To say that Neanderthals passed down genes to the present day is to say that Neanderthals did not “interbreed with modern humans” it is to say that Neanderthals WERE modern humans, otherwise the offspring would have been infertile hybrids and no genes would have been passed down to the present. Nobody seems to have any desire to get the story straight these days.
Maybe it's like languages where there are lots of local dialects--people in one village would be able to understand the people in nearby villages but not those a long distance away, even if there was no sharp break anywhere. I understand that was the case with the Germanic dialects (not sure if it is still the case today with more people speaking the standard national languages). People on either side of the Dutch/German border could understand each other even though standard Dutch is definitely a separate language from standard German. Supposedly you could go all the way from the North Sea coast of Holland to the German-speaking part of Switzerland and people in any area would be able to understand the nearby dialects, but when you are dealing with people a long distance away they can't understand each other. (Unless, of course, they both speak English.)