In follow up (from wiki):
While modern humans share some nuclear DNA with the extinct Neanderthals, the two species do not share any mitochondrial DNA,[139] which in primates is always maternally transmitted. This observation has prompted the hypothesis that whereas female humans interbreeding with male Neanderthals were able to generate fertile offspring, the progeny of female Neanderthals who mated with male humans were either rare, absent or sterile.[140]
I would posit female Neaderthals had no interest in weakling/stupid homo sapien sapiens and it was female homo sapien sapiens hooking up (or being captured/raped) that were “easy.”
It appears that Jean Auel's novel, 'The Clan of the Cave Bear', published in 1980, had it about right.