Yes, but in order to get Neanderthal genes into Cro-Magnon communities you need to have the Cro-Magnon woman get together with the Neanderthal man, and then return to the Cro-Magnon community to raise the child.
And, just looking at the cultural norms and sexual behavior of pre-urban, subsistence communities within the historical record, that scenario borders on the absurd. IMHO of course.
I think the insurmountable problem for a C-M/N hybrid was not surviving infancy but surviving the mating game later.
Presuming the C-M folks even knew that sex has consequences, they would probably not have been kind to the hybrid. A hybrid girl would have had the better odds, but even she would have faced horrendous odds at trying to keep HER baby alive. That's if she even reached sexual maturity before succumbing to the abuse she'd get. In other words, sex she'd get, regardless how ugly she was, and food she'd get, and protection from adults up to a point. But eventually the elders can't protect her and her contemporaries are dangerous---and her baby is not worth the resources nor under any one male's protection.
Add to ugliness a lack of eloquence, and her odds drop even lower. Speech was probably a big status symbol in those days.
(It still is, but you guys just don't want to believe it.)