Stone-age American Indians did this all the time. If 1000 Swedish women were dropped into a primitive tribal area, their DNA lines would not be lost. Simply put, I am saying that humans must have avoided interbreeding with Neanderthals for reasons that are much stronger than human-human activity indicates, or we would have some DNA trace of them. I'm not counting being furry, violently anti-social (like a wild animal), or such as "normal social exclusion". If you are, then we've reached a conclusion of our discussion.
What if the 1000 Swedish women were butt ugly, vicious, and more muscle-bound than the Stone Age Indian guys? What do you think would happen then?
Humans are of the group, that have few children and invest their lives in them.
Would not neanderthal do the same?