Images I've seen simply do not show that much difference between neanderthal and modern skeletons other than height. At least one article on the web describes the neanderthal as a 5'6" endomorph marginally different in build from modern people living in cold climates. That's basically a 5 foot six eskimo. How much of a chance would you give a five foot six eskimo in the ring with Rocky Marciano?
This particular article seems to imply that there was a degree of intermixing between early modern man and Neanderthal man in parts of Europe.
That contradicts everything I've read. One typical article describes modern man and neanderthals as last sharing a common ancestor about half a million years ago. Articles I've read describe neanderthal DNA as ape-like and not related to ours at all.