There is much in what you say. It may well be that a superior brain size of Neanderthals conferred little advantage because they were stymied by an inability to articulate an advanced language well.
Only when a gene for advanced brain size got somehow correlated with other genes for brain complexity, and language skills and vocal structures could societal skills be properly passed on to the next generations.
If we were but a single lineage back through the mists of history to the dawn of creation, there would be little reason for various blood types. Yet we share some blood types with gorillas and chimpanzees, as well as much of the organizational structure of the Y chromosome.
Clearly nature competes, even within a successful species, for favorable characteristics, which may have been holdovers from very long ago.