Unless I am mistaken, Neanderthals were not in our direct line of descent, not our ancestors. Distant cousins, perhaps, but not our own line.
They aren’t even the same species. Does that constitute ‘cousin’ when referring to animal relations? Is a wolf and dog the same species? Don’t know, just pondering.