No, in order for a new gene or set of genes to move to fixation, all members of the population that do *not* carry it and their offspring must be lost and replaced by new population members that carry it. No small feat. And the more differences, the higher the cost. Eventually, you just can't get here from there.
The only reason it is 'fascinating' is because you get to use your imagination to 'imagine' what may have happened.
But ultimately that's all it really is, 'imagining' unobserved 'events' that never really happened.
Actually, all the offspring of any population are eventually lost.
How about the cost of replacing the human race after all but five are lost? Natural isn't he only kind of selection.