Keep in mind that women are fertile into their 40's, and thus might have young children of their own alongside their own grandchildren.
Fathers have important skills to pass along. The ability to form a good spear head or arrow point from flint is a skill that takes a while to learn, as is the ability to know likely places to find flint deposits, the best way to take down a multi-ton mammoth, etc.
What happens when their mates die? And what are the chances that they'd be able to find another mate?
Assumption: early/pre humans were monogamous and an elder female would be able to find a new mate.
Seems the “tribe” would end up with a lot of elder females.