There's been a few major disasters in our past..
They've killed off major chunks of the world population..
And then there's the plagues.. some major die-offs there as well..
I'm guessing.... ( I admit it, I'm guessing )
There were larger populations in the past than that chart indicates..
It just shows what was left after the disasters at various points in our history..
This may be exaggerated, or overblown, but I seem to remember that the last great plague, in the 13th or 14th century killed off something like 1/3 of the population of europe.. and did similar damage in other parts of the world..
Add those deaths / losses as you move back in time, and the population in ancient times becomes larger..
Remember that 74,000 years ago the Toba explosion left (alive) only 2-5,000 humans worldwide.
There was also a huge genetic split at that time and produced many more 'branches' on the human tree. This occured again about 18k years ago during the Last Glacial Maximum (LGM).