Taking another example - the Indian sub-continent's population in 1900 was 300 million odd (that's India+pakistan+bangladesh). Today that is closer to 1.7 billion -- 6 times growth. And the birth rate tehre is reduced to abotu 2.7 and falling in India and Bangladesh (in fact Bangladesh's drop is faster than India - its at replacement rate now and dropping) -- this drop is good.
You don't need to have everyone having 4 kids. If population levels leveled off at 1900 levels things can work out
Note that importing immigrants is not needed -- most jobs now are automated so you don't need so many people as before.
Good point, although it does rather leave the elderly in a bind.