Since you started it (nearly two years ago), a smaller, less robust infrastructure makes it easier to destroy, and harder to rebuild.
Look at Larry Niven's "Ring World". That world had a power failure caused by a bug eating the organic control wiring, and they were unable to recover.
Read it one day in high school, between math classes.
It's what one would expect with a shrinking world population.
Just 2 generations ago, there were still enough people around to support such infrastructure easily.
Every year, more and more of our ancestors die, leaving fewer and fewer people. Simple powers of 2 regression:
Everyone has 2 parents; and 4 grandparents, etc. At that rate, it doesn't take long to see that it wasn't all that long ago that we had 100,000,000,000+ population.
Given the shear numbers, it's no wonder they were able to hand stack piles of rock that we can't even begin to duplicate with machinery.