If they could climb those hills then they could climb the wall. But the wall has many watchtowers to spot invading troops and it would be a lot easier for defending troops to run along the wall than for invading troops to run up the hills.
Actually, the Mongols entered China with the help of a treasonous general who unlocked a set of entry gates. The Mongols were primarily horsemen and nomads who didn’t have the engineering ability of the Romans.
There are several stages of the Wall. The earliest stage was mostly masonry while later portions were tamped down straw and earth between masonry walls. It’s these later parts which eroded faster in the freeze/thaw cycles of winter and hot winds of summer.