It’s weakening the structure of the ground.
Understand that the fracking is taking place 8-12,000 ft underground --there is still 1 1/2 to 2 miles of solid rock above the site.
Moreover, fracking does not create large underground caverns. Generally speaking, it's designed to fracture the rock in the immediate neighborhood and hydraulically inject grains of sand into the fissures so that the cracks remain open. It's all very local -- immediately adjacent to the drillstem. There is no large void.
After over sixty years of use with no ill effects noted, there is simply no logical reason to ban fracking.
There is, however, a political reason -- as Bernie cites; the climate change hoax.