They actually cleared all that land in order to plant crops. Today we grow probably 100 times more crops on an acre of land then we did back 200 years ago, so the need for as much cleared land has gone down remarkebly. There have certainly been places where it was cleared for railroad ties and wood energy, and there is no doubt it did damage from which we recovered as soon as it was allowed to grow back. The problem is, the astronauts look at cleared land and automatically decide it is damage. That then gets passed on as gospel and we pass a lot of absurd laws that do nothing but hurt the poorest in our society. We now have more trees in the United States than we did 200 years ago, so the argument that clearing jungle timber is doing permanent damage to the environment is not proven. We need better data than just the observations of astronauts.
Soil under the rain forest is different, and more fragile, than that under our forests. Habitat destroyed when forests are cleared is not restored simply by growing new trees. If population keeps increasing the restorations due to better technology will be lost. You're certainly right when you say that every ill-considered comment by an astronaut is seized on by ignorant partisans to support their agendas.