That's because "many people" fail to take into account why a process which includes variation, reproduction, and selection is very different from erosion, and is capable of much more.
If you're going to make an analogy, make sure it's a valid comparison.