Fermi was an idiot. We literally have no way to know if there’s life out there. And we are largely undetectable to any life that may be out there. Less than 1/100 of 1% of the Milky Way has had any ability to detect humans (our radio and later TV transmissions). Fermi walked into an unlit room, didn’t turn the light on, and declared the city has no people.
We reverse this idea in what I'll call the Standard Model of Science Fiction -- the view that humans, were we able, would expand to occupy all habitable planets within reach (unless or until stopped by an alien counterforce). People will always seek new worlds to colonize, or to which to flee. (The big question is whether it is the Muslims who would stay, or go; or liberals vs. conservatives, etc. I think we would separate if we could ....)