I don’t disagree. I just think the distance is the issue. You just don’t go from one place to the next without invoking some physics.
Based on our current understanding of the world around us you are correct. What might our species know in the next 1,000, 10,000, or 100,000 years though? If we believe the universe (or universes) is billions of years old, another species may have a million or 2 million year head start on us. I for one am not ready to rule that possibility out.