Well, that's a valid argument, and several pundits (I believe that T.A. Heppenheimer is one) have advanced it even further, saying that Great Works (such as the Pyramids) are actually the driving force of economic and technical advancement. I myself believe that large engineering feats are a waste of resources except in so far as they serve an economic or political purpose; the Pyramids may well fall into this category (impressing foreign vassal states into obedience, for example), but a starship would likely not. Opinion is by no means unanimous on this point, however.
Also, the limits of spacetime we now see as an insurmountable obstacle may not be so with future discoveries regarding the conjunction of space and time.
This I put in the category of "What if there's magic?" Certainly I can't rule it out, but if we're going to suppose that there is magic out there to solve problem x, then no amount of calculation or opinion serves any relevant purpose. All bets are off. The fact that we're discussing it at all means we are discussing it in the context of what we know.
Let's not forget that the biological limits we now recognize are on the verge of yielding to telomerase adjunct extensions, possibly netting nearly unlimited lifetimes.
I would never claim that we've reached the limits of material possibility, but nevertheless, such limits ultimately exist.
the universe is what it is, but our understanding is not yet complete.
The very existence of my profession is predicated on the truth of that statement.
The difficulty with this is that we can never know what might have been, had resources not been conscripted for such projects. If the US had "enjoyed" a centrally controlled economy for the past century, we would probably have several more projects like the Panama Canal and Hoover Dam, but -- for one obvious example -- would we now be communicating with personal computers over the internet? Egypt got its pyramids, but for all their glory, Egypt seems in retrospect to have been a remarkably stagnant society. So my guess is that the pyramids cost them more than they could ever realize. But as I said, we can never know.