It could be that the ruins really were ruined, i.e., that it was just foundations and some chunks of pillars. Rome itself was being dismantled even in imperial times, to the point that emperors from the 3rd century on were constantly making decrees and setting punishments for pulling parts off public buildings. Now and again some construction project turns up statues and pieces of statues from ancient Rome -- the pieces being all that is left of ancient works of art which were rendered into lime (for mortar) during the Middle Ages.
Horace quote follows, not the one I wanted:
"Time will bring to light whatever is hidden; it will cover up and conceal what is now shining in splendor."
Maybe this is it, but it doesn't sound quite right:
"I have reared a memorial more enduring than brass, and loftier than the regal structure of the pyramids, which neither the corroding shower nor the powerless north wind can destroy; no, not even unending years nor the flight of time itself. I shall not entirely die. The greater part of me shall escape oblivion." [Horace, referring to his own written work]
Could be.
But now we'll never know.