If the universe is infinite, and if there was a Big Bang radiating from one point, and if stars ultimately burn out, then you would have to think that finding the oldest galaxy is unattainable. It would no longer exist.
Now, yes, but the light travels slowly.............
Now, dead stars create new stars....................
The light we are NOW seeing has been traveling for millions or billions of years just to get here. We may be seeing it now but we are seeing it as it was THEN—not in the present.
More than a few of the stars we see each night are likely no longer in existence.