Rememeber, the first stars did not have planets with heavy elements circling them. All the heavier elements than helium and lithium were generated by the life cycle and death of stars. Followed by the dispersion of these elements to be gathered into clouds then galaxies that eventually generated stars and planets as we have now. In short, we may be at the leading edge of the process which can allow for the formation of conditions absolutely necessary for the rise of life and continued development thereof. How long does it take to destroy the first stars, collect the dust in clouds then generate galaxies? How long must the process run to reach a quiet enough state such as our planet has arisen in?
If you would like to read and listen to more along this line of reasoning, try linking to reasons.org an astrophysicists explains the notion much better than simple ol' me.
The assumption that life has been around elsewhere for 11 billion years, or 15 billion years, 4 billion years, or whatever, makes no difference to what I am saying. And that is that 1) the odds of our having been noticed, much less visited, by another civilization in the 70,000 or so years of human existence, or in the 107 years since we started generating signals that might be detected beyond the bounds of our own planet, are incredibly small, and 2) that there is still NO credible, concrete, scientifically verifiable and conclusive evidence proving the existence of aliens.
Sure, we could be the leading edge of the development of life in the universe, but that misses the point I am trying to make. Whether I allow for the possibility of life being able to develop any sooner than it has here or not doesn't change things in any significant way.