That, concisely stated, is the point of the argument. Young Earthers, myself included, are sola scriptura. Old Earthers take extra Biblical sources to inform their interpretation of scripture.
We are free to believe what we want, and must inform those opinions with some source. What gets me, though, is that Old Earthers try to reach conclusions that are at odds with the plain, obvious reading of scripture. To wit:
Exodus 20: 11 For in six days the Lord made the heavens and the earth, the sea, and all that is in them, but he rested on the seventh day. Therefore the Lord blessed the Sabbath day and made it holy.
There is no reason whatsoever to interpret this means anything other than 6 solar days, except the introduction of the assertions of the big bang and macro-evolution, and it's required time frames, and the ages reached by some dating methods. What's not so obvious to Old Earthers, for reasons that escape me, are the myriad other scriptural compromises they must then force.
For example:
1. If God really used evolution of millions of years, then that means He used mutation, suffering, and death to reach the level of sophisticated life forms we, and Adam, observe(d). How then could God call this creation "good"?
2. If death existed prior to the fall, why does Romans say death entered by Adam, and that all creations is under a curse as a result.
3. The days of creation are all out of order, and do not fit with the aforementioned extra-Biblical theories, e.g. Earth created as sphere of water, and dry land appeared 2 days later. Earth created before the sun, moon and stars. Birds were created before land animals.
In conclusion, believe what you want, but be aware of the baggage that comes along with it. And don't compromise scripture to fit current pet theories that will be different in 10 years!
Just go ahead and throw away Genesis, and be done with it.
Perhaps you can explain how plants lived “ages” without the sun. 6 days. Period. That is the Bible.