“It doesn’t really matter.”
Well actually brother it does. With all kindness in what I am saying, understand that if we can’t take all of Genesis literally, then how can we take the Garden of Eden situation literally which is essentially to the idea of original sin for which the Lord Jesus Christ had to be the second Adam without sin so he could be the bearer of our sins as described in the Bible. I tell you in great humility that it is indeed important.
My bias is to take Genesis overwhelmingly literally.
However, on issues of Scripture about which reasonable people can differ . . . I have not found those areas to be
CENTRAL to my relationship with God nor CENTRAL to my relationship with those who Love God unsurpassingly.
My youngest brother described the evolution (if you'll pardon the word) of his faith in almost exactly the terms you described. Once he had recognized Jesus as his saviour, then it led logically, just as you described, to embracing the facts which account for Jesus' life on earth. Without the Original Sin committed specifically by Adam and Eve, there would be no mission on earth for Jesus.
One can see how this interconnectedness of ideas could lead to a uniformity in Christian thought and how the Bible would have a secure and literal place in the teaching of Christianity. If one does not have faith in all of the teachings, then it just wouldn't make sense.