I'm Jewish, and most Jews that I know don't interpret the Bible literally.
I am Christian and the tree of life and the tree of the knowledge of good and evil are NOT literal trees. So I would have to agree in part not all things are literal. These trees are used figuratively because one can study the horticulture of a tree, how it functions to grasp the point being made.
Maybe you need a wider circle of friends...
Did Moses REALLY strike a rock to get water out?
Did God REALLY rain down fire and brimstone on Sodom?
Did His finger REALLY write on stone tablets and a wall?