Doesn't Genesis 2 give a different order for Creation, though? How does that fit in?
This indicates to me that Genesis 2:4 through 3:24 are describing events concurrent to Creation week, but taking place in the Spiritual realm. In the end, Adam/Eve are banished to mortality, i.e. the physical realm.
Once that distinction of Spiritual v physical was made, all the other pieces seemed to fall into place easily - for me.