Science is by definition materialist, since it doesn't deal with spiritual mumbo-jumbo. As for the past being unobservable, I would suggest you go on a fossil dig some time, but maybe you would just think fossils are God's way of testing your faith, as they believed in the middle ages.
I would hazard to guess that you don't live in a cave or a monastery, so your dedication to spirituality over materialism is honored more in the breach than the observance.
"Religion... is a smile on a dog."