Don’t have to look at fossils.
Moving glaciers leave traces in the geological record (morraine).
We’ve had 25% of the time of the past 500 million years in ice age conditions (including today).
That means 75% of the time, it’s generally warmer.
Since that’s the majority, you could only conclude that warmer is normal.
North America was once covered by a glacier!