Antarctica was not always where it is today. There's a thing called plate tectonics, where the continents shift position over geologic time
Over geologic time spans, a lot of climate change is caused by continents drifting in ways that affect ocean convection currents. When warm currents are prevented from flowing to polar regions, you get massive ice ages
The variability of the sun also can cause major climate change. See for example, the Maunder Minimum and it's correlation to the Little Ice Age 16451715