At 33 degrees you get rain. At 32 degrees you get snow. A large accumulation of snow reflects sunlight and further lowers the temperature. The difference between "normal" and "ice age" is often as little as 7 degrees. The key issue is whether more snow falls and accumulates vs melting off. A steady accumulation, even at a slow rate, is the mark of an ice age.
It's not the coldness of the winter that makes ice ages, it's whether the summer is warm/long enough to melt the winters' accumulations.