The "hockey stick" graph is complete nonsense. It's the kind of thing you can get when you look at a tiny slice of data. Over the last 600 million years the Earth's average temperature has fluctuated from a low of about 53 degrees (F) to a high of about 74. Currently it's about 59.
When you look at graphs covering longer and longer periods of time, starting from your "hockey stick" time period, you see all kinds of interesting patterns.
Over shorter periods of time (a few centuries) the fluctuations seem to be controlled largely by changes in the amount of heat put out by the Sun.
Over tens of thousands of years, the cycles seem to be tied more to the Earth's orbital dynamics (precession, changes in the eccentricity of Earth's orbit, etc.).
Over tens of millions of years, it seems to be continental drift that is most important. Right now, we have a continent covering the South Pole and ocean with little latitudinal circulation at the North Pole. this makes the Earth cold, and has us locked in a series of ice ages.
The hockey stick refers to the reent rate of increase and IF it continues it will be a problem.
The fact that it was warmer a million years ago doesn't help us much - we weren't trying to sustain a civilization then.