Global Warming Bombshell: A prime piece of evidence ... turns out to be artifact of poor math

Technology Review ^ | October 15, 2004 | Richard Muller

[Steve Schulin]

Global Warming Bombshell: A prime piece of evidence linking human activity to climate change turns out to be an artifact of poor mathematics.

by Richard Muller (prof physics - University of California, Berkeley)

Progress in science is sometimes made by great discoveries. But science also advances when we learn that something we believed to be true isn’t. When solving a jigsaw puzzle, the solution can sometimes be stymied by the fact that a wrong piece has been wedged in a key place.

In the scientific and political debate over global warming, the latest wrong piece may be the “hockey stick,” the famous plot (shown below), published by University of Massachusetts geoscientist Michael Mann and colleagues. This plot purports to show that we are now experiencing the warmest climate in a millennium, and that the earth, after remaining cool for centuries during the medieval era, suddenly began to heat up about 100 years ago--just at the time that the burning of coal and oil led to an increase in atmospheric levels of carbon dioxide.

I talked about this at length in my December 2003 column. Unfortunately, discussion of this plot has been so polluted by political and activist frenzy that it is hard to dig into it to reach the science. My earlier column was largely a plea to let science proceed unmolested. Unfortunately, the very importance of the issue has made careful science difficult to pursue.

But now a shock: independent Canadian scientists Stephen McIntyre and Ross McKitrick have uncovered a fundamental mathematical flaw in the computer program that was used to produce the hockey stick. In his original publications of the stick, Mann purported to use a standard method known as principal component analysis, or PCA, to find the dominant features in a set of more than 70 different climate records.

But it wasn’t so. McIntyre and McKitrick obtained part of the program that Mann used, and they found serious problems. Not only does the program not do conventional PCA, but it handles data normalization in a way that can only be described as mistaken.

Now comes the real shocker. This improper normalization procedure tends to emphasize any data that do have the hockey stick shape, and to suppress all data that do not. To demonstrate this effect, McIntyre and McKitrick created some meaningless test data that had, on average, no trends. This method of generating random data is called “Monte Carlo” analysis, after the famous casino, and it is widely used in statistical analysis to test procedures. When McIntyre and McKitrick fed these random data into the Mann procedure, out popped a hockey stick shape!

That discovery hit me like a bombshell, and I suspect it is having the same effect on many others. Suddenly the hockey stick, the poster-child of the global warming community, turns out to be an artifact of poor mathematics. How could it happen? What is going on? Let me digress into a short technical discussion of how this incredible error took place.

In PCA and similar techniques, each of the (in this case, typically 70) different data sets have their averages subtracted (so they have a mean of zero), and then are multiplied by a number to make their average around that mean to be equal to one; in technical jargon, we say that each data set is normalized to zero mean and unit variance. In standard PCA, each data set is normalized over its complete data period; for the global climate data that Mann used to create his hockey stick graph, this was the interval 1400-1980. But the computer program Mann used did not do that. Instead, it forced each data set to have zero mean for the time period 1902-1980, and to match the historical records for this interval. This is the time when the historical temperature is well known, so this procedure does guarantee the most accurate temperature scale. But it completely screws up PCA. PCA is mostly concerned with the data sets that have high variance, and the Mann normalization procedure tends to give very high variance to any data set with a hockey stick shape. (Such data sets have zero mean only over the 1902-1980 period, not over the longer 1400-1980 period.)

The net result: the “principal component” will have a hockey stick shape even if most of the data do not.

McIntyre and McKitrick sent their detailed analysis to Nature magazine for publication, and it was extensively refereed. But their paper was finally rejected. In frustration, McIntyre and McKitrick put the entire record of their submission and the referee reports on a Web page for all to see. If you look, you’ll see that McIntyre and McKitrick have found numerous other problems with the Mann analysis. I emphasize the bug in their PCA program simply because it is so blatant and so easy to understand. Apparently, Mann and his colleagues never tested their program with the standard Monte Carlo approach, or they would have discovered the error themselves. Other and different criticisms of the hockey stick are emerging (see, for example, the paper by Hans von Storch and colleagues in the September 30 issue of Science).

Some people may complain that McIntyre and McKitrick did not publish their results in a refereed journal. That is true--but not for lack of trying. Moreover, the paper was refereed--and even better, the referee reports are there for us to read. McIntyre and McKitrick’s only failure was in not convincing Nature that the paper was important enough to publish.

How does this bombshell affect what we think about global warming?

It certainly does not negate the threat of a long-term global temperature increase. In fact, McIntyre and McKitrick are careful to point out that it is hard to draw conclusions from these data, even with their corrections. Did medieval global warming take place? Last month the consensus was that it did not; now the correct answer is that nobody really knows. Uncovering errors in the Mann analysis doesn’t settle the debate; it just reopens it. We now know less about the history of climate, and its natural fluctuations over century-scale time frames, than we thought we knew.

If you are concerned about global warming (as I am) and think that human-created carbon dioxide may contribute (as I do), then you still should agree that we are much better off having broken the hockey stick. Misinformation can do real harm, because it distorts predictions. Suppose, for example, that future measurements in the years 2005-2015 show a clear and distinct global cooling trend. (It could happen.) If we mistakenly took the hockey stick seriously--that is, if we believed that natural fluctuations in climate are small--then we might conclude (mistakenly) that the cooling could not be a natural occurrence. And that might lead in turn to the mistaken conclusion that global warming predictions are a lot of hooey. If, on the other hand, we reject the hockey stick, and recognize that natural fluctuations can be large, then we will not be misled by a few years of random cooling.

A phony hockey stick is more dangerous than a broken one -- if we know it is broken. It is our responsibility as scientists to look at the data in an unbiased way, and draw whatever conclusions follow. When we discover a mistake, we admit it, learn from it, and perhaps discover once again the value of caution.

Richard A. Muller, a 1982 MacArthur Fellow, is a physics professor at the University of California, Berkeley, where he teaches a course called “Physics for Future Presidents.” Since 1972, he has been a Jason consultant on U.S. national security

[Izzy Dunne]

published by University of Massachusetts geoscientist Michael Mann and colleagues.

It's all a misunderstanding !!

What they MEANT to say was:
It's a fact that global warming is caused by Mann."

[MonroeDNA]

Located here:

http://www.uoguelph.ca/~rmckitri/research/trcres.html

[Fresh Wind]

Junk science. Garbage in, garbage out.

Junk science. Garbage in, research grants out.

[Steve Schulin]

Thanks for showing the Mann et al. hockey stick. McIntyre and McKitrick tried to get the Mann et al. method to show other than hockey stick, but it could not do so. Here's their Figure 2, which compares the effect of using conventional principal component analysis (top graph) versus using the Mann et al. method (bottom graph) in one experiment. The authors report that Mann et al. method produced the same distinctive hockey-stick-shaped graph for all ten experiments they performed.

Here are some other recent criticisms of the hockey stick which apply not just to Mann et al., but to other multiproxy studies too:

* In the body of the UN IPCC 2001 science working group assessment report, it was noted that if our best understanding of glacier retreat evidence is on track, the warming trend started earlier than depicted by the hockey stick. This would spread out the recent warming currently restricted to the 20th century part of the curve. Here's the frank statement in the body of the TAR: "...the timing of the onset of glacier retreat implies that a significant global warming is likely to have started not later than the mid-19th century. This conflicts with the Jones et al. (2001) global land instrumental temperature data (Figure 2.1), and the combined hemispheric and global land and marine data (Figure 2.7), where clear warming is not seen until the beginning of the 20th century. This conclusion also conflicts with some (but not all) of the palaeo-temperature reconstructions in Figure 2.21, Section 2.3 , where clear warming, e.g., in the Mann et al. (1999) Northern Hemisphere series, starts at about the same time as in the Jones et al. (2001) data. These discrepancies are currently unexplained."

* In his 2004 paper, Craig Loehle showed that the multiproxy method inherently tends to produce flatter shape due to smearing. Each proxy record has dating error. By combining the proxies, the multiproxy approach inherently tends to smear variability out of the record [Ref: Loehle. Using Historical Climate Data to Evaluate Climate Trends: Issues of Statistical Inference. Energy & Environment, 15(1):1-10, 2004]. This was a meaningful challenge to the flat shape of the curve, quite independent from the McIntyre and McKitrick point about the Mann et al. PCA technique.

* In their Science paper released this month, von Storch et al. showed that the multiproxy study error ranges do not include a significant uncertainty. This was a meaningful challenge to the gray area of the curve. [Ref: http://www.sciencemag.org/cgi/content/abstract/1096109v1 -- von Storch et al. Reconstructing Past Climate from Noisy Data. ScienceExpress, October 1, 2004]

[Semper Paratus]

There's more evidence of WMDs in Iraq then there is of Global Warming.

[js1138]

Maybe it's a horse hockey stick.

[Arrowhead1952]

Similar thread here

[grey_whiskers]

Is the faulty computer program availabile on the web or elsewhere?

The obvious study is to take their program and systematically vary the length of the time period over which the normalization is performed, and compare the resultant predictions.

Somehow, it would nor surprise me to find that they
'accidentally' (of course) happened to choose the one
time period for the normalization procedure which would
maximize the appearance of global warming in the late 20th
century and beyond.

[aculeus]

Big bump!

[RadioAstronomer]

I NEVER believed in "global warming". Thirty years ago the hue and cry was "global cooling".

Yup! I remember those dire predictions. Don't forget we would all be starving by 2010 as well.

[jonno]

Global Warming bump

[Gothmog]

Great post, thanks

[Pagey]

A "Save this in your Global Warming 'Word' File Later" BUMP!

[longshadow]

Yup! I remember those dire predictions. Don't forget we would all be starving by 2010 as well.

I'm always on the lookout for those ubiquitous late '60's Volvos sporting "End Global Cooling!" bumperstickers.....

[dfwgator]

"now the correct answer is that nobody really knows"

Translation: We need more funding.

[American Sovereignty Defender]

American Hockey enthusiasts were hoping for better news, like an inverted "hockey stick". Cooler makes for better outdoor practice.

[Steve Schulin]

The fortran code Mann et al 1998 used to prepare the tree ring data for PCA is available at:

ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/ITRDB/NOAMER/pca-noamer.f

As for the PCA itself, Mann et al just say it's conventional, without mentioning any particular software.

[grey_whiskers]

The fortran code Mann et al 1998 used to prepare the tree ring data for PCA is available at:

ftp://holocene.evsc.virginia.edu/pub/MBH98/TREE/ITRDB/NOAMER/pca-noamer.f

As for the PCA itself, Mann et al just say it's conventional, without mentioning any particular software.

G'rrr. It figures. Well, at least I can download the code and make it run twice as fast ;-)

[farmfriend]

Rights, farms, environment ping.
Let me know if you wish to be added or removed from this list.
I don't get offended if you want to be removed.

[knews_hound]

Interesting read, thanks for the ping.

The graphics and references were a real plus.

Cheers,

knews hound