Posted on 08/15/2010 11:10:08 AM PDT by Ernest_at_the_Beach
NOTE: this will be the top post at WUWT for a couple of days, see below for new stories – Anthony
Sticky Wicket – phrase, meaning: “A difficult situation”.
Oh, my. There is a new and important study on temperature proxy reconstructions (McShane and Wyner 2010) submitted into the Annals of Applied Statistics and is listed to be published in the next issue. According to Steve McIntyre, this is one of the “top statistical journals”. This paper is a direct and serious rebuttal to the proxy reconstructions of Mann. It seems watertight on the surface, because instead of trying to attack the proxy data quality issues, they assumed the proxy data was accurate for their purpose, then created a bayesian backcast method. Then, using the proxy data, they demonstrate it fails to reproduce the sharp 20th century uptick.
Now, there’s a new look to the familiar “hockey stick”.
Before:
Multiproxy reconstruction of Northern Hemisphere surface temperature variations over the past millennium (blue), along with 50-year average (black), a measure of the statistical uncertainty associated with the reconstruction (gray), and instrumental surface temperature data for the last 150 years (red), based on the work by Mann et al. (1999). This figure has sometimes been referred to as the hockey stick. Source: IPCC (2001).
After:
FIG 16. Backcast from Bayesian Model of Section 5. CRU Northern Hemisphere annual mean land temperature is given by the thin black line and a smoothed version is given by the thick black line. The forecast is given by the thin red line and a smoothed version is given by the thick red line. The model is fit on 1850-1998 AD and backcasts 998-1849 AD. The cyan region indicates uncertainty due to t, the green region indicates uncertainty due to β, and the gray region indicates total uncertainty.
Not only are the results stunning, but the paper is highly readable, written in a sensible style that most laymen can absorb, even if they don’t understand some of the finer points of bayesian and loess filters, or principal components. Not only that, this paper is a confirmation of McIntyre and McKitrick’s work, with a strong nod to Wegman. I highly recommend reading this and distributing this story widely.
Here’s the submitted paper:
A Statistical Analysis of Multiple Temperature Proxies: Are Reconstructions of Surface Temperatures Over the Last 1000 Years Reliable?
(PDF, 2.5 MB. Backup download available here: McShane and Wyner 2010 )
It states in its abstract:
We find that the proxies do not predict temperature significantly better than random series generated independently of temperature. Furthermore, various model specifications that perform similarly at predicting temperature produce extremely different historical backcasts. Finally, the proxies seem unable to forecast the high levels of and sharp run-up in temperature in the 1990s either in-sample or from contiguous holdout blocks, thus casting doubt on their ability to predict such phenomena if in fact they occurred several hundred years ago.
Here are some excerpts from the paper (emphasis in paragraphs mine):
This one shows that M&M hit the mark, because it is independent validation:
In other words, our model performs better when using highly autocorrelated
noise rather than proxies to ”predict” temperature. The real proxies are less predictive than our ”fake” data. While the Lasso generated reconstructions using the proxies are highly statistically significant compared to simple null models, they do not achieve statistical significance against sophisticated null models.
We are not the first to observe this effect. It was shown, in McIntyre
and McKitrick (2005a,c), that random sequences with complex local dependence
structures can predict temperatures. Their approach has been
roundly dismissed in the climate science literature:
To generate ”random” noise series, MM05c apply the full autoregressive structure of the real world proxy series. In this way, they in fact train their stochastic engine with significant (if not dominant) low frequency climate signal rather than purely non-climatic noise and its persistence. [Emphasis in original]
Ammann and Wahl (2007)
…
On the power of the proxy data to actually detect climate change:
This is disturbing: if a model cannot predict the occurrence of a sharp run-up in an out-of-sample block which is contiguous with the insample training set, then it seems highly unlikely that it has power to detect such levels or run-ups in the more distant past. It is even more discouraging when one recalls Figure 15: the model cannot capture the sharp run-up even in-sample. In sum, these results suggest that the ninety-three sequences that comprise the 1,000 year old proxy record simply lack power to detect a sharp increase in temperature. See Footnote 12
Footnote 12:
On the other hand, perhaps our model is unable to detect the high level of and sharp run-up in recent temperatures because anthropogenic factors have, for example, caused a regime change in the relation between temperatures and proxies. While this is certainly a consistent line of reasoning, it is also fraught with peril for, once one admits the possibility of regime changes in the instrumental period, it raises the question of whether such changes exist elsewhere over the past 1,000 years. Furthermore, it implies that up to half of the already short instrumental record is corrupted by anthropogenic factors, thus undermining paleoclimatology as a statistical enterprise.
…
FIG 15. In-sample Backcast from Bayesian Model of Section 5. CRU Northern Hemisphere annual mean land temperature is given by the thin black line and a smoothed version is given by the thick black line. The forecast is given by the thin red line and a smoothed version is given by the thick red line. The model is fit on 1850-1998 AD.
We plot the in-sample portion of this backcast (1850-1998 AD) in Figure 15. Not surprisingly, the model tracks CRU reasonably well because it is in-sample. However, despite the fact that the backcast is both in-sample and initialized with the high true temperatures from 1999 AD and 2000 AD, it still cannot capture either the high level of or the sharp run-up in temperatures of the 1990s. It is substantially biased low. That the model cannot capture run-up even in-sample does not portend well for its ability
to capture similar levels and run-ups if they exist out-of-sample.
…
Conclusion.
Research on multi-proxy temperature reconstructions of the earth’s temperature is now entering its second decade. While the literature is large, there has been very little collaboration with universitylevel, professional statisticians (Wegman et al., 2006; Wegman, 2006). Our paper is an effort to apply some modern statistical methods to these problems. While our results agree with the climate scientists findings in some
respects, our methods of estimating model uncertainty and accuracy are in sharp disagreement.
On the one hand, we conclude unequivocally that the evidence for a ”long-handled” hockey stick (where the shaft of the hockey stick extends to the year 1000 AD) is lacking in the data. The fundamental problem is that there is a limited amount of proxy data which dates back to 1000 AD; what is available is weakly predictive of global annual temperature. Our backcasting methods, which track quite closely the methods applied most recently in Mann (2008) to the same data, are unable to catch the sharp run up in temperatures recorded in the 1990s, even in-sample.
As can be seen in Figure 15, our estimate of the run up in temperature in the 1990s has
a much smaller slope than the actual temperature series. Furthermore, the lower frame of Figure 18 clearly reveals that the proxy model is not at all able to track the high gradient segment. Consequently, the long flat handle of the hockey stick is best understood to be a feature of regression and less a reflection of our knowledge of the truth. Nevertheless, the temperatures of the last few decades have been relatively warm compared to many of the thousand year temperature curves sampled from the posterior distribution of our model.
Our main contribution is our efforts to seriously grapple with the uncertainty involved in paleoclimatological reconstructions. Regression of high dimensional time series is always a complex problem with many traps. In our case, the particular challenges include (i) a short sequence of training data, (ii) more predictors than observations, (iii) a very weak signal, and (iv) response and predictor variables which are both strongly autocorrelated.
The final point is particularly troublesome: since the data is not easily modeled by a simple autoregressive process it follows that the number of truly independent observations (i.e., the effective sample size) may be just too small for accurate reconstruction.
Climate scientists have greatly underestimated the uncertainty of proxy based reconstructions and hence have been overconfident in their models. We have shown that time dependence in the temperature series is sufficiently strong to permit complex sequences of random numbers to forecast out-of-sample reasonably well fairly frequently (see, for example, Figure 9). Furthermore, even proxy based models with approximately the same amount of reconstructive skill (Figures 11,12, and 13), produce strikingly dissimilar historical backcasts: some of these look like hockey sticks but most do not (Figure 14).
Natural climate variability is not well understood and is probably quite large. It is not clear that the proxies currently used to predict temperature are even predictive of it at the scale of several decades let alone over many centuries. Nonetheless, paleoclimatoligical reconstructions constitute only one source of evidence in the AGW debate. Our work stands entirely on the shoulders of those environmental scientists who labored untold years to assemble the vast network of natural proxies. Although we assume the reliability of their data for our purposes here, there still remains a considerable number of outstanding questions that can only be answered with a free and open inquiry and a great deal of replication.
===============================================================
Commenters on WUWT report that Tamino and Romm are deleting comments even mentioning this paper on their blog comment forum. Their refusal to even acknowledge it tells you it has squarely hit the target, and the fat lady has sung – loudly.
(h/t to WUWT reader “thechuckr”)
Great...thanks for adding those links...
*******************************************************
Lew Skannen says:
Nice article.
“instead of trying to attack the proxy data quality issues, they assumed the proxy data was accurate for their purpose, then created a bayesian backcast method”
What I especially like about this particular article is that it now moves the battlefield to where we want it. Rather than squabble over how thick the ice is this year or how hot last April was in South Tuvalu we need to get to the heart of the matter and ask – How accurate are the models?
It is quite clear that any half believable model will have to include thresholds, tipping points, runaway processes etc. ie they will be highly non-linear and most likely chaotic.
I want to see more work in this area – exposing the models for the hamfisted guesses that they are.
***************************************************
CRS, Dr.P.H. says:
Why should the community of climatologists object to this peer-reviewed publication? After all, they stood up & cheered on RC etc. when the Oxburgh inquiry exonerated the Hockey Team of professional malfeasance.
However, they could have used a few undergraduate classes in linear regression!
–
The panel found that the statistical tools that CRU scientists employed were not always the most cutting-edge, or most appropriate.
“We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians,” reads the inquiry’s conclusions.
However, “it is not clear that better methods would have produced significantly different results,” the panel adds.
http://www.newscientist.com/article/dn18776-climategate-scientists-chastised-over-statistics.html
——
This latest publication seems to indicate that, yes, better statistical methods DO produce significantly different results!
IOWs We are going to have a debate about the science whether Algore likes it or not. LOL
*******************************************************************
MarkG says:
“You need a theory to explain what is happening now. It needs to be falsifiable. And you have to either accept that new scientific papers fit your theory, or explain why they don’t.”
No, we don’t, because we’re not the ones making extraordinary claims. Instead we’re faced by a theory of EVIL BABY-KILLING CARBON DIOXIDE which doesn’t appear to be considered falsifiable in any way no matter how far its predictions diverge from reality. Hot, cold, wet, dry, windy or not, any change in the weather always turns out to be due to EVIL BABY-KILLING CARBON DIOXIDE.
This is why those of us with a science background have gone from amazed to appalled as the ‘Global Warming’… sorry… ‘Climate Change’… charade has continued to gain momentum when it’s clearly pseudoscientific bunk.
I have been busy upgrading my hardware and Software ...I am not sure where all my Links are...
**************************************************
Thrasher says:
What an ugly disaster for the IPCC/Mann/CRU crowd. This really casts a lot of doubt on their statistical reasoning. The paper is sound because it doesn’t question recent warming (which most definitely exists), but questions their claim that prior warmings have been nothing like this and their ridiculous reconstruction that lacks a medieval warm period.
A true statement with no real revelations in it.
It can't be clear what results you might get if you used methods that weren't employed before. Things left untried leave things left unanswered.
*********************************************************
Mike Edwards says:
duckster says:
August 14, 2010 at 8:18 pm:
…So where exactly would you place a medieval warming period here? Asking me to accept a medieval warming period…
It isn’t just a question of a medieval warm period – there are a whole series of periods in the past 500,000 years that appear to have been warmer than the present. 4 of the previous interglacials have been considerably warmer than the current one, as shown by Ice Core data and by by significantly higher sea levels (~ 6 metres higher in the last interglacial, for example).
If you don’t believe me, Wikipedia has a good discussion and links to much of the data here:
http://en.wikipedia.org/wiki/Ice_core
The question for the climate modellers is whether their models can account for this behaviour of the Earth’s climate WITHOUT resorting to CO2, since the ice cores don’t show CO2 above pre-industrial levels.
******************************************************************
Graph of CO2 (green), reconstructed temperature (blue) and dust (red) from the Vostok ice core for the past 420,000 years
*****************************************************************
Latimer Alder says:
@Mike Roddy
‘The authors of the 20- odd studies that confirmed Mann’s data are not really interested in what professional statisticians and mathematicians are saying about it’
What an astonishing remark!
For a community that is ever quick to criticise others for lacking the ‘right’ qualifications in climatology, (whatever those may be) to be indifferent to professional statisticians verdict on their work is quite amazing.
As far as I can tell, having established the basic data to be used, there is no climatological knowledge required to manipulate the numbers and produce the graphs that Mann and his chums have relied on for over a decade. The knowledge and skills required are purely statistical.
And here we have two professional statisticians demonstrating that this part of the work has not been done to a professional standard and that many of the supposed conclusions cannot be derived from the data. And that the basic premise – that tree ring data can somehow tell us about past temperatures – is unsound.
Wow! no doubt there will be a considerable brouhaha once the paper is properly published…now it is in the public domain it cannot be suppressed anyway…but it is difficult to imagine what robust defence the Team can come up with.
That the authors are unqualified in their field…nope..better qualified than Mann et al
That the authors have cherry picked the data…nope…they used the same data as Mann
That the authors are funded by Big Oil..even if true, unlikely to be taken seriously as an argument apart from by True Believers
That its all terribly unfair and the poor polar bears are going to fry just about ten minutes before they would have drowned……about the best that they can do.
Its been a great summer so far. The total debunking of CRU already, and now this earth-shattering paper.
That they choose
********************************************************************
latitude says:
Brad says:
August 15, 2010 at 6:27 am
So what happens when you use the real data? I guess the whole thing was made up?
=========================================================
Brad, there’s no real data, they used Mann’s data.
This is not a reconstruction of temperature data, this is a reconstruction of Mann’s data.
It’s not meant to prove or disprove or anything to do with the MWP.
It’s only looking at Mann’s reconstruction of his own data.
Mann ran his data and came up with a flat line with a up-tic on the end, the hockey stick.
They ran his own data, and came up with warmer temperatures at the beginning than the end, no hockey stick.
If this paper proves to be true, then it can only mean one of two things:
1 Mann lied and cheated
2 Mann doesn’t know what he’s doing and is inept
See #33.
***********************************************************
Keith Battye says:
The message to me, and I have read the whole thing now, is that using the data the AGW set have used, Mann’s graph cannot be derived with the correct statistical tools.
They make no comment as to the robustness of the data but simply point out , indirectly, that Mann and the AGW set had to improperly torture their own data to produce the FUD hockey stick.
They are saying that Mann et al have cooked the books. Like those boys over at ENRON or Bernie Maddof they have carried out an act of fraud and 20 other good ol’ boys have validated their deceit. This was proved incontrovertibly without having to delve into the suspected alterations to the data before they were used in the fraud.
I now await a similar expose on the data by people who have the same kudos . This CO2 thing is just another boondoggle, and a small whiff of causation as seen by Arhennius is not an argument for blaming carbon for any climatic changes we may be going through. It rather reminds me of a trick we would play on the credible at school , getting people to blow into a beaker of quick lime solution and point out that there was something wrong with them as the solution turned cloudy. We always had the “cure” and the beaker of fresh water handy.
Mountebanks is what they are.
Monckton: Why current trends are not alarming ( RE:Global Warming)
1 Mann lied and cheated
2 Mann doesn’t know what he’s doing and is inept
I’ll go with #1
A big blow to the EPA and more ammunition for the State of Virginia investigation
This morning when I read it at WUWT there was only 85 comments. Now there is over 335.
Looks like Mann is not out of the dog house yet.
lied and cheated works for me...
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.