CRU's Source Code: Climategate Uncovered

American Thinker ^ | 11/25/2009 | Marc Sheppard

[Smogger]

As the evidence of fraud at the University of East Anglia's prestigious Climactic Research Unit (CRU) continues to mount, those who've been caught green-handed continue to parry their due opprobrium and comeuppance, thanks primarily to a dead-silent mainstream media. But should the hubris and duplicity evident in the e-mails of those whose millennial temperature charts literally fuel the warming alarmism movement somehow fail to convince the world of the scam that's been perpetrated, certainly these revelations of the fraud cooked into the computer programs that create such charts will.

-snip-

One can only imagine the angst suffered daily by the co-conspirators, who knew full well that the "Documents" sub-folder of the CRU FOI2009 file contained more than enough probative program source code to unmask CRU's phantom methodology.

In fact, there are hundreds of IDL and FORTRAN source files buried in dozens of subordinate sub-folders. And many do properly analyze and chart maximum latewood density (MXD), the growth parameter commonly utilized by CRU scientists as a temperature proxy, from raw or legitimately normalized data. Ah, but many do so much more.

Skimming through the often spaghetti-like code, the number of programs which subject the data to a mixed-bag of transformative and filtering routines is simply staggering. Granted, many of these "alterations" run from benign smoothing algorithms (e.g., omitting rogue outliers) to moderate infilling mechanisms (e.g., estimating missing station data from that of those closely surrounding). But many others fall into the precarious range between highly questionable (removing MXD data which demonstrate poor correlations with local temperature) to downright fraudulent (replacing MXD data entirely with measured data to reverse a disorderly trend-line).

In fact, workarounds for the post-1960 "divergence problem," as described by both RealClimate and Climate Audit, can be found throughout the source code. So much so that perhaps the most ubiquitous programmer's comment (REM) I ran across warns that the particular module "Uses 'corrected' MXD - but shouldn't usually plot past 1960 because these will be artificially adjusted to look closer to the real temperatures."

What exactly is meant by "corrected” MXD," you ask? Outstanding question -- and the answer appears amorphous from program to program. Indeed, while some employ one or two of the aforementioned "corrections," others throw everything but the kitchen sink at the raw data prior to output.

For instance, in the subfolder "osborn-tree6\mann\oldprog," there’s a program (Calibrate_mxd.pro) that calibrates the MXD data against available local instrumental summer (growing season) temperatures between 1911-1990, then merges that data into a new file. That file is then digested and further modified by another program (Pl_calibmxd1.pro), which creates calibration statistics for the MXD against the stored temperature and "estimates" (infills) figures where such temperature readings were not available. The file created by that program is modified once again by Pl_Decline.pro, which "corrects it" – as described by the author -- by "identifying" and "artificially" removing "the decline."

But oddly enough, the series doesn’t begin its "decline adjustment" in 1960 -- the supposed year of the enigmatic "divergence." In fact, all data between 1930 and 1994 are subject to "correction."

And such games are by no means unique to the folder attributed to Michael Mann.

A Clear and Present Rearranger

In two other programs, briffa_Sep98_d.pro and briffa_Sep98_e.pro, the "correction" is bolder by far. The programmer (Keith Briffa?) entitled the "adjustment" routine “Apply a VERY ARTIFICAL correction for decline!!” And he or she wasn't kidding. Now IDL is not a native language of mine, but its syntax is similar enough to others I'm familiar with, so please bear with me while I get a tad techie on you.

Here's the "fudge factor" (notice the brash SOB actually called it that in his REM statement): yrloc=[1400,findgen(19)*5.+1904]

valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,2.6,2.6,2.6]*0.75 ; fudge factor

These two lines of code establish a twenty-element array (yrloc) comprising the year 1400 (base year, but not sure why needed here) and nineteen years between 1904 and 1994 in half-decade increments. Then the corresponding "fudge factor" (from the valadj matrix) is applied to each interval. As you can see, not only are temperatures biased to the upside later in the century (though certainly prior to 1960), but a few mid-century intervals are being biased slightly lower. That, coupled with the post-1930 restatement we encountered earlier, would imply that in addition to an embarrassing false decline experienced with their MXD after 1960 (or earlier), CRU's "divergence problem" also includes a minor false incline after 1930.

And the former apparently wasn't a particularly well-guarded secret, although the actual adjustment period remained buried beneath the surface.

Plotting programs such as data4alps.pro print this reminder to the user prior to rendering the chart: IMPORTANT NOTE: The data after 1960 should not be used. The tree-ring density records tend to show a decline after 1960 relative to the summer temperature in many high-latitude locations. In this data set this "decline" has been artificially removed in an ad-hoc way, and this means that data after 1960 no longer represent tree-ring density variations, but have been modified to look more like the observed temperatures. Others, such as mxdgrid2ascii.pro, issue this warning: NOTE: recent decline in tree-ring density has been ARTIFICIALLY REMOVED to facilitate calibration. THEREFORE, post-1960 values will be much closer to observed temperatures then (sic) they should be which will incorrectly imply the reconstruction is more skilful than it actually is. See Osborn et al. (2004).

Care to offer another explanation, Dr. Jones?

Gotcha

Clamoring alarmists can and will spin this until they're dizzy. The ever-clueless mainstream media can and will ignore this until it's forced upon them as front-page news, and then most will join the alarmists on the denial merry-go-round.

But here's what’s undeniable: If a divergence exists between measured temperatures and those derived from dendrochronological data after (circa) 1960, then discarding only the post-1960 figures is disingenuous, to say the least. The very existence of a divergence betrays a potential serious flaw in the process by which temperatures are reconstructed from tree-ring density. If it's bogus beyond a set threshold, then any honest man of science would instinctively question its integrity prior to that boundary. And only the lowliest would apply a hack in order to produce a desired result.

And to do so without declaring as such in a footnote on every chart in every report in every study in every book in every classroom on every website that such a corrupt process is relied upon is not just a crime against science, it’s a crime against mankind.

Indeed, miners of the CRU folder have unearthed dozens of e-mail threads and supporting documents revealing much to loathe about this cadre of hucksters and their vile intentions. This veritable goldmine has given us tales ranging from evidence destruction to spitting on the Freedom of Information Act on both sides of the Atlantic. But the now-irrefutable evidence that alarmists have indeed been cooking the data for at least a decade may be the most important strike in human history.

Advocates of the global governance/financial redistribution sought by the United Nations at Copenhagen in two weeks, and also those of the expanded domestic governance/financial redistribution sought by Liberal politicians, both substantiate their drastic proposals with the pending climate emergency predicted in the reports of the Intergovernmental Panel on Climate Change (IPCC). Kyoto, Waxman-Markey, Kerry-Boxer, EPA regulation of the very substances of life -- all bad policy concepts enabled solely by IPCC reports. And the IPCC in turn bases those reports largely on the data and charts provided by the research scientists at CRU -- largely from tree ring data -- who just happen to be editors and lead authors of that same U.N. panel.

Bottom line: CRU's evidence is now irrevocably tainted. As such, all assumptions based on that evidence must now be reevaluated and readjudicated. And all policy based on those counterfeit assumptions must also be reexamined.

Gotcha. We know they've been lying all along, and now we can prove it. It's time to bring sanity back to this debate.

It's time for the First IPCC Reassessment Report.

[avacado]

I happened to watch CBS Evening News last night and Katie Couric didn’t mention a thing about this climate scam. MSM trying to bury it hard and deep.

[Unrepentant VN Vet]

FORTRAN. hey man grandpa used that way back before pythagoras. Hey man, that is a horrible language no one would still use that!!!

Okay, which incarnation of FORTRAN are you kids talking about?

Did some grad school work back in the early 70's using FORTRAN 4 or thereabouts, so think I'll download some of this CRU stuff and thrash around a bit.

I'd ordinarily give 'em the benefit of the doubt, but after reading through some of their more crass political/tactical memos, I really can't decide if we should re-designate CRU as Crooks or CHARLEY FOXTROTs-R-Us.

Anybody with a technical or scientific background should be so PO'd by this, they'd be about to go postal on 'em.

Don't even ask what I'm thinking about......

[kempo]

The reason CBS, ABC, NBC are losing viewers by the day is because they do not report the news. They use to get away with it until the Internet came along.

[USNBandit]

I don't know if forecasting and recording past weather are really the same thing.

Here is a better question to ask. If you turned in your high school chemistry lab results which you "smoothed" with a correction factor, what kind of grade would you get?

"Blutarski, zero point zero." zero

[mmichaels1970]

Here's the "fudge factor" (notice the brash SOB actually called it that in his REM statement):

yrloc=[1400,findgen(19)*5.+1904]

valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,2.6,2.6,2.6]*0.75 ; fudge factor

Yikes. Don't need to know FORTRAN to understand that.

Reminds me of computer games I used to write when I was a kid. I'd randomize dice rolls, but skew the roll based on who was playing. I was more likely to get a 6 while my sister took her chances.

Oldest programming trick in the book.

[Stegall Tx]

I doubt if this will have any affect on the GW thing at all. I mean, facts have been on our side of the debate throughout this entire argument, but they (the GW believers) have never allowed facts to get in the way of their presentation. And they (the GW believers) have been very effective in getting others on board in spite of the lack of factual data.

I just don’t see how this admission that the few “facts” they presented were just created out of thin air will sway anyone.

[Think free or die]

These ethical breaches sicken me.

For quite a few years before the derivatives mess imploded, I was uncomfortable with the sheer complexity of hyper-finance. It seemed that things had reached a point at which most investors couldn't critically evaluate what was going on. We have since seen how trust was abused, and how the skills and technological tools available on Wall Street were too often used to fleece investors rather than to enhance productivity of capital.

Now in the realm of climate science, we again see complex modeling using data which are relatively inaccessible to citizens. The data were held confidential or altogether obscure. Computer models were either hidden or difficult to comprehend. Many people put their trust in experts wielding technology. Again, their trust was abused and their pockets picked.

Dishonesty is nothing new. Unfortunately, with the addition of technology and globalization, fraud and abuse of intellectual power can have a huge impact. With the amplifying effect of technology and global reach, the need for ethics and transparency is perhaps greater than ever. I'm grateful we have so many keyboard experts willing to dig into these specialized subjects. Sunshine is good.

[sergeantdave]

“Kyoto, Waxman-Markey, Kerry-Boxer, EPA regulation of the very substances of life — all bad policy concepts enabled solely by IPCC reports.”

I’m confused. Simple question: Can we lawfully hang these bastards? And I include Waxman, Markey, certain members of the supreme court and the traitor kerry and his drunk wife Tereezaa.

I really want to see the crypto-Nazi Tides Foundation nailed, which is funded by the ketchup heiress tereeeza. Tides leads back to the Nazi kappo Soros and his crimes of murdering Jewish citizens for 30 pieces of silver in Hitler’s Germany.

We need an American-style Nuremburg trial to sort out those who need to be hanged and those who we ship one-way to North Korea.

BTW, you people who understand this GW farce are doing a great job exposing it. Mucho thanks.

[rawhide]

Whew! Do you have a simple man’s Reader’s Digest version of your post?

[sergeantdave]

Well, I’ve never seen or heard of you before. It’s good to see you old FReeper guys stepping into the spotlight.

Welcome to FR!

:-)

(Just having fun with you...)

[VOA]

“Anybody with a technical or scientific background should be so
PO’d by this, they’d be about to go postal on ‘em.”

Amen. If you don’t hear more noise, most of the honest, toiling academic
scientists are generally a quiet lot (you’ve never seen anyone do
a victory dance, spray themselves with champagne, or do a rebel yell
when they get a Nobel Prize in the sciences).

The real tragedy in my view (and that of my brother who earned his
grad. degree in plant biology at U. Minnesota) is that these
“climatologists” have POISONED THE WELL.
Now even the Average Joe/Josephine will wonder if their hard-earned
tax dollars are being p-ssed away by a lot of self-serving jerks
that write their conclusions before they construct the rest of their publications.

And who won’t have a cynical arching of the eyebrows when they
hear some scientist (however honest and sincere) sing the praises
of “peer review”?

The best way to restore integrity to the scientific enterprise is
to let Senator Inhofe issue plenty of subpoenas and get as many of
“The Unindicted Co-Conspirators” on the record (under penalty of perjury).

Maybe then academic science in the USA (and maybe even elsewhere)
might regain the respect of the citizens that pony up so many tax
dollars for what should be AT THE LEAST a fairly decent human enterprise.
(No rational person ever considered it the work of a bunch of saintly monks.)

Just my $0.02 of thought and venting...

[sergeantdave]

“There NEVER WAS 2,500 Climate Scientists...”

Two were phrenologists and one sacrificed virgins to volcanoes in a previous life. Sorry, no pictures.

[danielmryan]

I always understood that the models have been problematic from the start. If you ran the models forward from the early twentieth century, you didn’t get current empirical results. And if you ran it backwards, you didn’t line up with actual temperatures from a century ago. So the models are faulty and their observations have been fudged.

Bingo! Other than that vector, the fudge factors were introduced to cover up the fact that the tree-ring method did not give accurate replications of real temperatures.

I know that this point seems less damning than political corruption, but that's only because it's more subtle.

To get what I'm driving at, imagine a gold-mining geologist who comes up with a new assay technique, which he says will do wonders. When he tests it on a known core sample, he finds out that it underestimates the amount of gold in some of the core. So, he resorts to the "gold-ring trick" to make his assay square with the already-derived results.

It may seem harmless, because it just squares the results off with standard technique, but the guy's committed a fraud. He didn't falsely inflate the gold in the core, but he did falsely inflate the reliability of his spanking-new assay technique. If this hypothetical geologist should get away with it, exploration companies (and their investors) would be bilked into thinking that valuable properties are valueless.

This hypothetical geologist with a snake-oil assay technique is the best analogy I can think of to get the point across. Feel free to correct me if I'm wrong.

[hyperconservative]

Very funny.
Thanks for sharing it.

[Marty62]

Hummm this is interesting since I remember being innundated with talk of GLOBAL COOLING during the 60’s and 70’s.
I would like to know WHO decided to start the GLOBAL WARMING BS.

[Marty62]

Check this out very readable.

http://www.climatecooling.org/Current-USA-Climate-Change-Review.html

[Melchior]

Years ago I took a graduate course in climatology from a Penn State professor. It was the most challenging, the most difficult course, I ever had in my academic career. What I took from it (aside from the fact that I needed an extensive grounding in physics that I did not have) was that climate was dependent upon solar activity, which in turn had its effect on sea currents, which in turn had its effect on the so-called intertropical convergence zone. I can recall discussion of the Nino effect in Peru based on nineteenth century observations. Years later I would argue with the U. of Arizona types — the tree ring circus — that their analyses were what they wanted to see, not that which was quatifiable. Anyway, I feel that the Penn State prof, now deceased, must be flipping in his grave to see what has happened to the science he loved.

[Melchior]

Regarding the mortgage meltdown, when the mortgages were bundled the sale of the bundle itself to banks and institutions were backed by an evaluation of risk whose mathematical equation (which could not be understood by even the best of mathematicians) was found to be pure bunk. When it comes to modern scientists, caveat emptor.

[cantfindagoodscreenname]

This whole “Michael Mann” thing is cracking me up because my 17 year old son’s name is Michael Mann. Today Rush yelled, “Michael Mann is a fraud!” and I’m trying to find it on today’s mp3 file so that I can play it for my son, lol. It’s so strange to hear Rush say my son’s name over and over on the air.

[Gondring]

“All models are wrong; some models are useful.”