Free Republic
Browse · Search
Bloggers & Personal
Topics · Post Article

To: grey_whiskers
I'm guessing here that these bozos got some climate science graduate students with minimal programming experience to do all of their coding for them instead of hiring professionally trained programmers and database managers. The students were smart enough, however, to cover their ass by making a list of all the mistakes that have accumulated both unintenionally and intentionally in the programs and data files.
9 posted on 11/25/2009 10:54:24 PM PST by Kirkwood
[ Post Reply | Private Reply | To 1 | View Replies ]


To: Kirkwood
I'm guessing here that these bozos got some climate science graduate students with minimal programming experience

First of all, you are apparently right. Someone called Tim Mitchell was working there on his PhD, and he wrote most of that "code." Once he got the PhD he just walked out of the door, never to be seen again - and that's where Harry comes into the spotlight. Harry was tasked with reverse-engineering Tim's mess, and he couldn't make any sense out of it. No surprise if the "settled science" that the CRU published was based on a PhD work of a student, and that work was never even checked, let alone peer-reviewed.

I worked with a few scientists, and they are typically awful programmers. This is because they never studied computer science. They may be geniuses in their area of expertise (not the case here, likely) but they are pathetic in implementation of their wondrous algorithms. They write spaghetti code as if there is no tomorrow, and in many cases that's true - they get their degrees and move on. They never need to worry about version control, documentation, maintenance - none of that usual stuff that is *required* for any production software.

So I understand how it all happened. There was no management in the whole process, ever. Stations were sending monthly data in similar, but minutely different formats; the disks with reports were not filed but just thrown into desk drawers until they are needed. There were no backups, there was no process established. Your average car mechanic has a better ran computer - he knows that if his database of customers is lost he'd be in trouble. But those guys just didn't care. All they wanted is the "correct" results, and Tim worked hard to produce them. And when you have millions of measurements over tens of years, it's easy - especially if you don't allow anyone to check your work. Tim's boss knew what's up and it was his task to make sure nobody gets to see the data and the code - until now.

So when all these files started coming, people like Tim, due to lack of formal CS training, screwed it up big time. Results were erroneous. They noticed that, and Tim added a layer of patches to "fix" the problem - which only made it worse, so they had to patch again and again. At some point they just inserted a huge fudge factor to fit the measurements into the politics. And when it was time to publish they ran the program, in its sorry state, one last time and published whatever came out. And it was all garbage, of course.

So when Harry was given this mess to sort out, no surprise he couldn't recreate the published results. How could he, if there is not a single written note on how to do it! And the programs were changed many times since then, and there is no record of what was changed and how. To add to that, some data files are simply lost, and other mislabeled, and not a single person present knows how Tim did what he did. Probably even Tim couldn't repeat the calculations on the next day, so disorganized the whole work was. Professional programmers know very well how to avoid such disasters - by using version control software - but obviously CRU geniuses were too much above all that mundane stuff.

Is it possible to save the situation? Yes, but it's not easy. First of all, they need to publish *all* source data (station data) that they accumulated over time. Scientists need to reconcile this data with other records and agree on how to interpret the data. Then it must be converted to a common format and again published. Everyone must agree that this data is reasonable. Only then the scientists may start working on analysis of that new batch of data. This will take a few years, especially none of the CRU crew may be involved - they can't be trusted.

15 posted on 11/26/2009 12:25:33 AM PST by Greysard
[ Post Reply | Private Reply | To 9 | View Replies ]

To: Kirkwood
I'm guessing here that these bozos got some climate science graduate students with minimal programming experience to do all of their coding for them instead of hiring professionally trained programmers and database managers. The students were smart enough, however, to cover their ass by making a list of all the mistakes that have accumulated both unintenionally and intentionally in the programs and data files.

My experience in academia was that professors preferred working on the cheap, using free student labor in exchange for good "project" grades. I was a student software developer working on a grant for statistical analysis of software reliability for the Air Force. Based on what I saw, the professor did a song and dance and pocketed most of the grant money.

26 posted on 11/26/2009 9:53:51 AM PST by PapaBear3625 (Public healthcare looks like it will work as well as public housing did.)
[ Post Reply | Private Reply | To 9 | View Replies ]

Free Republic
Browse · Search
Bloggers & Personal
Topics · Post Article


FreeRepublic, LLC, PO BOX 9771, FRESNO, CA 93794
FreeRepublic.com is powered by software copyright 2000-2008 John Robinson