The Science Journal, Nature publishes a Flawed Global Warming Paper : Peer Review Problem

INFORMATH ^ | 11/03/2006 | Douglas J. Keenan

[SirLinksalot]

On 18 November 2004, Isabelle Chuine and co-workers published a research paper on global warming. The paper appeared in Nature, the world's most highly-regarded scientific journal. And it gathered some publicity. Chuine et al. claimed to have developed a method for estimating the summer temperature in Burgundy, France, in any given year back to 1370 (based on the harvest dates of grapes). Using their method, the authors asserted that the summer of 2003 was the warmest summer since 1370, in Burgundy.

I had been following global warming studies only as a disinterested outside spectator (and only occasionally). Someone sent me the paper of Chuine et al., though, and wondered what I thought of it from a mathematical perspective. So I had a look.

To study the paper properly, I needed to have the authors' data. So I e-mailed Dr. Chuine, asking for this. The authors, though, were very reluctant to let me have the data. It took me eight months, tens of e-mails exchanged with the authors, and two formal complaints to Nature, before I got the data. (Some data was purchased from Météo France.) It is obviously inappropriate that such a large effort was necessary.

Looking at the data made it manifest that there are serious problems with the work of Chuine et al. In particular, the authors' estimate for the summer temperature of 2003 was higher than the actual temperature by 2.4 °C (about 4.3 °F). This is the primary reason that 2003 seemed, according to the authors, to be extremely warm.

There is also another reason. The three warmest years on record, prior to 2003, were 1945, 1947, and 1952. (The instrumental record goes back to 1922, or even 1883 if we accept some inaccuracies.) The estimate of Chuine et al. for the summer temperature in each of those years was much lower than the actual temperature.

That is, the authors had developed a method that gave a falsely-high estimate of temperature in 2003 and falsely-low estimates of temperatures in other very warm years. They then used those false estimates to proclaim that 2003 was tremendously warmer than other years.

The above is easy enough to understand. It does not even require any specialist scientific training. So how could the peer reviewers of the paper not have seen it? (Peer reviewers are the scientists who check a paper prior to its publication.) I asked Dr. Chuine what data was sent to Nature, when the paper was submitted to the journal. Dr. Chuine replied, “We never sent data to Nature”.

I have since published a short note that details the above problem (reference below). There are several other problems with the paper of Chuine et al. as well. I have written a brief survey of those (for people with an undergraduate-level background in science). As described in that survey, problems would be obvious to anyone with an appropriate scientific background, even without the data. In other words, the peer reviewers could not have had appropriate background.

What is important here is not the truth or falsity of the assertion of Chuine et al. about Burgundy temperatures. Rather, what is important is that a paper on what is arguably the world's most important scientific topic (global warming) was published in the world's most prestigious scientific journal with essentially no checking of the work prior to publication.

Moreover—and crucially—this lack of checking is not the result of some fluke failures in the publication process. Rather, it is common for researchers to submit papers without supporting data, and it is frequent that peer reviewers do not have the requisite mathematical or statistical skills needed to check the work (medical sciences largely excepted). In other words, the publication of the work of Chuine et al. was due to systemic problems in the scientific publication process.

The systemic nature of the problems indicates that there might be many other scientific papers that, like the paper of Chuine et al., were inappropriately published. Indeed, that is true and I could list numerous examples. The only thing really unusual about the paper of Chuine et al. is that the main problem with it is understandable for people without specialist scientific training. Actually, that is why I decided to publish about it. In many cases of incorrect research the authors will try to hide behind an obfuscating smokescreen of complexity and sophistry. That is not very feasible for Chuine et al. (though the authors did try).

Finally, it is worth noting that Chuine et al. had the data; so they must have known that their conclusions were unfounded. In other words, there is prima facie evidence of scientific fraud. What will happen to the researchers as a result of this? Probably nothing. That is another systemic problem with the scientific publication process.

See also Peer review and the IPC

--------------------------------------------------------------------------------

Chuine I., Yiou P., Viovy N., Seguin B., Daux V., Le Roy Ladurie E. (2004), “Grape ripening as a past climate indicator”, Nature, 432: 289–290. doi: 10.1038/432289a.

Keenan D.J. (2007), “Grape harvest dates are poor indicators of summer warmth”, Theoretical and Applied Climatology, 87: 255–256. doi: 10.1007/s00704-006-0197-9.

[SirLinksalot]

See also : Problems With the Peer Review Process.

http://www.informath.org/apprise/a3600.htm

[GSlob]

I have a better idea: maybe we should start trying to estimate the temperatures not by grape harvest data, but by the comparative tasting of actual vintages, since 1370. A good research grant would, of course, be needed - and I fully trust that among the FReepers there will be found enough selfless volunteers in the cause of climate science.

[Arm_Bears]

Peer-review is over-rated.

Most contemporary "scientific" journals have adopted a liberal world-view, and editorial boards/reviewers are selected primarily from academia. This arrangement ensures that favorable "research" makes it into print.

[facedown]

The paper appeared in Nature, the world's most highly-regarded scientific journal.

Only by lefty numbnuts.

[atlaw]

Out of curiosity, which scientific journals do you regard highly?

[NicknamedBob]

"... selfless volunteers in the cause of climate science."

Hear, hear! It is forward thinking individuals such as you who make this country great!

Your willingness to step forward brings a tear to my eye.

[NaughtiusMaximus]

Here, publish this: "Famous FReeper says, "The weather is more the way it is today than it has ever been before." Grant monies threatened, scientists stunned."

[Kenny Bunk]

.....comparative tasting of actual vintages, since 1370. A good research grant would, of course, be needed - and I fully trust that among the FReepers there will be found enough selfless volunteers in the cause of climate science....

Even though this work would be arduous in the extreme, I for one, am willing to leave my luxurious retirement to join your team of selfless oenotechs.

Much of this work has been done by Ted Kennedy,with fatal results to members of his team. But science demands the duplicability of experiments, and I, in the tradition of Pasteur, am willing to lay my life on the line.

And that's even if I have to drink dessert wines. We'll git'erdone!

This just in from California: Gallo Brothers report that in an emergency, some of their products could be made from grapes.
This just in from France: The French have a 100 million-gallon wine surplus. They plan to turn it into brake fluid, antifreeze, Citroen Suspension Fluid and a host of other industrial chemicals, rather than go the 'Two-Buck-Chuck' route.
Warning: Do not attempt to stop your Renault, Drive your Citroen on rough roads, or use le chauffage dela voiture (dans Québec, du chariot)

[Blueflag]

I would argue that Scientific American is more highly regarded and more objectve in fact, though equally "socially conscious" to use a eupemism.

[Blueflag]

As I understand wine making (which is admittedly a very small understanding of the finer points) the grapes are harvested more around a measure of the desired sugar content -- much of which is controlled by solar power hours -- no so much temperatures. Photosynthesis yields sugar, not temperature (although an increase in temp yields a higher metabolic rate).

What's my point?

The seasons that were harvested EARLY may have been years with lots of sunshine late in the season, so the grapes were ready sooner. Later season harvest might have been cloudier.

I think the premise is flawed that later harvests result from cooler summers/falls.

I remain to be convinced.

[facedown]

Out of curiosity, which scientific journals do you regard highly?

In my field: Biochemistry, Journal of Biological Chemistry, and for might-be-something-to-it curiosity, Proceedings of the National Academy of Sciences.

[maine-iac7]

I have a better idea: maybe we should start trying to estimate the temperatures not by grape harvest data, but by the comparative tasting of actual vintages, since 1370. A good research grant would, of course, be needed - and I fully trust that among the FReepers there will be found enough selfless volunteers in the cause of climate science.

Count me in!

[maine-iac7]

apropos of nothing, I love your ID, SirLinksalot.

[r9etb]

Isabelle Chuine.

[William Tell]

Blueflag said: "The seasons that were harvested EARLY may have been years with lots of sunshine late in the season, so the grapes were ready sooner."

Isn't it also possible that signs of an early winter can rush the harvest? I know that the grape-growers around here panic if the crop gets rained on at the wrong time. Rain water sitting on the grapes can cause them to burst.

[cogitator]

...and another demonstration of the overconfidence placed in peer review.

The peer-review performed by reviewers for journal publications is only the first of a multiple-step process that encompasses the term "peer review".

Once papers are published, the scientific community then has the opportunity to examine the publication, determine if the results are worth additional investigation, perform additional research, and publish either supporting or rebuttal publications. This is also peer-review, and frequently far more important than journal pre-publication review.

Deficient papers do get published. That fact doesn't undermine scientific results, in general. But it's a good indicator that multiple supporting results are indicative of actual scientifically-evaluated "truth".

[A. Pole]

Looking at the data made it manifest that there are serious problems with the work of Chuine et al. In particular, the authors' estimate for the summer temperature of 2003 was higher than the actual temperature by 2.4 °C (about 4.3 °F). This is the primary reason that 2003 seemed, according to the authors, to be extremely warm. There is also another reason. The three warmest years on record, prior to 2003, were 1945, 1947, and 1952. (The instrumental record goes back to 1922, or even 1883 if we accept some inaccuracies.) The estimate of Chuine et al. for the summer temperature in each of those years was much lower than the actual temperature.

Political Correctness is DESTROYING science !

[AndrewC]

Rather, what is important is that a paper on what is arguably the world's most important scientific topic (global warming) was published in the world's most prestigious scientific journal with essentially no checking of the work prior to publication.

Why should it be checked? It supports the paradigm.

[metmom]

Peer review problem? Nah! can't be. I mean, it's supposed to be about infallible, isn't it?