Document drop: A new critique of the 2004 Lancet Iraq death toll study

One of the most useful roles of the blogosphere is its service as an open-source intelligence-gathering medium. You can draw on the expertise of people around the world at the touch of a button. We saw this with typography experts during the Rathergate scandal; Photoshop experts during the Reutersgate debacle; and military experts during the Jesse Macbeth unmasking.

Now, it’s the statisticians and math geeks’ turn. Remember that massively-publicized 2004 Lancet Iraq death toll study? It was cited in nearly 100 scholarly journals and reported by news outlets around the world. “100,000 Civilian Deaths Estimated in Iraq” blared the Washington Post in a typical headline.

There were attempts made by lay journalists to debunk the 2004 study (as well as the 2006 follow-up study that purported to back up the first). But none of those dissections comes close to a damning new statistical analysis of the 2004 study authored by David Kane, Institute Fellow at the Institute for Quantitative Social Science at Harvard University. I read of Kane’s new paper at this science blog and e-mailed him for permission to reprint his analysis in its entirety here so that a wider blog readership could have a look. He has given me his permission and adds that he welcomes comments and feedback. He’ll be presenting the paper at the Joint Statistical Meetings in Salt Lake City on Monday — the largest conference of statisticians in North America.

Much of the math here is mind-numbingly complicated, but Kane’s bottom line is simple: the Lancet authors “cannot reject the null hypothesis that mortality in Iraq is unchanged.” Translation: according to Kane, the confidence interval for the Lancet authors’ main finding is wrong. Had the authors calculated the confidence interval correctly, Kane asserts that they would have failed to identify a statistically significant increase in risk of death in Iraq, let alone the widely-reported 98,000 excess civilian deaths.

An interesting side note: as Kane observes in his paper, the Lancet authors “refuse to provide anyone with the underlying data (or even a precise description of the actual methodology).” The researchers did release some high-level summary data in highly aggregated form (see here), but they released neither the detailed interviewee-level data nor the programming code that would be necessary to replicate their results.

Kane has sent his paper to Lancet. But the blogosphere need not wait for Lancet to complete its review. If you’ve got a statistics background or know someone who does, have a look. Kane’s e-mail address is dkane-at-iq.harvard.edu. He’s a blogger himself at EphBlog.

2 posted on 07/25/2007 3:21:07 PM PDT by coloradan (Failing to protect the liberties of your enemies establishes precedents that will reach to yourself.)

The following is an example of the typical journalist’s level of math/statistics knowledge.

“California transportation officials recently announced that 30% of traffic fatalities involved drivers under the influence of alcohol or drugs.”

It is therefore imperative that we rid the roads of the 70% sober people causing the vast majority of accidents.

The Lancet was once a major medical journal. Now it is third rate political propaganda.

It is totally unheard of in true science to “refuse to provide anyone with the underlying data (or even a precise description of the actual methodology).”

This is Al Gore type global warming “research.”

6 posted on 07/25/2007 4:16:38 PM PDT by FormerACLUmember (The ideal tyranny is that which is ignorantly self-administered by its victims.)

Cumulative US deaths month by month in Iraq	Cumulative US deaths month by month in Korea
65	2,849
138	4,603
175	7,948
205	8,334
252	11,809
287	13,963
317	14,620
360	16,688
442	17,574
482	18,646
529	19,807
548	20,613
600	20,927
747	21,431
835	22,825
879	24,484
940	25,015
1,011	25,236
1,095	25,442
1,163	25,589
1,303	25,764
1,376	25,990
1,484	26,191
1,546	26,675
1,582	27,125
1,634	27,537
1,716	28,156
1,793	29,173
1,848	29,475
1,933	29,672
1,987	29,845
2,083	30,088
2,171	30,629
2,238	30,939
2,303	31,138
2,359	31,609
2,392	32,615
2,473	32,622
2,544	32,627
2,605	32,690
2,656	32,742
2,726	33,102
2,802	33,233
2,910	33,362
2,986	33,445

7 posted on 07/25/2007 4:55:16 PM PDT by syriacus (If the US troops had remained in S. Korea in 1949, there would have been no Korean War (1950-53).)

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.