Gunning for John Lott

JOHN LOTT is being accused, of well, something. The complaint doesn't run to his published scholarly work, but to public statements he's made about a survey whose results were never published. Some people are now saying that he never conducted the survey at all. Here's an email from James Lindgren from a list that I also belong to, posted by vociferous Lott critic (and, if I recall correctly, erstwhile Bellesiles defender) Tim Lambert, that lays it out at some length.

I've been following this on that list for months (it goes back before that email) but haven't posted on it because (1) I thought it would violate list etiquette; and (2) I expected a response from Lott that would lay it all to rest.

But now -- as a recent email to the list from Eugene Volokh points out -- the story has broken out into the Blogosphere. Here's a post by Jim Henley, here's one by Julian Sanchez, and here's another by Marie Gryphon.

And no satisfactory response by Lott has been forthcoming. I don't know what to say about that. Lott's critics want, rather too obviously, for this to be another Bellesiles affair, though to my mind it is, even if the accusations pan out, something less than that, perhaps more akin to the Joseph Ellis scandal. And I can't help but feel that there's going to be a strained effort to turn every criticism of every bit of non-PC scholarship into a reverse-Bellesiles affair for a while, as the lame effort to draw a Lomborg-Bellesiles connection seems to demonstrate.

But if the charges against Lott are true -- and thus far, the evidence is suggestive, not dispositive -- it's a serious matter indeed even if it's not of Bellesiles caliber. The only one who can really clear this up is John Lott,. If he fails to do so, well, under the facts of the matter it will be difficult for anyone else to prove anything, but many will choose to draw unflattering conclusions.

As I proofed the above I checked, and Clayton Cramer has a post on this, too. He reports that Lott has repeated the 1997 study now, and posts a letter from, and a summary of a phone call from, John Lott.

UPDATE: Tim Lambert, who has been the main figure driving this matter, has a page rounding up weblog coverage and also offering this observation:

Bearing in mind the source of this statement, to which I have added emphasis, I think that those who are too anxious to turn Lott into another Bellesiles should exercise caution.

ANOTHER UPDATE: As I plow through the built-up emails in my office account, it's obvious that Jim Lindgren is investigating this matter rather thoroughly. Stay tuned.

I don't think that most people have actually read Lott's side of this account. Extra Ordinary Ideas

http://www.mariegryphon.com/archives/003393.html#003393

UPDATE: Dr. Lott has given me permission to post the following comments, which I quote:

"Dear Everyone:

Here is a response to some of what has been going on over the web. I have already sent much of this information to people who have already contacted me in person. If Eugene would like to post this on his web site, I must ask that all the e-mail addresses and telephone numbers be removed. If you all don't trust the leg work done by Dan Polsby on this issue since Christmas, you can nominate someone else to go and do it, but I don't think that it is appropriate for everyone from Lambert on to go and harass these people.

Regnery (the publisher of my new book due the middle or end of March) wants me not to release the results from the poll last year. They want me to keep quite about the book until it comes out. As has been reported previously, the survey was done with similar questions in a very similar way to what was done earlier and the results were essentially the same. I will check with the publisher about releasing this data early, but it is still two months before the book is due to be published. In the interim, I am sure that I could arrange it so that interested parties could question the person who keep the survey results as they came in to confirm that we only got one person who said that they had actually fired a gun and that the rest were brandishings.

Here are some of the things that I have done to try to establish a record of events. 1) My wife contacted the bank that we had in Chicago and tried to get copies of bank statements and checks from the period of time. Unfortunately, the bank does not keep copies of statements or checks longer than five years. (If you would like to verify, we talked to Yvonne Macias in the book keeping department at University National Bank.) Lindgren does not accurately report my conversation with him about how I paid people (in that I said that I possibly paid by check), but this information makes that point irrelevant. 2) I asked Sam Peltzman last year about whether the Alumni Association has the e-mail of past students. Sam, who seems to know virtually everything that is going on at the University, told me that they have the e-mail addresses for at most 10 percent of the former students. 3) I had a former alumni and several time co-author, John Whitley, placed in an ad in the Alumni magazine in the December issue to track down the students. I don't know if the ad has appeared but thus far I have gotten no response.

I have given out massive amounts of data to people on the guns and other issues, and I will be happy to do so on the new survey. Data has been given to critics as well as people who have been unwilling to share their own data on other projects. I have given out county, state, and city level crime data to academics at dozens of universities, with data sets ranging from 36MB to over 300MB. I have given out data on multiple victim public shootings as well as safe storage laws. These different data have often been given out before the research is published and sometimes even before it has been accepted for publications. We are not talking about recent events or conversations and there is a question about what is a reasonable time period for people to keep records. There is also a question as to why people have waited so long to ask for this additional information when people have known about the lost data for years.

As to the claims about 'apparently changing positions,' I disagree. I have told people directly (including Otis Duncan) from the beginning that the data were lost. Op-ed pieces and other public statements where I mention these numbers briefly usually do not lend themselves to discussions of the sources of numbers. The fact that David Mustard does not remember exactly when we discussed the survey 6+ years ago does not surprise me given how long ago this was.

Unfortunately, there are many problems with Lindgren's write up. He gives essentially uncritical acceptance of Otis Duncan's discussion of events in 1999. Yet, while Lindgren writes that 'Otis Dudley Duncan raised questions about the 98% figure . . . after exchanges between Lott and Duncan,' Duncan's write-up in the Criminologist news letter failed to mention any such possible discussions. In fact his newsletter piece leaves the opposite impression as he endlessly speculates about what I may have meant about certain statements. My response in the Criminologist also discussed other incorrect claims by Duncan.

As to the attribution of sources, look at the complete context of the quote Lindgren mentions:

Polls by the Los Angeles Times, Gallup and Peter Hart Research Associates show that there are at least 760,000, and possibly as many as 3.6 million, defensive uses of guns per year. In 98 percent of the cases, such polls show, people simply brandish the weapon to stop an attack. -- August 6, 1998, Chicago Tribune and August 14, 1998, Washington Times

References by Lindgren to things like the Linnet Myers piece in the Chicago Tribune to provide evidence that I didn't do a survey or that I have changed my statements over time are simply bizarre. Attached below is an edited down version of the letter that was published by me in the Tribune. Myers used her article to refloat claims such as my Olin Funding, inaccurately reported exactly what the concealed handgun research covered, and claimed that 'others haven't confirmed (my) findings.' I no longer have the original letter to the editor, but as I recall this is just a partial listing of her inaccurate statements. The Tribune was not willing to run a longer letter, though the letter that they ran was quite long.

As to so-called technical problems, I am have always acknowledged that theseare small samples, especially when one breaks down the composition of those who use guns defensively. Even the largest of the surveys have few observations in this category. The attached e-mail that I sent to Glenn Reynolds goes into this more in depth.

'No direct evidence of survey' discussing Lindgren's point-by-point discussion of our conversation

1) 'No funding for the project' I regularly have paid for research myself. Sometimes large amounts of money have been spent, but it is not uncommon for me to spend several thousand dollars. On the paper on multiple victim public shootings, I know that one payment that I made to Kevin, a research assistant to Landes and Posner, was $750. I paid for the special issue of the JLE in 1999 on sentencing myself, and the special issue and part of the conference cost me around $30,000. I have not applied for funds from outside sources over the years.

2) 'No financial employee records' This is not unrelated to the first point. Incidentally, I told Jim that there were 'two' Chicago students. Those students had also gotten others that they knew from other campuses from places such as I think the University of Illinois at Chicago circle (but I am not sure that I remember this accurately). What I told him was that I remembered that one of the two University of Chicago students was a senior.

3) 'calling was done by the undergraduates from their own phones.' most of this next statement is correct except the point about the 'possible' use of checks. But as noted earlier this point is irrelevant in terms of evidence.

4) 'does not remember names' I have had 12 interns and RAs just since I arrived at AEI. This excludes people whose only work was on the survey. I am horrible at names and I couldn't even give you the names for all of these folks let alone people who did something six years ago. All my names and addresses for everything were on my computer when the hard disk crashed.

5) 'no discussions with any samplers'

I had lunch Tom Smith during the fall of 1996. However, while I asked him many questions about surveys, I did not tell him what I was planning on doing because Tom works very closely with gun control organizations.

6) weighting the sample

I did not weight the sample by household size but used the state level age, race, and sex data that I had used in the rest of my book. There where 36 categories by state. Lindgren hypotheses why you can get such small weights for some people and I think that this fine of a breakdown easily explains it. I don?t remember who answered what after all these years, but suppose someone who fired a gun was a elderly black in Utah or Vermont.

7) 'commercially available CD-ROM with names on it. He does not remember where he got it from.'

It is true that I don't have the original CD-ROM. I have a telephone number CD from the end of 1997, but it is not the one that we used. I only picked up the other one on the off chance that I was going to have the time and resources to redo the lost data. The CD did have the features that the earlier one had and was not very useable. I was so rapped up in trying to replace my lost data on so many other projects that I had no thought of going back to what I regarded as a minor project. I had revise and resubmits at the JPE and other journals that had much greater importance and the data for the book had to be replaced.

8) 'Lott does not remember how he drew his sample from the CD-ROM'

Not true. I told Jim that one of the students had a program to randomly sample the telephone numbers by state. My guess is that it was part of the CD, but on that point I can?t be sure.

9) 'doesn't remember the wording of the questions.'

It is also not quite correct to say that 'doesn't remember the wording of the questions.' I told Jim that I don't remember the 'exact wording' of the questions, but I gave him the general outline of the questions.

10) more on weighting

See point 6 above.

11) 'A chapter he had not yet written'

This is not correct. What I had done is write up the section, but I only had a computer file of it. When the hard disk crashed, I only had a hard copy of the book and I had to spend considerable time scanning in the book and correcting the new file. I was unable to replace the lost polling section that I had recently added. I didn't think that it was worthwhile relying solely on memory for different things and I had too much else to do to concern myself with something that wasn't central to the book.

12) 'did not retain any of the tally sheets'

I have looked through some things but I haven't found anything. As Lindgren correctly notes, I have moved three times in the last six years.

13) Sheets versus entry of data into computers

Lindgren has the 'impression' that the students entered the data on sheets. I do not directly recall this part of our conversation, but I would have said that both were done.

I sent Lindgren two e-mails on December 26th. Just so no one accuses me of adding new things in now, one of my e-mails to Lindgren noted: 'I did not take the time to correct or respond to all the issues raised, but I wanted to mention a few points.' Recent e-mails to Lindgren have also already responded to some of these points beyond the e-mail that he apparently posted.

I have not participated in the firearms discussion group nor in the apparent online newsgroup discussions, but what I have done is respond to e-mails. (The one exception are those from Lambert whose e-mail address was placed on my blocked list.) If you all have questions, I will be happy to discuss them, but I am not going be involved in these online groups. My response to Glenn below goes through some of the history of what I heard on this and when I heard it. The bottom line is that you all should not assume that everyone participates in these discussions."

Please Note: Dr. Lott's email also included a clipped letter to the editor of the Chicago Tribune published on June 20, 1999, which should be accessible through public sources, and shortened version of another, slightly older and mostly duplicative email also reviewing his account of events and discussing his new 2002 study. I'd post it all, but this is a heck of a long blog entry. Go ahead and email me at marie@mariegryphon.com if you must have the balance. I'll supply a link if I find any additional comments elsewhere. Posted by Marie Gryphon on January 14, 2003 | Comments (0) | TrackBack (0)

January 10, 2003 Tangled Webs

Defenders of gun rights were rightly unstinting in their criticism of Michael Bellesiles, the former Emory professor who was found to have fabricated evidence for his book, Arming America. Unfortunately, a time may also arrive when they are obligated by their high standards of academic honesty to censure one of their own. A controversy that has festered on academic email discussion lists for some time may finally be coming to a head.

Dr. John Lott, Jr., author of More Guns, Less Crime - a groundbreaking study of the relationship between concealed-carry laws and crime rates - has been accused in some detail of fabricating a survey in order to support his oft made claim that merely brandishing, rather than firing, a firearm will scare off an attacker 98% of the time.

Dr. Lott made this claim in the first edition of More Guns, Less Crime, published in 1998, citing only “national surveys.” He wrote at page 3, “If national surveys are correct, 98 percent of the time that people use guns defensively, they merely have to brandish a weapon to break off an attack.” According to critic Tim Lambert, Dr. Lott has made reference to the 98% statistic at least 48 different times.

During this same period, other commentators were also citing a 98% figure. They were apparently attempting to cite a study on defensive gun use published by Florida State University criminologist Gary Kleck. However, they were forgetting that the 98% figure cited by Kleck included defensive gun uses involving warning shots and missed shots at an attacker as well as mere brandishing cases.

When confronted with a critique of the 98% figure, individuals including C.D. Tavares have explained their misinterpretation of Kleck’s work and apologized. Dr. Lott, though, began attributing the 98% figure to a national survey he now says he personally conducted in the year 1997. In the second edition of More Guns, Less Crime, he offers the following alteration on page 3: “If a national survey that I conducted is correct, 98 percent of the time that people use guns defensively, they merely have to brandish a weapon to break off an attack.” (emphasis added)

Dr. Lott’s accusers suggest that he was unwilling to admit that he had misinterpreted Kleck’s study, opting instead to credit a never-before-mentioned national survey of his own for the 98% figure. Wrong as it would have been, one can certainly imagine an author of a 321-page book convincing himself that a small sleight of hand on page 3 would never be questioned. If the naysayers are correct, though, Dr. Lott significantly worsened matters for himself in 2000, when he responded in The Criminologist to criticism from U.C. Santa Barbara’s Otis Duncan with a detailed description of the survey Lott supposedly conducted in 1997, and its findings. Dr. Lott apparently discussed the details of the survey over the telephone with Duncan as well.

Having reviewed credible-sounding critiques of Dr. Lott’s description of his survey by Lambert and Duncan, Northwestern University Professor James Lindgren undertook to investigate whether the survey actually took place. It is Lindgren’s report on his efforts, dated December 24, 2002, that is most concerning.

Dr. Lott claims that his telephone survey was conducted over three months in 1997, and garnered responses from “2,424 people from across the United States.” Because many called parties fail to answer calls or refuse to respond to survey questions, it is certain that thousands more calls would have been made in order to generate 2,424 responses.

But Lindgren reports that Dr. Lott had no funding for this survey, and says he covered the expenses out of his own pocket. Dr. Lott claims to have used student volunteers to help him place thousands of phone calls, but lacks any records listing student participants, and cannot remember the name of even a single volunteer. Dr. Lott says he had the students make these phone calls on their own home telephone lines, reimbursing their long distance charges from his personal account, but that he cannot prove this because he discards his cancelled checks after three years. Dr. Lott apparently told Lindgren that he did not recall discussing this survey project with any colleagues at the University of Chicago.

As for the data itself, Dr. Lott apparently told both Lindgren and Duncan that he cannot produce the data because it was lost in a computer crash in 1997. He apparently explained at least once that survey data was entered directly into students’ computers, and then electronically transferred to his computer during the survey process, but apparently no student retained copies of the data allowing him to reconstruct the survey following his computer crash. He has also apparently suggested that some handwritten data collection may have occurred, but that any handwritten survey results must have been inadvertently discarded when he moved out of his office at Yale University.

Dr. Lott is an exceptionally bright man who has conducted some of the most important research to date on criminological issues related to gun control. If he did indeed conduct the 1997 survey – as I hope he did – he should proactively work with others to find at least one of the graduate students who assisted him, obtain his old bank records and their old telephone records. In short, he should take the time right now to set this controversy at rest.

If Dr. Lott’s accusers are correct, immediate action is even more important. In very little time this issue will find its way into the mainstream media. If Dr. Lott has lied to his colleagues he must now tell the truth, difficult as that might be. The alternative will surely transform a journalistic footnote into a media circus.

I also looked up the CHicago Tribune letter

Chicago Tribune

June 20, 1999 Sunday, CHICAGOLAND FINAL EDITION

SECTION: MAGAZINE; Pg. 4; ZONE: C; LETTERS TO THE EDITOR.

LENGTH: 684 words

HEADLINE: GUNS AND CRIME

BODY:

The article accompanying "Anne, Get Your Gun" (May 2), discussing my book "More Guns, Less Crime" (University of Chicago Press, 1998), made several inaccurate claims.

Despite the claims in the article, my research looked at much more than just the "impact of laws that allow guns to be carried outdoors." My book analyzed FBI crime statistics for all 3,054 American counties from 1977 to 1994 as well as extensive cross-county information on accidental gun deaths and suicides. This is by far the largest study ever conducted on crime, accidental gun deaths or suicide. I examined not only concealed-handgun laws, but also other gun-control laws such as state waiting periods, the length of waiting periods, the Brady law, criminal background checks, penalties for using guns in commission of crime and the impact of increasing gun ownership. The only gun laws that produced benefits were those allowing concealed handguns. The evidence also strongly indicates that increased gun ownership on net saves lives.

More disappointing were inaccurate references to the funding of my research. The claims previously floated by gun-control groups like Handgun Control were found by the Tribune's own Steve Chapman to be false (Aug. 15, 1996). Chapman pointed out that not only was the Olin Foundation "independent" of the ties the Sunday Magazine article discussed, but also that the "foundation didn't (1) choose Lott as a fellow, (2) give him money or (3) approve his topic."

The article's claim that "others haven't confirmed (my) findings" is bizarre. To date, I have made the data available to academics at 37 universities, from Harvard to Berkeley. Everyone who has tried has been able to replicate my findings, and only three have written pieces critical of my general approach. Although the vast majority of researchers concur that concealed weapons deter crime, not even those three critics have argued that more guns cost lives or increase crime.

-- John R. Lott Jr., University of Chicago

Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.