Posted on 10/28/2002 5:57:01 PM PST by Sabertooth
Edited on 07/12/2004 3:39:24 PM PDT by Jim Robinson. [history]
Michael Bellesiles, the history professor who wrote that firearms were rare in early America, has resigned from Atlanta's Emory University after an investigation found he "willingly misrepresented the evidence"in his award-winning book.
The three-person committee
(Excerpt) Read more at washingtontimes.com ...
Here is the only section of the report where I found the word "randomly" used:
Unfortunately, this seemingly randomly gathered information later took on a life of its own. He appears carelessly to have assumed that his counts were complete, and moved forward with a larger project on the basis of this unsystematic research that appears to have involved dipping into rather than seriously sampling the records. While this certainly constitutes sloppy scholarship, it does not prove a deliberate attempt to mislead, however misleading the result.
Whereas the article from the Washington Times summarized that passage (and possibly other parts of the report) in this way:
The committee's report, written by professors Stanley Katz of Princeton, Hanna Gray of Chicago and Laurel Thatcher Ulrich of Harvard, found Mr. Bellesiles guilty of "sloppy scholarship," using "randomly gathered information" and "unsystematic research" in reporting data from Vermont probate records.
So, I don't believe that they are ignorant of the role of sampling in data collection.
The dog ate my homework...
That is EXACTLY what I thought when I read that. Here is the original Sept 11 NRO article:
Disarming AmericaRead the Northwestern study. It slams this guys finding so hard I can't believe he even stands around with his head up.
One of the worst cases of academic irresponsibility in memory.By Melissa Seckora, NR editorial associate
October 15, 2001 issue
Originally posted September 11, 2001The power of image and myth repeatedly overwhelms reality in discussions of early American firearms," writes Michael A. Bellesiles, a professor of history at Emory University, in his book, Arming America: The Origins of a National Gun Culture. "America's gun culture is an invented tradition."
Bellesiles has done some inventing of his own. His book — which won this year's Bancroft Prize, the most prestigious award in the writing of American history — is one of the worst cases of academic irresponsibility in memory. Yet his remarkable performance has not been enough to cause either his employer, Emory, or his publisher, Knopf, to act.
Some of the most significant statements in Arming America are "based" on data that simply do not exist. In other words, they are based on nothing. For example, Bellesiles claims to have counted guns in probate records of the estates of people who died in 1849 or '50 and 1858 or '59 in San Francisco. The problem is that, according to everyone who should know, all the probate records that Bellesiles allegedly reviewed were destroyed in the 1906 San Francisco earthquake and fire. Rick Sherman of the California Genealogical Society has said, "I am unaware of the existence of any surviving San Francisco probate files for 1849-1859. If this involves an out-of-body experience, I'd like to know how to pull it off."
The thesis Bellesiles sets out to prove is that there were very few guns in early America, let alone a gun culture, and that most of the guns that did exist were old and broken. He published an article on the subject in 1996, in the Journal of American History — a piece that won "Best Article of the Year" from the Organization of American Historians. Five months before the book came out, Anthony Ramirez of the New York Times promoted Bellesiles's thesis, presumably based on the earlier, award-winning article. He said that the probate records were the author's "principal evidence." John Chambers, a military historian from Rutgers, reviewed the book for the Washington Post, saying that the probate records were Bellesiles's "freshest and most interesting source." Edmund Morgan, one of the country's leading historians of colonial America, followed suit, exclaiming in the New York Review of Books, "The evidence is overwhelming. First of all are the probate records."
Certain historians — especially those with a strong quantitative bent — were skeptical of Bellesiles's data, but many others, including Garry Wills, uncritically embraced Arming America, most likely because it appeared to confirm what they have long wanted to believe: that the Second Amendment protects only a collective right to bear arms, that individual gun rights were deemed unimportant at the time of the writing and ratification of the Constitution.
One of those who were skeptical was Joyce Malcolm, a history professor at Bentley College who has written a book on the Anglo-American conception of gun rights. She says, "Bellesiles fails to provide even basic information about the probate figures that form the basis of his claims for the rarity of guns. And he repeatedly makes general statements that are extreme. But if you check his footnotes, a more disturbing pattern emerges. It is not just an odd mistake or a difference of interpretation, but misrepresentation of what his sources [if they exist] actually say, time after time after time."
In documents on his website, and in correspondence with other scholars, Bellesiles has not only reasserted that he used records in San Francisco, he has embellished his story by giving a location: the Superior Court. In a long interview I conducted with him, he confirmed more than once that the archives he used were at the Superior Court, and said further that he had read hundreds of probate records in both San Francisco and Los Angeles. "There were only a few hundred cases, but that's a lot of cases."
When I asked him point-blank whether he had used probate records from San Francisco County, he answered: "I used county probate records from the Superior Court. I had to go the courthouse — the San Francisco Superior Court." But the deputy clerk of the court, Clark Banayad, says flatly, "Every record at the San Francisco Superior Court predating 1906 was destroyed by fire, or other causes, in the 1906 earthquake." This is common knowledge among California probate and genealogical authorities. The probate records cannot be found at the History Center of the San Francisco Public Library either. Librarian Susan Goldstein says that the center stores some archives of the city and the county, including property deeds, general indexes, and contracts, but that it has "no record of the number of guns owned in the city of San Francisco prior to the 1906 earthquake." Kathy Beals, author of three books on San Francisco's early probate records, says that "to my knowledge, there are no official probate files in existence for years prior to 1880, and only scraps from 1880 until 1905."
I then informed Professor Bellesiles that the probate records could not be found at the San Francisco Superior Court. He changed his story: "Did I say San Francisco Superior Court? I can't remember exactly. I'm working off a dim memory. Now, if I remember correctly, the Mormon Church's Family Research Library has these records. You can try the Sutro Library, too."
Bellesiles appears to have been in error yet again. According to Martha Whittaker, a reference librarian at San Francisco's Sutro Library, the records do not exist. "All official probate records were destroyed by the San Francisco earthquake and fire because the city hall burned down." As for the Family Research Library, Elaine Hasleton, supervisor of public affairs, says that the library has an index of all estates in probate in the city and county of San Francisco from 1850, but that the index does not list information about gun ownership. "The index only lists names and the locations of the actual probate records. It does not list possessions."
Curiouser and Curiouser
The San Francisco Superior Court, the Sutro Library, and the Mormon Church's Family Research Library are not the only places that do not have an archive that Bellesiles purports to have used. In correspondence last year, Bellesiles assured would-be replicators of his research that for all but "several" of the 40 counties he examined, he did his probate research via microfilm at the Federal Archives in East Point, Ga. He wrote: "The probate records are all on microfilm in the National Archives. I went to the East Point, Georgia, federal center to read these microfilms." When it turned out that the archives in East Point did not have these materials, Bellesiles admitted that he might have been mistaken. His new story was that he went around the country doing most of his probate research in over 30 different county or state archives looking at original records, not microfilm of the original records, as he had first claimed. One wonders how he could have forgotten whether he did his research in a single library near his office or in more than 30 archives around the country.It could have something to do with the fact that his "database" consisted merely of tick marks on pads of paper. "For the research in the book," he said in our interview, "I was going through sample years, counting numbers of inventories and checking off how many had guns with cross hatches. It was very rough reading of very difficult documents." Bellesiles claims to have counted 11,170 probate inventories; his was, at a minimum, a rather unorthodox and not very modern system for recording great amounts of historical information. What's more, Bellesiles says that all of his tick-mark notes got wet in a flood in his office and had to be discarded, leaving him with no documentation at all to back up his probate study. "All of my notes were destroyed in boxes on the floor of my office, and they got drenched, pulped."
James Lindgren, a Northwestern University law professor, criminologist, and probate expert, trained in quantitative methods, and his coauthor, attorney Justin Heather, have identified many other errors in Bellesiles's work in their paper "Counting Guns in Early America." They found that Bellesiles purported to examine about a hundred wills in Providence, R.I., that simply do not exist, and that he also misused records that do exist: repeatedly counting women as men, counting as old or broken guns that were not so listed, and claiming that there were a "great many" state-owned weapons when there was only one.
Lindgren and Heather have also determined that Bellesiles's main 18th-century probate contentions are mathematically impossible. In his book, Bellesiles reports a national mean for the period 1765-90 of 14.7 percent of probate inventories listing a gun, when every other published probate researcher — including Anna Hawley, Gloria Main, Alice Hanson Jones, and Harold Gill — has found three to five times that many. "It's a simple sixth-grade math problem, computing an overall mean from samples," says Lindgren. He and his coauthor elaborate on this at length in their paper. Oddly, Bellesiles has never disputed any of the pair's main charges. The closest he has come so far to admitting any wrongdoing was in a letter he sent to the Wall Street Journal (which had reported critically on his case): "I have come to feel that the use of my sample sets is not as statistically sound as I would like."
Other scholars who have looked into the matter concur that Bellesiles's allegations are false. Lindgren "utterly devastates Bellesiles's research," says Albert Alschuler, a law professor at the University of Chicago and author of a recent biography of Oliver Wendell Holmes. Randolph Roth of Ohio State University, the leading expert on early-America homicide rates, has looked at Lindgren and Heather's counts of guns in the Providence inventories and found them correct. A third scholar, UCLA's Eric Monkkonen, one of the country's foremost quantitative historians, says, "Lindgren's data show that Bellesiles was not correct."
Despite all this, the book's editor, Jane Garrett, still defends the book. Knopf declined to answer questions, but in a written statement Garrett said, "Hosts of reputable scholars continue to defend [Bellesiles's] methods and his conclusions. Controversies of this nature are not uncommon in the historical profession. That's what makes history so interesting."
Yet it is not so "interesting" as that. Bellesiles suggested that I speak to ten professors who knew his work, of whom I was able to speak to six. All were either negative about his work — though stopping short of accusing him of outright fraud — or cautious in their praise. And almost none of those who continue to support Bellesiles has bothered to explore the criticism of him. None could identify a single criticism as false. In fact, when I asked Bellesiles himself to point out such a criticism, he could only identify a few criticisms that, on examination, proved true. In general, my investigation of this case did not support the impression of some — such as Knopf's Garrett — that there are distinguished scholars on both sides who have carefully gone over the challenged evidence. Those who have taken the trouble have Bellesiles dead-to-rights.
Saul Cornell of Ohio State is one of the historians whom Bellesiles suggested I contact. He has looked into criticism of the author, but could not identify any particular errors in it. Cornell says that some of the criticism has been "hysterical [and] ideologically driven," while other criticism — such as Lindgren and Heather's — has been "thought-provoking [and] powerful": "We are all waiting to see what Michael's response to it will be. Even if he is proven wrong, it is possible to write an important book that moves the debate forward, even if it is flawed. We are all in Bellesiles's debt for opening the debate."
Yet Stanford's Jack Rakove, winner of a 1997 Pulitzer Prize in history, takes a much different view. A historian's claims, he says, ought to be verifiable: "Other scholars should be able to go to the archives to see whether you've quoted them accurately and used them correctly. Bellesiles will have to defend himself. If he can't do it, we will know. If he has really screwed up, he will be held accountable."
Randolph Roth, who has also found several major problems with Bellesiles's homicide counts (which he will discuss in a forthcoming article in the William & Mary Quarterly as part of a forum on Arming America), agrees: "If I am wrong, I want you to be able to fix my mistakes. The only way that can happen, however, is if I share my data and make clear exactly what sources I looked at. Michael hasn't done that yet. That is the big problem."
Those who have examined Arming America have documented hundreds of possibly intentional misconstructions of sources or outright falsehoods. The probate records aside, Bellesiles has egregious problems in the areas of homicide data, gun censuses, reports on militia arms, hunting accounts, travel accounts, the opinions of the anti-Federalists, and laws governing guns. So far it appears that Bellesiles has not been able to validate any of the challenged portions of his book. Indeed, in a new paperback edition of Arming America being released this fall, he has quietly capitulated on some of the critics' claims, retracting or altering the challenged statements about probate data from Providence.
A Career Sails On
Bellesiles told me he plans on spending the next ten years or so reconstructing the information he lost in the flood and posting it on his website. "That is what I am working on now — it is my hobby," he says.Meanwhile, his university position seems secure, and he continues to win academic plums. Walter Adamson, outgoing chairman of the Emory history department, told me, "Nothing has come forward to me that indicates that Bellesiles deserves to be treated or included in that company of people who have been judged to fabricate their sources." He said this, however, before the revelation of Bellesiles's San Francisco inventions.
And what about Knopf? Says one scholar, "What has happened to ethics [there]? They have allowed a rogue author and an editor of apparently compromised judgment to go forward unchecked. If Knopf doesn't know that it is publishing false data, it is only because it chooses not to know. The very image of Knopf as a serious publishing house has been compromised."
One could only imagine the outcry if a conservative scholar, fabricating evidence to prove a pet conservative point, had been found to be careless (to say the least). If the purpose of historical scholarship is to construct a past that is congenial to oneown biases, then Michael A. Bellesiles can be judged a success. But that is not, of course, the purpose of historical scholarship. Those who understand that can take heart: Arming America has been disarmed by its own dishonesty.
Additionally, you will find some mention of last years devestating 3 of 4 articles in William and Mary Quarterly, one of THE history journals that slammed him after the controversey could be looked at in depth.
The only link I know to those scholarly re-examinations is through historycoop.org and the reviews are near the bottom of this table of contents page HERE for those who want to grab a copy.
The debunking and conviction of this fraudualant propoganda book is now more volumuous than the book itself.
One more lesson for me not to trust the newspaper reporters: they cannot even quote faithfully.
I agree completely with your conclusion: the authors do speak intelligently about data gathering techniques.
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.