"FBI Bullet Tests Seriously Flawed" (but still used, apparently)

FOX News

[You Gotta Be Kidding Me]

Study: FBI Bullet Tests Seriously Flawed

Friday, November 21, 2003

WASHINGTON — In a finding that could affect thousands of criminal cases, the National Academy of Sciences has concluded that some techniques the FBI has used for decades to match bullets to crimes are flawed or imprecise.

The study, expected to be released in the next few weeks, makes about a half-dozen recommendations to improve the FBI lab's science used to match bullets through their lead content.

The academy's findings, which are in final draft form, were described to the Associated Press by several people involved in the study. They would speak only on condition of anonymity.

The study specifically urges the bureau's chemists to stop a practice known as "data chaining" that chemists have used in the past to match bullets to a crime.

In data chaining (search), scientists can conclude that if the lead content of bullet A matches bullet B, and bullet B's content matches bullet C, then it is safe to testify that bullet A and bullet C are a match even if their test results don't match identically.

Put another way, the FBI can match two dissimilar bullets if they can find a third — from a manufacturer, for instance — that comes close to matching both.

The FBI science relies on the theory that bullets from the same batch of lead share a common chemical fingerprint.

Charles Peters, an FBI expert witness in cases involving bullet lead comparison, testified recently that data chaining — the technique disavowed by the academy — was important to matching bullets.

"I'm a fan of chaining," Peters testified in April in a case in Alaska. "If ... we didn't do something like chaining, or something like that, nothing would ever match."

A reference in the latest draft of the academy report indicates the FBI may abandon the data-chaining technique, the sources said. FBI officials said Thursday night they had not seen the report and could not comment on it.

National Academy of Sciences (search) spokesman Bill Kearney also said he could not comment on a "draft report that is still being peer-reviewed and subject to change."

Citing specific examples of conflicting or inconsistent testimony by FBI experts, the study also recommends that lab analysts' work and testimony be reviewed by a peer to ensure accuracy and precision, the sources said.

The FBI lab's director has been trying to increase the number of peer reviews inside the lab.

The academy's recommendations are likely to have a huge impact, opening the door for appeals from defendants convicted in past cases where bullets were matched by the FBI using lead analysis. It also could force FBI lab witnesses to more narrowly describe the statistical significance of their findings in future cases.

The FBI has been the prime practitioner of bullet-lead comparisons in the United States, and has used it for decades, dating to around the time of President Kennedy's assassination 40 years ago. A database of lead test results kept by the agency had more than 13,000 samples in the late 1990s, FBI officials have told the AP.

The FBI most commonly identifies bullets recovered from a crime by firing new bullets from the suspect's weapon and comparing the markings left by the gun barrel on the test bullet with the crime scene bullet. But that method only works when the crime scene bullet is in good shape or if police have the suspect weapon.

In cases where recovered crime scene bullets are fragmented or disfigured or a suspect's weapon is unavailable, the FBI has turned to chemical analysis to try to determine whether the bullet's lead content is comparable to the same manufacturer, lead source or box of bullets connected to the suspect.

When the lab makes a match, its experts testify that two bullets are "analytically indistinguishable."

FBI Lab Director Dwight Adams earlier this year asked the academy to review the lead bullet identification process after one of the bureau's most respected metallurgists, after he retired, began openly challenging his former employer's science.

The FBI paid for the study by the academy, which is one of the nation's premiere scientific institutions.

The former FBI metallurgist, William Tobin, and his colleagues have published research stating that bullets from the same lead source had different chemical makeups and bullets from different lead sources appeared chemically similar, challenging the very premise of the FBI's science.

Testifying as a defense expert, Tobin has cited evidence that FBI lab experts have testified in conflicting manners about how lead composition can identify bullets and link them to criminals.

Iowa State University (search) has conducted research that drew similar conclusions.

"The fact that two bullets have similar chemical composition may not necessarily mean that both have the same origin. ... The leap from a match to equal origin is enormous and not justified given the available information about bullet lead evidence," Iowa State researchers reported.

[You Gotta Be Kidding Me]

"The study specifically urges the bureau's chemists to stop a practice known as "data chaining" that chemists have used in the past to match bullets to a crime."

All the "urging" in the world won't stop it. It will take a prominent case where the evidence is thrown out, before the practice will stop.

Take notice of this report, all you "law & order at any cost" advocates.

[templar]

...scientists can conclude that if the lead content of bullet A matches bullet B, and bullet B's content matches bullet C, then it is safe to testify that bullet A and bullet C are a match even if their test results don't match identically.

If A=B and B=C but A<>C then there is something seriously (scientifically) wrong with the definintion of 'matches'.

[joesnuffy]

good thing they proved Oswald acting alone killed kennedy...back in the old days when labs and tests were more accurate huh!

[Comus]

Wait a minute. I thought bullets came out of guns with serial numbers that matched the guns. This has to be a lie. Why else do they collect fired bullets from guns?

[You Gotta Be Kidding Me]

"If A=B and B=C but A<>C then there is something seriously (scientifically) wrong with the definintion of 'matches'."

For fingerprints, you need twelve points where print A precisely overlays print B, before you can say in a court of law that the prints "match". Even now, however, fingerprint "matching" is comming under fire. Also 2 in 10,000 people have "the same" (scientifically) fingerprints.

Bullet rifling marks are a whole 'nutha matter. This puts a real damper on the "balistic fingerprint" database crap, doesn't it.

[Travis McGee]

BANG!

[Dark Wing]

You are going to have some real fun with this one.

[Servant of the 9]

This is a matter of tolerance stacking, something that always must be avoided in science and engineering.

Say you are matching to an accuracy of 0.0000001

Sample 'A' is 0.1234567
Sample 'B' is 0.1234568

They match within the margin of error.

Sample 'B' is 0.1234568
Sample 'C' is 0.1234569

They match within the margin of error

By the FBI standards

Sample 'A' = 0.1234567
and
Sample 'C' = 0.1234569
Difference = 0.0000002
are a 'match' by data chaining even though they differ by twice the margin of error.

With a longer chain, you can make any two samples match.
In other words, it is pure pseudoscience.
They are making it up as they go along.

So9

[You Gotta Be Kidding Me]

"This is a matter of tolerance stacking, something that always must be avoided in science and engineering. Say you are matching to an accuracy of 0.0000001 Sample 'A' is 0.1234567 Sample 'B' is 0.1234568 They match within the margin of error. Sample 'B' is 0.1234568 Sample 'C' is 0.1234569 They match within the margin of error By the FBI standards Sample 'A' = 0.1234567 and Sample 'C' = 0.1234569 Difference = 0.0000002 are a 'match' by data chaining even though they differ by twice the margin of error. With a longer chain, you can make any two samples match. In other words, it is pure pseudoscience. They are making it up as they go along."

I once asked a coworker why the International Olympic Committee kept changing the rules on Olympic basketball competition (no "allyoops" allowed, no vertical jumps over 30", no ...black guys allowed would be next, etc.). My coworker replied: "They'll keep changing the rules until the US teams lose. Europeans are tired of having their teams lose."

You're right. I think that happened at the FBI forensic labs, as well. "Close enough" ain't good enough, when it comes to evidence.

[FreePaul]

I've often heard the term "Close enough for government work." I guess that applies here. Crime labs, that of the FBI and local authorities, too often are charged with producing evidence to fit conclusions of police and prosecutors. Seems they do this all too well.

[neverdem]

Thanks for posting this article.

[neverdem]

BANG

[yarddog]

I remember a case about 30 years ago when a person in Mobile, Alabama was convicted of killing someone with a sawed off shotgun.

The key evidence was ballistics.

Now obviously the question is, what markings are left by a shotgun barrel. The answer is that the barrel was sawn off roughly and it is said to have left marks on the buck shot which matched the rough edges from the cut.

I thought at the time that I didn't believe it and I still don't.

[TXnMA]

Crime labs, that of the FBI and local authorities, too often are charged with producing evidence to fit conclusions of police and prosecutors. Seems they do this all too well.

Very true. Consider these final words from Conclusion of the report by the DOJ's own OIG (Office of the Inspector General) concerning the FBI Lab's work on the bomb that allegedly destroyed the Murrah Building in OKC:

----------

(Following a list of twelve egregious scientfic and evidentiary errors:)

"These errors were all tilted in such a way as to incriminate the defendants."

"We are troubled that the opinions in [FBI Labs' Agent] Williams' report may have been tailored to conform to the evidence associated with the defendants."

We conclude that Williams failed to present an objective, unbiased, competent report."

----------

If I learned one thing in all my years of work on the OKC bombing, it is this: Never trust the integrity of the FBI's collection, handling, analysis, and reporting of evidence.

[El Gato]

If A=B and B=C but A<>C then there is something seriously (scientifically) wrong with the definintion of 'matches'.

It's more like A~=B and B~=C -> A~~=C. I sure wouldn't want to be convicted of a crime because I happened to stop by the sporting goods store, gun shop, or gun show table, right after or before the real killer. I can't believe anyone was ever convicted on such flimsy evidence. But I guess many have been. Those lead batches are each used in tens of thousands (or more) of bullets, and it stands to reason that a shipment of a particular caliber, to any one dealer will probably contain bullets all, or almost all, made from the same batch. How does that prove anything?

[El Gato]

This puts a real damper on the "balistic fingerprint" database crap, doesn't it.

It should, but I wouldn't place bets you can't afford to lose, that it will. Gun grabbers lie, cheat and have absolutely no sense of integrity. (They steal too, your tax money, which is why they want your guns).