Donald Eugene Gates served 27 years in prison for armed murder and rape, seven years past the length of his minimum sentence. He was sent to prison in part due to the testimony of informants and in part due to forensic analysis of hair associated with the case. In 2009, however, DNA evidence indicated that he was not guilty of the crime.

On Monday, the FBI released what was almost certainly a painful admission: this case was not unusual. The vast majority of the agents that it had sent to court to testify about a specific forensic analysis had submitted erroneous statements to the court. In its preliminary review of relevant cases, at least 90 percent of them were problematic.

The technique in question is hair analysis, in which forensic specialists attempt to match features of hairs associated with a crime to either a suspect or a victim. Prior to the advent of widespread DNA testing in the '90s, this technique was used in thousands of cases as evidence tying a suspect to a specific crime; the Bureau discontinued its reliance on it in 1996. But the issue was revived in 2009, when the National Academies of Science performed a systematic review of the forensic sciences. That review set in motion a process that is just now bearing fruit.

Flunking science

On the surface, the approach seemed scientific. Features of hair associated with a crime could be compared to those of people thought to be involved in order to determine whether they came from the same individual. These features could be obvious ones like color (including dyes) and curliness. But other things were more subtle and could only be observed under a microscope. These features include things like the diameter of the hair shaft and how evenly the pigment is distributed along it.

Match enough of the criteria and you could positively identify the source of the hair. Some forensic experts argued in court that they could do just that, and the approach was widely accepted within the judicial system.

However, problems were apparent quite early on. As early as 1976, a paper concluded that “due to the fact that so many of the characteristics coded are subjective—for example, color, texture—it was not possible to get complete reproducibility between two or more examiners coding the same hair." And once DNA analysis became available, problems with this approach were obvious. Researchers at the FBI itself found that in 11 percent of the cases where hair examiners declared hair "similar," DNA testing revealed that there wasn't a match.

It was the report from the National Academies that gathered all this evidence in one place. (The report also noted problems in many other areas of forensic analysis.) Rather than being "generally accepted in the scientific community," as some courts had found it, the committee that prepared the report found that hair analysis lacked any rigorous underlying science. There were no population-wide statistics on how often specific features were present in hair, meaning that it was impossible to tell how often an apparent match would occur at random. And forensic experts had never set a standard on how many features needed to match in the first place.

At best, the committee concluded, hair analysis could either rule suspects out or indicate that they belonged to the same group as the source of the hair—but there was no way of knowing how large that group was. Despite this finding, "several members of the committee have experienced courtroom cases in which, despite the lack of a statistical foundation, microscopic hair examiners have made probabilistic claims based on their experience."

From academic analysis to the legal system

Perhaps because they are so large and comprehensive, National Academies reports don't often attract much attention beyond the specialists in the fields they target. In this case, however, two groups took note: the National Association of Criminal Defense Lawyers and the Innocence Project. After what one assumes were several years of negotiations, the two announced a collaboration with the FBI and Department of Justice that would reexamine all cases handled by the Bureau that involved hair matching.

In a preliminary scan of 15,000 cases that were tried before 2000 (by which point DNA matching had replaced hair analysis), there were 2,100 in which hair matching was used, suggesting that this was a widespread problem. This scan triggered a systematic reanalysis of these and other cases to determine if the analysis involved scientifically unsupportable statements. That analysis ultimately led to this week's announcement: 26 of 28 examiners had made unsupportable statements in court or reports used during prosecutions, tainting at least 90 percent of the trials in which they were involved.

These findings come from a review of just 17 percent of the total cases identified as potentially involving hair analysis. Over half of the cases that have been reviewed (268 of 500) involved hair analysis being used in trial. Thirty-five of these cases resulted in the death penalty, and errors have been identified in 33 of those cases; nine of the convicted have already been executed.

These findings will trigger other aspects of the agreement. The FBI will notify both prosecutors and defense attorneys in all problematic cases and offer to perform DNA testing on any evidence that has been retained. In any federal cases, the Department of Justice will waive any problems with the statute of limitations that might otherwise prevent the cases from being reexamined. However, many of the cases involved FBI agents testifying in state trials (41 different states are involved in the analysis that has taken place so far), so the outcomes in those cases may vary considerably.

In significant cases where the prosecution takes no action and the defense attorneys cannot be found, the two non-governmental organizations involved in the agreement will make efforts to reopen the case.

The agreement that produced this analysis is striking in the extent to which it appears that the federal government is attempting to do the right thing in terms of waiving statute of limitation issues and offering to perform DNA analysis. Also striking is the fact that the FBI is hosting a statement by the Innocence Project's Peter Neufeld that takes the Bureau to task: “These findings confirm that FBI microscopic hair analysts committed widespread, systematic error, grossly exaggerating the significance of their data under oath with the consequence of unfairly bolstering the prosecutions’ case.”

But the National Academies' report suggests that hair analysis isn't the only area of concern. The report highlights a number of areas where claims have been made without a robust statistical understanding or forensic scientists have substituted their personal experience for rigorous analysis.

Part of this may stem from the fact that the standards of acceptance for techniques are often set by judges and juries without scientific training. But the Academies' report also suggests that forensic researchers suffer from a potentially dangerous defensiveness: "The insistence by some forensic practitioners that their disciplines employ methodologies that have perfect accuracy and produce no errors has hampered efforts to evaluate the usefulness of the forensic science disciplines."

Apparently, it's also hampered the ability of many individuals to receive a fair trial.