Although a report released this week by the President’s Council of Advisors on Science and Technology concludes that there is scant scientific underpinning to a number of forensic practices that have been used, for years, to convict thousands of individuals in criminal cases, the U.S. Department of Justice has indicated that it will ignore the report’s recommendations while the FBI has blasted the report as “erroneous” and “overbroad.”

The report, titled “Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods,” concludes that a number of common, pattern-matching forensic disciplines – bite-mark analysis, fingerprint and firearm comparison, shoe-tread analysis, and complex DNA mixture analysis – need additional support to be deemed scientifically valid and reliable – a conclusion in line with that reached in the groundbreaking 2009 report on forensics issued by the National Academy of Sciences National Research Council.

In a statement reported by the Wall Street Journal, Attorney General Loretta Lynch said that the agency remains “confident that, when used properly, forensic science evidence helps juries identify the guilty and clear the innocent, and the department believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning.” As such, she said, while “we appreciate their contribution to the field of scientific inquiry, the department will not be adopting the recommendations related to the admissibility of forensic science evidence.”

The DOJ did not respond to The Intercept’s request for additional information, but based on her statement, it appears Lynch is saying there’s simply nothing to see here and that the criminal justice system is working just fine.

Subjective Eyeballing vs. Scientific Method

The Intercept first reported on the report’s conclusions earlier this month, after obtaining a draft copy. The text of the final report, released Sept. 20, appears to be nearly identical to the leaked draft.

Foundational validity and reliability are essential to shore up forensic practices, the report concludes – attributes that are largely absent in the disciplines it reviewed, which rely heavily on the subjective determinations of practitioners. Pattern-matching forensics involve an examiner determining whether a piece of crime scene evidence can be visually matched to a suspect – whether an alleged bite mark on a victim’s hand matches a suspect’s dentition, for example, or whether a partial, or smudged, fingerprint found at the scene of a crime matches a clean print obtained from a suspect – determinations currently based primarily on a subjective eyeballing of the objects at issue.

“Foundational validity requires that a method has been subjected to empirical testing by multiple groups under conditions appropriate to its intended use,” reads the report. Such studies must demonstrate that a practice is “repeatable and reproducible” and must provide “valid estimates of the method’s accuracy” – in other words, a meaningful error rate. “The frequency with which a particular pattern or set of features will be observed in different samples, which is an essential element in drawing conclusions, is not a matter of ‘judgment.’ It is an empirical matter for which only empirical evidence is relevant,” the report continues. “For forensic feature-comparison methods, establishing foundational validity based on empirical evidence is thus a sine qua non. Nothing can substitute for it.”

For years forensic practitioners in many of the disciplines included in the White House report (as well as the National Academy of Sciences report) have overstated in court the validity and reliability of their results. Consider the case of Bill Richards, for example, who spent nearly 23 years in prison for murdering his wife, Pamela, before the California Supreme Court last May overturned his conviction, concluding that Richards had been a victim of junk science and false testimony. In his case, a renowned forensic dentist testified that a mark found on Pamela’s hand was a clear match to Richards’s supposedly unique dentition. Notable, the dentist testified, was that Richards had an under-erupted canine tooth that would account for a void in the alleged bite-mark injury to Pamela’s hand; only “one or two or less” people out of 100 would have such a feature, he testified. The dentist, Dr. Norman “Skip” Sperber, ultimately recanted that testimony, saying that it had no scientific basis. The new White House report notes that it is unlikely that bite-mark evidence will ever be scientifically supported.

In all, the report makes eight overarching recommendations for improvement — to the National Institute of Standards and Technology, to the FBI, to the attorney general, and to the judiciary — and called for “a vigorous research program” to improve forensic sciences building off “recent important” research conducted into fingerprint analysis, for the judiciary to take into account actual scientific criteria when assessing whether forensic evidence and testimony should be allowed into court, and for the attorney general to “direct attorneys appearing [in court] on behalf of the [DOJ] to ensure expert testimony in court about forensic feature-comparison methods meets the scientific standards for scientific validity.”

“Where there are not adequate empirical studies and/or statistical models to provide meaningful information about the accuracy of a forensic feature-comparison method,” the report concludes, “DOJ attorneys and examiners should not offer testimony based on the method.” And in the event that testimony is necessary, the report says, the expert should “clearly acknowledge to courts” the lack of scientific evidence to support the underlying forensic practice.

Current Standards Permit Junk Science

Under “current legal standards,” and under the U.S. Supreme Court ruling in the 1993 case Daubert v. Merrell Dow Pharmaceuticals, federal judges are tasked with acting as gatekeepers over what expert testimony will be allowed into evidence. Where scientific – or supposedly scientific – evidence is concerned, the Supreme Court concluded that before allowing expert testimony in a case the trial judge must ensure that “any and all scientific testimony or evidence admitted is not only relevant, but reliable,” which necessitates, in part, an assessment of “whether the reasoning or methodology underlying the [expert’s] testimony is scientifically valid.”

This, the new report correctly notes, is where science and the law intersect. But in practice, legal scholars note, the Daubert standard has not kept pseudoscience out of the courtroom. And when courts rely on precedent to allow certain questionable forensic practices into evidence, the result is something like a feedback loop. “Bite-mark analysis has passed every Daubert challenge that it has ever faced and [yet] there isn’t a scientist on the planet that would argue that bite-mark analysis is a valid and reliable science, aside from the few practitioners who still cling to that belief,” said Chris Fabricant, director of strategic litigation for the Innocence Project and a vocal critic of the use of junk science.

Fabricant said the DOJ’s rejection out-of-hand of the White House report is disheartening. “You would think that they would want to get it right. The idea is not that we’re going to spring open the jailhouse doors and let everybody free. The idea is that scientific evidence ought to be scientific,” he said. “To simply reject the call for more research and to say that Daubert is sufficient is ludicrous, because Daubert is obviously not sufficient,” he continued. “So, the idea that you would point to the courts and to precedent for the idea that forensic evidence is good enough for government work is a joke.”

A Threat to the Forensics Industry

Attorney General Lynch was not alone in her rejection of the science council’s report. The FBI also issued a statement taking issue with the group’s work. While the “FBI agrees … that forensic science plays a critical role in the criminal justice system, and therefore needs to be held to high standards,” and that additional funding “is needed to develop stronger ties between the academic research community and the forensic science community,” the agency said, the “report makes broad unsupported assertions regarding science and forensic science practice.”

Among the FBI’s concerns: that the report says the “only way” to establish validity as applied is through testing and the development of error rates. The agency contends that this assertion is “fundamentally at odds” with the 2009 NAS report. And the agency complains that the report omits “numerous” empirical studies into the various practices it critiques.

Asked about the FBI’s complaints, Eric Lander, co-chair of the presidential council and president and founding member of the Broad Institute of MIT and Harvard, a biomedical research group, told The Intercept that the FBI is mistaken. “Neither report says that proficiency testing be used to estimate the ‘error rate’ of forensic methods,” he wrote in an email, and both reports agree that examiners should be subject to proficiency tests. And Lander said he is “not aware” of what studies the FBI believes were ignored by the report. “We specifically received FBI’s input on studies to consider and we did so.”

An FBI spokesperson told The Intercept he could not offer additional comments on the report outside the agency’s published statement, but he did email a list of six studies the FBI believes should have been considered. In a follow-up email, Lander wrote that the presidential council did in fact review the six studies the FBI complains that it missed. “However, these studies are clearly not empirical studies ‘providing support for foundational validity,’” he wrote. “Indeed, only one of the papers even reports an empirical study of current forensic method at all!” Still, he wrote, if the FBI can provide an “actual list of empirical studies providing support for foundational validity,” they would “be delighted” to review and comment.

Meanwhile, some of the criticism has turned nasty — and personal. On September 21, the American Congress of Forensic Science Laboratories (ACFSL), an industry trade and lobby group formed last year in response to the forensic reform movement, published a “position statement” that suggests the White House report’s work was motivated by politics or perhaps by some desire to undermine the criminal justice system. “Our greatest concern is that the intellectual exercise of evaluating the reliability of forensic science … is too often ignorant of the ugly realities associated with solving crimes like murder and rape as quickly and accurately as possible,” reads the statement.

“Interestingly, the PCAST report comes during a presidential administration that has demonstrated a deep sensitivity to the needs and demands of trial attorneys, criminal defendants and advocates of sweeping criminal justice reform. Future administrations may take a different approach, tending to champion positions traditionally held by police and prosecutors.” While the group says it has “no opinion in these matters,” it also suggests that at least two individuals involved in the report’s creation — including Lander — have some sort of political agenda that is adverse to the ACFSL. Lander is a member of the Innocence Project’s board of directors, a group that the forensics congress considers a “legal-activism group” that it claims has been “publicly criticized … for the unfairness of its public statements” and says critics have questions about the Innocence Project’s “motives.” While the forensics congress said its “intent is not to disparage any individuals,” it nonetheless feels it has “no choice but to recognize the relevance of these biases as we evaluate the legitimacy of the PCAST report. Indeed, forensic science is being judged by such a standard.”

Fabricant said the congress’s assertions were absurd. “To suggest that the leading scientists in the country would cash in their credibility to do – what? What possible agenda could they be pursuing except scientific validity?” he asked. “I’d like to know what agenda they propose is being driven, and how somebody like Eric Lander – who mapped the human genome — is going to preside over a process that is intended to undermine criminal justice.”

Ultimately, what actors in the criminal justice system need to accept is that science must be injected into forensics for the disciplines to be legitimized. “We know that nearly 50 percent of wrongful convictions are attributable, at least in part, to forensic sciences that were misapplied,” he said. “So the idea that it’s been working great is, at best, whistling past the graveyard.”