How well does the US justice system work? Given that many states still carry out the death penalty, it's a rather significant question.

Some biostatisticians have teamed up with lawyers in an attempt to provide a scientific answer to the question. Based on their figures, at least 4.1 percent of the individuals sentenced to death will eventually be exonerated. Given the high level of scrutiny that capital cases are subjected to, the authors conclude that the percentage of innocents stuck with life sentences is even higher.

The study seems to have been inspired in part by a comment from US Supreme Court Justice Antonin Scalia, who once claimed that US Courts have "a success rate of 99.973 percent." This, the authors say, is ludicrous. "In fact, the claim is silly," they write. "Scalia’s ratio is derived by taking the number of known exonerations at the time, which were limited almost entirely to a small subset of murder and rape cases, using it as a measure of all false convictions (known and unknown), and dividing it by the number of all felony convictions for all crimes, from drug possession and burglary to car theft and income tax evasion."

The vast majority of felony cases don't end in decisions regarding guilt or innocence. Instead, 93 percent are subject to plea bargains. Of the remainder, most convictions aren't reexamined carefully—appeals tend to focus on technicalities of the case rather than matters of guilt or innocence.

The exception, the authors argue, is death penalty cases. Here, matters of guilt and innocence are examined in detail, often for decades after sentencing. Even though these cases account for less than 0.1 percent of the prison sentences in the US, 12 percent of the individuals who are exonerated after conviction were sentenced to death. So if we're ever to have an accurate measure of the rates of erroneous convictions, this is the place.

Their study starts with the "modern" death penalty era, ushered in by a Supreme Court decision in 1973. It extends to 2004. The cutoff comes in part because it takes time—an average of more than 10 years after sentencing—for the evidence to build for exoneration. In fact, one case took 33 years between sentencing and exoneration, meaning that some of the individuals from their study period will almost certainly be exonerated in the future. This means that researcher estimates will err on the low side.

Two other factors also tend to mean that their estimate is going to be low. One is that people on death row die for other reasons, at which point attempts to exonerate them generally stop. Finally, in many cases where there are lingering doubts about guilt, individuals will have their sentences changed to life in prison (this happens to 36 percent of those sent to death row). The rate of exoneration among these individuals plummets compared to those who remain on death row, suggesting that people view the lesser sentence as a victory and stop pushing as hard for a full exoneration.

Even with all these reasons to think their estimate is low, the authors calculate that the cumulative rate of exoneration at 20 years is 4.1 percent. For contrast, over the entire period of their study, 12.6 percent of those on death row were actually executed.

Does this mean we're probably executing innocent people? The authors don't really commit to that conclusion, but they do suggest that instances of this happening will be rare, given the heavy scrutiny and relatively high rate of exonerations. But their finding certainly does mean we are regularly convicting innocent people, in stark contrast to Justice Scalia's claim.

What the authors do find disturbing is that we accept that our job is done once a person's sentence is converted to life in prison. Given that these changes are often made because of the possibility of innocence, the authors conclude that far more than 4.1 percent of these individuals are probably innocent. "Our research adds the disturbing news that most innocent defendants who have been sentenced to death have not been exonerated," the authors conclude. "Many—including the great majority of those who have been resentenced to life in prison—probably never will be."

PNAS, 2014. DOI: 10.1073/pnas.1306417111 (About DOIs).