Prosecutors and sheriff’s deputies in California’s Orange County used jailhouse informants in an extraordinary and long-running scheme to illegally obtain confessions from criminal defendants, the American Civil Liberties Union (ACLU) is alleging in a new lawsuit.

The suit, filed early Wednesday, alleges that the district attorney’s office and sheriff’s department in the suburban county south of Los Angeles routinely employed prisoners – including hardened gang members – as informants and used “threats of violence to coerce confessions” from defendants, violating their rights to an attorney.

The ACLU cited a mountain of evidence, amassed in criminal cases over the past five years, that prosecutors obtained material illegally, suppressed parts favorable to the defence, and sought to cover up the existence of the scheme.

“For 30 years, the Orange County sheriff’s department and district attorney’s office have been operating an illegal informant program out of the jails,” the ACLU lawyer Brendan Hamme told the Guardian. “They’ve used it to coerce information from defendants, including with threats of death, and at the same time they’ve been systematically hiding evidence of that program. These sorts of tactics are offensive to basic constitutional principles and ethical duties.”

Local officials have sought for years to play down the extent of their role. On Wednesday, the district attorney’s office said in a statement it had “not been served the lawsuit the ACLU has provided to the media” and that it would “continue to lawfully use all evidence lawfully developed by local law enforcement”. Orange County Sheriff’s Department said it could not comment on pending litigation but insisted it had “cooperated fully” with state investigations.

A 'hellish world': the mental health crisis in America's prisons Read more

The ACLU’s 41-page complaint lays out the particulars of the scandal, much of it based on documents produced by the county and by the courtroom testimony of county officials, in unforgiving detail: evidence of informants telling their cell mates they were marked for death by a “shot caller”, or gang leader, and could only save themselves if they talked in detail about their crimes; evidence that one informant was encouraged by the police to help frame a 14-year-old boy for attempted murder even though several witnesses said the boy was nowhere near the scene of the crime; evidence that informants were paid tens of thousands of dollars for their services; evidence that prosecutors had repeatedly failed to hand over documentation to defence lawyers as required by law; evidence that sheriffs had repeatedly lied under oath when asked about the existence of the informant program.

The scandal in Orange County, a bastion of law-and-order conservatism, first erupted in 2013, when Scott Sanders, a public defender, noticed that the same jailhouse informant was cited in two high-profile murder cases he was working. That informant turned to be a member of the Mexican mafia, whose jailhouse file included the instruction “Do not use as a CI [confidential informant]”.

When Sanders requested more information, the district attorney’s office stonewalled. When the trial judge sided with Sanders and the county was forced to hand over hundreds of documents revealing a troubling pattern, the district attorney’s office sought to discredit the judge and asked more than 50 times in a single year to have cases reassigned to another courtroom.

It wasn’t just civil libertarians who were outraged. Many Orange County conservatives were disturbed that, in case after case, the district attorney’s office was dropping charges or requesting lesser sentences – all, according to the ACLU suit, to avoid revealing more about the jailhouse informant scheme.

“There is still a significant amount of information being hidden from defence attorneys,” Hamme said. “We’ve seen in at least 18 cases that are publicly known that the DA’s office gets around this by dismissing or reducing charges … But the DA can’t plea-bargain [our suit] away. It will finally have to reckon with its misconduct.”

One of the compromised cases was the biggest mass killing in the county’s history. Prosecutors went all out for the death penalty for Scott Dekraai, who shot and killed eight people in a hair salon in 2011, but the scandal delayed sentencing for years and the judge forced the county to settle for a life sentence instead.

Many sheriff’s departments run jailhouse informants, usually in an effort to thwart crimes being planned inside or outside the jail. If a prisoner volunteers information, unprompted, about a crime, that too is usually regarded as legitimate evidence. But the US supreme court stipulated in 1963 that once a defendant was charged he or she could not be questioned by the state or its agents (including informants) without first being offered legal representation.

Smaller scandals than Orange County’s have erupted from time to time. Los Angeles was rocked in the 1980s by a conman who admitted he used the jail’s phone system to call prosecutors or police to learn the details of a crime, then fabricated evidence against the suspect as a way of reducing his own legal liabilities. Dozens of cases were tainted before the scam was shut down.

In Orange County, the scandal has been remarkable for the tenacity of the officials at the center of the storm. Sheriff Sandra Hutchens has announced she will not be seeking re-election this year, but has persisted in saying that any abuses were the work of a few rogue deputies only. Tony Rackauckas, the district attorney, has been stridently unapologetic and is locked in a furious re-election battle with a county supervisor who is using the scandal as his primary political weapon.