Stay on Top of Emerging Technology Trends Get updates impacting your industry from our GigaOm Research Community

The government is notoriously bad at tech but, still, this was a doozy: the federal court system this week quietly purged thousands of cases from its online database — including cases from four important appeals courts — in an effort to upgrade to the system. This means people must now go the court to retrieve the physical documents, or else submit a written request and wait to receive them by email for a $30 fee.

For users of the creaky database known as PACER, such frustrations are nothing new. The disappearing documents episode is par for the course for a public records system that gobbles money but remains buggy and out of date. Most frustrating of all is that, as the late internet activist Aaron Swartz showed everyone, the documents could be made easily available at little cost.

The PACER problem is not hard to solve. But it will require the right people to acknowledge that the obstacles are political and financial rather than technological.

Down the memory hole, how it happened

The legal community reacted angrily this week when they discovered the disappearance of the documents, which span years of online filings from venues like the patent appeals court, the 2nd Circuit and the bankruptcy court for Los Angeles. The frustration stems from a new and arbitrary impediment to public information, and from the manner of PACER’s explanation, which came as a one paragraph notice:

When contacted by phone, a PACER supervisor explained that the five courts in question were using a legacy platform for older court records, and that these records could not be migrated to an impending update to PACER. The supervisor, who did not want to be named, also explained that the purpose of the upgrade was to let users employ the same login credentials for both PACER (which lets users view and download documents) and a related system called CM/ECF, which lawyers use to file court documents online.

While the single sign-on feature is an improvement, it’s still unclear why the upgrade process couldn’t also have included the legacy system used by the five courts, or why the courts didn’t simply migrate the legacy system documents onto PACER. The supervisor also offered only an opaque explanation as to why the country’s various federal courts — which include district, appeals and bankruptcy courts– use different IT systems in the first place.

“Each court controls their own case information. Within each court, you’re going to have different individuals who are more innovative or pro-active,” he said.

Later on Tuesday, a spokesperson for the Administrative Office of the U.S. Courts, which overseas PACER, provided a statement to reporters about the disappearing documents — but one that simply repeated the original website notice.

More money and a “criminal” caution

Problems with PACER are nothing new but what is so vexing to the legal community is that no amount of money seems to solve them. In 2009, for instance, the judiciary had a reported surplus of $150 million in its Information Technology Fund while, in 2013, PACER prices jumped from 8 cents to 10 cents a page.

The result? PACER today is much like it was five years ago, offering an experience that feels more akin to Geocities than Google. Its motley display and search functions often baffle my techie colleagues, even though PACER’s purpose is rather simple: to sort and display PDF files:

It wouldn’t be hard to build a more elegant system and, what’s more, a single person could probably do it. Consider the example of Max Yuan, a young inventor who downloaded and indexed every filing from the U.S. Patent Office for use in his analytics startup. Or David Zvenyach, a lawyer who this year created a script that monitors and displays unannounced changes to every Supreme Court decision.

Why then, hasn’t someone come forward to offer a better public version of PACER? The answer is that they’ve tried, but the effort was short-lived. Specifically, in 2009, the late internet activist and Reddit co-founder Aaron Swartz took part in a plan with open government advocate Carl Malamud and others to scrape the entire contents of PACER’s database.

“Pacer is just so awful… The system is 15 to 20 years out of date,” Malumud told the New York Times at the time, explaining the scheme.

The project made some headway as Swartz downloaded nearly 20 million pages of text — about 20 percent of PACER — but soon the FBI intervened. Swartz was not charged (in this case at least) but the FBI investigation ended the project, and now the FAQ section on PACER’s homepage warns that unauthorized attempts to collect its data (which is in the public domain) may result in “criminal prosecution”:

Meanwhile, the Administrative Office of the U.S. Courts appears to be sticking almost defiantly to the past. The Office has little to say about IT upgrades but, on the “Publications and Reports” section of its website, visitors can view a “Courtroom Technology Manual” from 1999 that covers subjects like “A/V racks” and “paging systems.”

Starving a system to plug other holes

PACER is not inherently a bad system; it’s simply a dated one. Indeed, as the federal courts’ own 25-year retrospective article explains, PACER was a very disruptive piece of technology that changed courthouse culture by, among other things, eliminating a stream of couriers and court runners. And compared to some state courts, which have yet to offer digital records in the first place, PACER was ahead of its time.

So what happened? Why has the federal courts’ IT system stagnated despite more money? The answer, unsurprisingly, is that the money has gone elsewhere.

In 2010, researcher Steve Schultze revealed that most of the money collected from PACER is being used for a host of other court-related expenses, such as flat screen monitors and notification systems for bankruptcy creditors. In short, the Administrative Office of the Courts has taken a liberal interpretation of a fee-collection provision of the 2002 E-Government Act, and treated it as a general revenue source — while also failing to improve PACER.

While this apparent misallocation came under repeated scrutiny by former Senator Joe Lieberman and by journalists like Tim Lee, it has continued all the same. More recently, activist Aaron Greenspan has filed a lawsuit over the PACER fees.

Meanwhile, some suggest that PACER’s policies are driven in part by a desire to collect bulk download fees from private companies like [company]WestLaw[/company] and [company]Bloomberg[/company], which in turn charge their subscribers high fees to search and curate the same court records found on PACER itself. Easier access to the court records could compromise revenue streams for both the court system and the companies.

A better PACER

The culture of government, as well as its massive user base, makes public IT projects inherently prone to failure (see healthcare.gov), but still some government sites work better than others. The Security and Exchange Commission’s EDGAR system and THOMAS from the Library of Congress, for instance, are not exactly Facebook-quality design, but they’re better than PACER. And they’re free.

So how hard would it be to remake PACER and package it into something presentable and cheaper to use? The work of Swartz and others have shown that assembling the documents is easy enough, and the rise of cloud computing means storage fees are dropping ever-lower. But there’s still the question of who should pay.

While there is nothing wrong with court fees themselves, which have existed since the 13th century, PACER’s present practice of charging 10 cents for court records appears doubly misguided. The PACER charges not only bear no relation to the actual charge of supplying the records, but they also pose an access-to-justice obstacle for citizens, scholars and everyone else who uses the legal system (while PACER waives the fee for accounts that accrue under $15 a quarter, this policy is typically unhelpful since many filings cost $3 to view and a single case may contain hundreds of filings).

But since Congress is unlikely to replace any revenue lost from the PACER fees, the judiciary’s best option is to seek those fees elsewhere, including from the litigants who file the case in the first place.

To create pressure for such a move is hardly easy, however, since responsibility for such decisions lies with the Judicial Conference of the United States, a group that is (rightly) immune to most forms of political persuasion. More challenging still is that many of the people close to the courts are relatively old, conservative and uninterested in new technology. But not all of them.

Judge William Alsup of the District of California, for instance, taught himself to code Java ahead of a blockbuster trial between Google and Oracle. And former Supreme Court clerks like Tim Wu, Laurence Lessig and Rebecca Tushnet are now leading intellectuals in the technology sphere. They are likely frustrated too with PACER’s problems, but their attention is directed on other important issues.

The bottom line is that fixing PACER is not a technological challenge, but a political one, and technology has yet to solve the problem of getting the politically powerful to focus on the right things at the same time.

Update: Malamud is renewing his efforts to bringing attention to the PACER problem, with letters to senior judges: