I've written extensively on the many basic problems that make all government-run, computer-automated mass surveillance programs a waste of taxpayer money. But a new report (PDF) from the Offices of Inspectors General of the Department of Defense, Department of Justice, CIA, NSA, and Office of the Director of National Intelligence shows in some detail how our government took the bad idea of building powerful computers to sniff out a terrorist needle in a digital haystack, then made it even less useful in practice.

The new OIG report on the NSA-run Presidential Surveillance Program (PSP), of which the previously revealed warrantless wiretapping program was just a part, contains a number of stunning revelations; I'll go through some of those in subsequent articles. But perhaps the report's greatest value is in the way that it provides a glimpse into how the secrecy-obsessed Bush administration actually sabotaged the NSA's massive, law-free surveillance program by overly restricting intelligence personnel's knowledge of and access to it. In short, the PSP was too secret for its own good.

A throat so deep

One of the pervading themes of the OIG report is that the PSP was really, really, really secret. It was so secret, in fact, that the president himself picked which non-operational personnel were to be "read into" the program. So if you weren't actually involved in the day-to-day running of the NSA's giant SIGINT vacuum, then the commander-in-chief personally decided whether you should know that it even existed.

This extreme level of secrecy posed myriad practical problems when it came to actually using the PSP's output in the day-to-day counter-terror work that goes on at a number of agencies—DHS, CIA, FBI, the National Counterterrorism Center (NCTC), and so on. Almost none of the working-level analysts who might benefit from the PSP's output were allowed to know of the program's existence, so getting that output into those workers' hands meant carefully stripping it of any hints about its provenance, thereby rendering it significantly less valuable.

The problem was especially acute at the FBI, which wasn't as widely looped in on the PSP as the CIA (more on the latter, shortly), despite the fact that the Bureau is involved in domestic counter-terrorism. When the few people at the FBI who were in-the-know about PSP's existence got "product" from it, they had to be very careful about what they did with the information, lest some lowly FBI guy in the trenches learn of the existence of the program.

But even if FBI agents had known of the existence of PSP and of the origin of some of the tips they were getting, that still wouldn't have been much help. Just ask the CIA, where more people knew about the PSP but still had no idea how it worked.

The multibillion-dollar electronic anonymous tipster

The CIA seemed to have an easier time dealing with the PSP since more of its people were read into the program, but there were still serious problems. Most of those who had knowledge of the PSP were senior managers and not the working-level personnel who could have made practical use of the PSP's products.

The report notes that even for the few working-level CIA folks who were read in, "much of the PSP reporting was vague and without context," so they wound up relying more on other, more familiar and accessible analytical tools and sources. The briefing that CIA folks were given on read-in didn't tell them much about how PSP worked or how to use its products, and without that knowledge the output of the program was of limited intelligence value.

Like journalists, CIA officers are trained to consider the source of their incoming information in order to evaluate it by placing it in context. In the case of the PSP, the source was a giant black box—a sort of electronic anonymous tipster who would periodically drop vague, context-free nuggets into the already unmanageably wide inbound information stream that they had to sift each day.

This black box problem highlights the key barrier that the PSP's deep secrecy raised to its effectiveness in the war on terror. The output of any information-gathering system will eventually have to be evaluated by a human; but for any human knowledge worker who is tasked with looking for a slender needle of relevance in an overwhelmingly large informational haystack, any additional data that arrives free of context, where the worker doesn't have any understanding of the mechanisms that produced it, is noise, not signal.

You can easily imagine that when the NSA tells a CIA analyst, "Here's a tip to add to your pile of things to look into; it comes from our giant, computerized black box, and you have no idea how that box works or how it actually decided that this (potentially vague) tidbit was important," the analyst may prefer instead to tune out that incoming data and to turn instead to the tools and sources he knows.

It wasn't just the CIA that ran into the black box problem. According to the OIG report, "NCTC analysts noted that the NSA policy protecting the source of the PSP information would have resulted in them not fully understanding the value of the PSP information."

Another widely quoted section of the report bears out this same point:

NCTC analysts involved in preparing the threat assessments told the ODNI OIG that only a portion of the PSP information was ever used in the ODNI threat assessments because other intelligence sources were available that provided more timely or detailed information about the al-Qaida threat to the United States. During the interviews, the NCTC analysts noted that PSP information was only one of several valuable sources of intelligence information available to them.

In the end, the PSP's secrecy put it at a disadvantage vs. other sources of information that working-level analysts knew and trusted. So when the OIG sought to isolate the impact of the PSP on the nation's intelligence-gathering activities, the best that analysts in one agency after another could tell them was that the PSP product was just one source among many, and a difficult one to use at that.

The PSP was shrouded in such deep secrecy partly for operational security reasons, but also because of political considerations. Some of what went on under the auspices of the PSP was later determined to be illegal, and in the next article we'll take a closer look at the darker corners of the program.