AS MEMBERS OF CONGRESS struggle to agree on which surveillance programs to re-authorize before the Patriot Act expires, they might consider the unusual advice of an intelligence analyst at the National Security Agency who warned about the danger of collecting too much data. Imagine, the analyst wrote in a leaked document, that you are standing in a shopping aisle trying to decide between jam, jelly or fruit spread, which size, sugar-free or not, generic or Smucker’s. It can be paralyzing. “We in the agency are at risk of a similar, collective paralysis in the face of a dizzying array of choices every single day,” the analyst wrote in 2011. “’Analysis paralysis’ isn’t only a cute rhyme. It’s the term for what happens when you spend so much time analyzing a situation that you ultimately stymie any outcome …. It’s what happens in SIGINT [signals intelligence] when we have access to endless possibilities, but we struggle to prioritize, narrow, and exploit the best ones.” The document is one of about a dozen in which NSA intelligence experts express concerns usually heard from the agency’s critics: that the U.S. government’s “collect it all” strategy can undermine the effort to fight terrorism. The documents, provided to The Intercept by NSA whistleblower Edward Snowden, appear to contradict years of statements from senior officials who have claimed that pervasive surveillance of global communications helps the government identify terrorists before they strike or quickly find them after an attack. The Patriot Act, portions of which expire on Sunday, has been used since 2001 to conduct a number of dragnet surveillance programs, including the bulk collection of phone metadata from American companies. But the documents suggest that analysts at the NSA have drowned in data since 9/11, making it more difficult for them to find the real threats. The titles of the documents capture their overall message: “Data Is Not Intelligence,” “The Fallacies Behind the Scenes,” “Cognitive Overflow?” “Summit Fever” and “In Praise of Not Knowing.” Other titles include “Dealing With a ‘Tsunami’ of Intercept” and “Overcome by Overload?” The documents are not uniform in their positions. Some acknowledge the overload problem but say the agency is adjusting well. They do not specifically mention the Patriot Act, just the larger dilemma of cutting through a flood of incoming data. But in an apparent sign of the scale of the problem, the documents confirm that the NSA even has a special category of programs that is called “Coping With Information Overload.” The jam vs. jelly document, titled “Too Many Choices,” started off in a colorful way but ended with a fairly stark warning: “The SIGINT mission is far too vital to unnecessarily expand the haystacks while we search for the needles. Prioritization is key.” These doubts are infrequently heard from officials inside the NSA. These documents are a window into the private thinking of mid-level officials who are almost never permitted to discuss their concerns in public. AN AMUSING PARABLE circulated at the NSA a few years ago. Two people go to a farm and purchase a truckload of melons for a dollar each. They then sell the melons along a busy road for the same price, a dollar. As they drive back to the farm for another load, they realize they aren’t making a profit, so one of them suggests, “Do you think we need a bigger truck?” The parable was written by an intelligence analyst in a document dated Jan. 23, 2012 that was titled, “Do We Need a Bigger SIGINT Truck?” It expresses, in a lively fashion, a critique of the agency’s effort to collect what former NSA Director Keith Alexander referred to as “the whole haystack.” The critique goes to the heart of the agency’s drive to gather as much of the world’s communications as possible: because it may not find what it needs in a partial haystack of data, the haystack is expanded as much as possible, on the assumption that more data will eventually yield useful information.

“The problem is that when you collect it all, when you monitor everyone, you understand nothing.”

–Edward Snowden

The Snowden files show that in practice, it doesn’t turn out that way: more is not necessarily better, and in fact, extreme volume creates its own challenges. “Recently I tried to answer what seemed like a relatively straightforward question about which telephony metadata collection capabilities are the most important in case we need to shut something off when the metadata coffers get full,” wrote the intelligence analyst. “By the end of the day, I felt like capitulating with the white flag of, ‘We need COLOSSAL data storage so we don’t have to worry about it,’ (aka we need a bigger SIGINT truck).” The analyst added, “Without metrics, how do we know that we have improved something or made it worse? There’s a running joke … that we’ll only know if collection is important by shutting it off and seeing if someone screams.” Another document, while not mentioning the dangers of collecting too much data, expressed concerns about pursuing entrenched but unproductive programs. “How many times have you been watching a terrible movie, only to convince yourself to stick it out to the end and find out what happens, since you’ve already invested too much time or money to simply walk away?” the document asked. “This ‘gone too far to stop now’ mentality is our built-in mechanism to help us allocate and ration resources. However, it can work to our detriment in prioritizing and deciding which projects or efforts are worth further expenditure of resources, regardless of how much has already been ‘sunk.’ As has been said before, insanity is doing the same thing over and over and expecting different results.”

“We are drowning in information. And yet we know nothing. For sure.”

–NSA Intelligence Analyst