Director of National Intelligence James Clapper has released another trove of documents on the NSA and FBI's surveillance program, including the FISA court's reasoning for allowing sweeping collection of email and other metadata. An undated and heavily redacted court order issued by Judge Colleen Kollar-Kotelly approved the measure, which allows the NSA to collect several unidentified categories of information as long as they don't include the actual content of a call or message. Among other things, this includes email addresses, which the court found "are not part of the email's 'contents.'"

The NSA argued it needed to find unknown terrorist accounts, not just monitor known ones

While the ruling was made under the laws used for wiretaps of individuals, the FISA court said that the intelligence community didn't need to have the name of an actual person in order to collect metadata. The expansive program was also justified by the fact that it wasn't part of ordinary law enforcement actions but was a facet of the intelligence community's anti-terrorism efforts. "NSA asserts that more precisely targeted forms of collection against known accounts would tend to screen out the 'unknowns' that NSA wants to discover, so that NSA needs bulk collection in order to identify unknown [redacted] communications," wrote Kollar-Kotelly, quoting a memorandum: "Analysts know that terrorists' emails are located somewhere in the billions of data bits; what they cannot know ahead of time is exactly where." The NSA estimated that as part of the program, it would give around 400 email addresses a year to the FBI and CIA to help with investigations.

"The raw volume of the proposed collection is enormous."

Nonetheless, the court expressed concern over the program's scope. "The raw volume of the proposed collection is enormous," wrote Kollar-Kotelly; though the statistics she quoted were redacted, the number of Americans whose communications are intercepted is clearly a concern. In the end, though, the court approved the request both because of Patriot Act rules that loosened legal limits and because of a long-standing ruling says metadata is not protected by the fourth amendment. The ruling allowed the NSA to search email data through "hops" similar to those used for phone data collection, starting with a suspicious seed account and algorithmically analyzing the database for accounts that communicated with it, as well as accounts in contact with those accounts. This information would be kept for 18 months, then sent to an offline system. After four and a half years, it would be destroyed.

Because of how frequently it approves NSA requests, the FISA court has been described as a "rubber stamp" for the surveillance community. However, declassified documents show a lingering discomfort over the NSA's broad interpretation of laws, its periodic inaccurate and incomplete reports, and apparent technical or policy issues that led it to overstep its boundaries. Previously released documents show that the FISA court has admonished the NSA for unreasonably and intrusively collecting information of US citizens, and in a separate order released yesterday, FISA judge John Bates described "systemic overcollection" and "serious compliance problems" within the agency.

For now, the FISA court's secret rulings are the only oversight the program faces, and legal challenges to the laws that govern it have largely failed. This week, the Supreme Court declined to hear a suit from the Electronic Privacy Information Center and other civil liberties groups, and an ACLU lawsuit against phone surveillance seems likely to meet the same fate. Congress, meanwhile, is slowly moving towards amending the rules it passed last year, with generally pro-FISA Dianne Feinstein (D-CA) and surveillance opponent Patrick Leahy (D-VT) introducing competing oversight bills.