This spring, the government announced a change to the way the National Security Agency collects information targeting foreigners, using the telecom backbone in what it calls "upstream" collection. Whereas for 10 years, the agency had sucked up communications mentioning a target's selector—say, collecting all emails sent to someone in this country that include Osama bin Laden's phone number in the body of the email—in April it stopped doing so domestically (though it will still do tons of it in collection overseas).

In adopting the solution to the "about" problem pitched by the Trump Administration, FISC presiding judge Rosemary Collyer, the latest judge to deal with such violations, did less than her predecessors to ensure that such violations don't cause ongoing privacy violations. Not only did she stop short of ensuring that FISA remains the "exclusive means" to conduct surveillance, she allowed the government to keep data it got by breaking the rules. But it's not clear she has solved the problem. Given the NSA's own description of its understanding of the problem, it's not clear the agency has solved the problem either.

But under the current presiding judge, overseeing the plans of the Trump Administration, NSA will be allowed to keep such data, a change from her three predecessors.

There was a stink, at the time, accusing the Obama Administration of using Section 702 of FISA—which only permits the government to target foreigners—of using it to spy on Americans for five years. Those accusations were, technically, true (the NSA attributed such spying to technical failures, not legal ones). But the truth is far more troubling. In fact, from 2004 to 2016, the NSA was always engaging in collection the FISC would go on to deem unauthorized. For 12 years, under both the Bush and Obama Administrations, the NSA was collecting information that, if retained, would break the law.

Not long after the announcement, the government released documents explaining why it had dropped this kind of collection, which it calls "about" collection. Those documents amounted to a confession that the NSA failed to follow rules the Foreign Intelligence Surveillance Court put in place in 2011 to ensure upstream collection complied with the Fourth Amendment.

While the government claims it does a good job self-policing, the court hasn't always agreed. Even before the FISA Amendments Act passed, the government reorganized PRISM without telling judge Reggie Walton, who was overseeing a challenge to that program. A year later, judge Thomas Hogan was surprised to learn the NSA hadn't been reporting all violations to the court, reporting only systematic ones or specific misrepresentations the government made to the court. After the government revealed two different systematic problems in 2009 and a third in 2011, affecting three different programs, FISC judge John Bates complained about "the third instance in less than three years in which the government has disclosed a substantial misrepresentation regarding the scope of a major collection program." And after several delayed notifications last fall and this spring, Rosemary Collyer scoffed at the government's excuses for two different eleven- and five- month delays in notifying the FISC of violations. "Too often, however, the government fails to meet its obligation to provide prompt notification to the FISC when noncompliance is discovered."

In general, however, the FISC and Congress have to rely on the NSA or DOJ to report any violations of FISA; NSA, effectively, gets to police itself.

The FISC is not like a normal court. There are no cops patrolling the streets to make sure no one breaks the Foreign Intelligence Surveillance Act, the law the FISC oversees, and then charging scofflaws. Violations of FISA get discovered in just three ways. Hypothetically, defendants prosecuted using evidence collected under FISA sometimes can ask to review the underlying process for any problems, but aside from a few times prosecutors have told defendants the government spied on their conversations with lawyers, that has never once worked in practice. In very rare cases, most notably with Edward Snowden's leaks, whistleblowers will reveal details that even Congress didn't fully understand (such as that NSA sometimes bypasses FISA by stealing Google and Yahoo data from their servers overseas, rather than using the FISA program, called PRISM).

So one reason the government ends up spying on Americans inappropriately for years at a time is because it takes that long to get around to telling the FISC it has been doing so.

In 2009, in the wake of problems in the phone dragnet program, Reggie Walton made the NSA conduct an end-to-end review of the internet dragnet to look for problems. Yet it wasn't until, in response to a Walton order, NSA's Inspector General started investigating the internet dragnet that the NSA finally told the FISC that "virtually every … record generated [by the bulk Internet metadata program] included some data that had not been authorized for collection," As Bates observed when laying out NSA's remarkable failures to find these ongoing violations, "those responsible for conducting oversight at NSA failed to do so effectively.

The problems with upstream collection started in 2004, years before the FISA Amendments Act was passed. The government got the FISC to approve collection of what it claimed was only metadata off the telecom switches to replace the Internet dragnet part of President Bush's Stellar Wind program. But in approving the collection, judge Colleen Kollar-Kotelly limited which categories of data the NSA could obtain. Within three months, the government violated those category restrictions. So Kollar-Kotelly imposed twice quarterly "spot checks" to make sure NSA didn't continue to violate those restrictions—roughly 25 spot checks were performed between 2004 and 2009.

As Collyer said in her April opinion imposing a dubious fix on yet more upstream violations, upstream collection "has represented more than its share of the challenges in implementing Section 702."

That's especially true with upstream collection, which involves searching within internet packets to select what content to keep. To begin with, it's a more complex technical process than simply calling Google and asking for the contents of someone's email box (though even with PRISM, NSA gets more than a target's Google email box contents). Plus, when collecting by targeting at the level of packets, the legal categories of content and metadata on which much surveillance law is based break down, as a group of noted computer scientists explained in a report last fall. Similarly, it's often hard to tell whether a packet belongs to a foreigner, especially as more traffic travels internationally and as people adopt location-obscuring tools like VPNs and Tor. Finally, the government can use upstream surveillance to collect on different kinds of selectors—things like a server hosted at a particular IP address, or an encryption key—that create problems not covered by many of the FISC opinions envisioning targeting of email addresses.

In 2010, when Bates dealt with this collection —or more pointedly, with NSA's request to keep the data it collected in violation of specific category limitations—he did something novel. He said the NSA would be breaking the law if it kept and used the data it knew to have violated those category restrictions. Bates pointed to a section of FISA that said anyone using or disclosing data that they "knew or had reason to know [had been] obtained through electronic surveillance not authorized by" FISA might face criminal sanctions. For the first time we know of, a FISC judge was enforcing FISA's "exclusive means" provision—or perhaps more importantly, the authority of FISC to enforce the rules it set on NSA's collection—with threats of criminal sanctions.

But the important precedent to FISC's policing of exclusive means is another upstream collection problem. In 2011, after four years, the government first told the FISC that when it conducted "about" collection (searching for emails that refer to Osama bin Laden's phone number), it sometimes got entire bundles of communication. Sometimes those bundles included communications that were entirely domestic. Mind you, as part of that disclosure process, the NSA revealed it collected a whole bunch more domestic communications that weren't bundled but that because they mentioned Osama bin Laden's phone number (or whatever selector), were at least interesting to the NSA. But those other bundled domestic communications were particular problematic because they broke the rules and weren't of interest.

In another instance, FISC made NSA double check that data collected during a period when its post-collection checks ensuring targets really were located overseas were on the fritz, to make sure that targets hadn't entered the US while they were targeted.

On at least four other occasions, the FISC used that "exclusive means" section to ensure (or to try to ensure) that NSA didn't get away with keeping the data it obtained by breaking the rules. On two occasions, for example, the FISC made NSA get rid of data it retained in "management systems" that it was supposed to purge. Though not before the government argued that prohibitions on using unlawfully collected information "only applied to interceptions authorized by the Court and did not apply to the fruits of unlawful surveillance." NSA kept data collected under Section 702 like this for five years until finally destroying it last year.

So for four years, the NSA had been collecting entirely domestic communications that fit no intelligence purpose without telling the court. When he learned about it, Bates did what he had done the year before—he told the NSA they couldn't use the data that violated the rules, and within a year, the NSA deleted it.

But in spite of the fact that Bates knew the NSA would collect these entirely domestic communications going forward—both those communications that did mention something like Osama bin Laden's phone number, and those that did not but instead got sucked up as part of a bundle—he let the NSA continue to collect it, with a few new rules attached. He did, however, impose one prohibition. When he newly permitted the NSA to search on its collections using a selector used by Americans (sometimes referred to as "back door searches"), he prohibited such searches on upstream data. That way, no American's communications that were originally collected without being targeted or in mistaken belief they were foreign could be accessed later via a backdoor search.

That's the rule the NSA broke—was breaking, for five years—after Bates imposed the rule.

It was clear pretty quickly the NSA was violating this prohibition, that it didn't have the infrastructure in place to follow it. By May 2013, for example, NSA's overseers pointed out that NSA wasn't tracking back door searches in a centralized place, so it couldn't actually track them easily. Six months later, NSA's overseers identified one of the problems the agency continues to use to explain breaking this rule: the NSA's systems required analysts to opt out of upstream searching, rather than having that work automatically. The next year, overseers suggested NSA require analysts to say whether they thought they were querying on a US person while doing upstream searches to eliminate another of the excuses for the problem. So it's not like NSA didn't know it was breaking the rules; it's just that it never chose to make it harder to break the rules. But when, in 2016, the results of closer investigations conducted by NSA's Inspector General and Oversight Department started coming in, it became clear the problem was far worse than NSA's other overseers had been able to see.

"It will still be possible for the NSA to acquire [a bundled communication] that contains a domestic communication."

It took three years after identifying the problem before NSA figured out just how bad it was.