When the NSA wiretapping story first hit the pages of the NYT a few days ago, there were clearly a huge number of unanswered questions. Is the wiretapping that the President has authorized illegal under the FISA act? Is it unconstitutional? If it's illegal, does the President have the authority to violate the law if he's acting in the best interests of the republic? And then there's the question of why the NYT sat on this story for over a year before going public with it.

I'm not really going to make any attempt to answer questions of legality and constitutionality, because the Internet is full of armchair constitutional scholars right now who're fighting tooth and nail over these questions, generating much heat but very little light. Instead, I'd like to point your attention to some later developments in this case that clearly indicate that there's much more going on here than we initially assumed. When the truth comes out (if it ever does), this NSA wiretapping story will almost certainly be a story not just about the Constitutional concept of the separation of powers, but about high technology.

To return to the last question in the first paragraph, let's take a look at the NYT's own answer. The quotes below are from NYT executive editor Bill Keller's statement on the matter:

A year ago, when this information first became known to Times reporters, the Administration argued strongly that writing about this eavesdropping program would give terrorists clues about the vulnerability of their communications and would deprive the government of an effective tool for the protection of the country's security... As we have done before in rare instances when faced with a convincing national security argument, we agreed not to publish at that time. "We also continued reporting, and in the ensuing months two things happened that changed our thinking... Second, in the course of subsequent reporting we satisfied ourselves that we could write about this program -- withholding a number of technical details -- in a way that would not expose any intelligence-gathering methods or capabilities that are not already on the public record.

(Emphasis in the above quote and in all subsequent quotes is added.)

So the NYT sat on this story for a year in part because they were concerned that they wouldn't be able to report it without revealing some crucial technical details of how the program works.

Now let's take a look a statement of former senator Bob Graham (D-FL), who was one of the few senators to be briefed on the program. From a new Washington Post article:

"I came out of the room with the full sense that we were dealing with a change in technology but not policy," Graham said, with new opportunities to intercept overseas calls that passed through U.S. switches.

Kevin Drum at the Washington Monthly has rounded up a few more quotes like those above (including the NYT quote), that also help make a very good case that what's at issue here is some kind of new NSA surveillance technology:

Attorney General Alberto Gonzales, telling reporters why Bush didn't simply ask Congress to pass a law making the program clearly legal: "We've had discussions with members of Congress, certain members of Congress, about whether or not we could get an amendment to FISA, and we were advised that that was not likely to be ? that was not something we could likely get, certainly not without jeopardizing the existence of the program, and therefore, killing the program."

President Bush, answering questions at Monday's press conference: "We use FISA still....But FISA is for long-term monitoring....There is a difference between detecting so we can prevent, and monitoring. And it's important to know the distinction between the two....We used the [FISA] process to monitor. But also....we've got to be able to detect and prevent."

Senator Jay Rockefeller, in a letter to Dick Cheney after being briefed on the program in 2003: "As I reflected on the meeting today, and the future we face, John Poindexter's TIA project sprung to mind, exacerbating my concern regarding the direction the Administration is moving with regard to security, technology, and surveillance."

This last quote above, the one about TIA, is especially telling. TIA was a massive electronic intelligence gathering program designed to mechanically sift through phone calls, emails, and other electronic communications in order to build pictures of how individuals fit into larger networks. We covered TIA here on Ars, but of all the coverage I think Caesar's initial take on it seems the most directly applicable to the current situation:

This system's purpose would be to monitor communications and detect would-be terrorists and plots before they happen... This project is not interested in funding "evolutionary" changes in technology, e.g., bit-step improvements to current data mining and storage techniques. Rather, the amount of data that the directors are anticipating (petabytes!) would require massive leaps in technology (and perhaps also some massive leaps in surveillance laws). According to DARPA, such data collection "increases information coverage by an order of magnitude," and ultimately "requires keeping track of individuals and understanding how they fit into models."

"Massive leaps in surveillance laws" indeed. TIA became public in 2002, and Congress quickly put the kibosh on it. This is right about the time that Bush secretly signed the executive order authorizing the new NSA wiretap program.

So, are TIA and the NSA wiretapping directive related? That probably depends on what you mean by "related." I doubt seriously they're the same thing, but it's entirely possible that the undescribed new technology used in the NSA wiretapping program was also going to be deployed as a part of TIA's massive data collection efforts.

My main point in bringing up TIA is twofold: 1) TIA-like efforts are still going on (Defensetech catalogs some), and 2) the government has been trying to use new technology, like database tech and voice recognition, for domestic surveillance for a long time. And when I say a long time, I mean well before the current administration came into office.

The domestic electronic surveillance ball really got rolling under the Clinton administration, with the 1994 Communications Assistance for Law Enforcement Act (CALEA). CALEA mandated that the telcos aid wiretapping by installing remote wiretap ports onto their digital switches so that the switch traffic would be available for snooping by law enforcement. After CALEA passed, the FBI no longer had to go on-site with wiretapping equipment in order to tap a linethey could monitor and digitally process voice communications from the comfort of the home office. (The FCC has recently ruled that CALEA covers VOIP services, which means that providers like Vonage will have to find a way to comply.)

CALEA opened up a huge can of worms, and PGP creator Phil Zimmermann sounded the alarm back in 1999 about where the program was headed:

A year after the CALEA passed, the FBI disclosed plans to require the phone companies to build into their infrastructure the capacity to simultaneously wiretap 1 percent of all phone calls in all major U.S. cities. This would represent more than a thousandfold increase over previous levels in the number of phones that could be wiretapped. In previous years, there were only about a thousand court-ordered wiretaps in the United States per year, at the federal, state, and local levels combined. It's hard to see how the government could even employ enough judges to sign enough wiretap orders to wiretap 1 percent of all our phone calls, much less hire enough federal agents to sit and listen to all that traffic in real time. The only plausible way of processing that amount of traffic is a massive Orwellian application of automated voice recognition technology to sift through it all, searching for interesting keywords or searching for a particular speaker's voice. If the government doesn't find the target in the first 1 percent sample, the wiretaps can be shifted over to a different 1 percent until the target is found, or until everyone's phone line has been checked for subversive traffic. The FBI said they need this capacity to plan for the future. This plan sparked such outrage that it was defeated in Congress. But the mere fact that the FBI even asked for these broad powers is revealing of their agenda.

Read the quote above carefully, and see if it doesn't ring any bells for you. The salient points that Zimmermann makes are these:

In 1995, back when the Pentium Pro was hot stuff, the FBI requested the legal authorization to do very high-volume monitoring of digital calls.

There's no way for the judicial system to approve warrants for the number of calls that the FBI wanted to monitor.

The agency could never hire enough humans to be able to monitor that many calls simultaneously, which means that they'd have to use voice recognition technology to look for "hits" that they could then follow up on with human wiretaps.

It is entirely possible that the NSA technology at issue here is some kind of high-volume, automated voice recognition and pattern matching system. Now, I don't at all believe that all international calls are or could be monitored with such a system, or anything like that. Rather, the NSA could very easily narrow down the amount of phone traffic that they'd have to a relatively small fraction of international calls with some smart filtering. First, they'd only monitor calls where one end of the connection is in a country of interest. Then, they'd only need the ability to do a roving random sample of a few seconds from each call in that already greatly narrowed pool of calls. As Zimmermann describes above, you monitor a few seconds of some fraction of the calls looking for "hits," and then you move on to another fraction. If a particular call generates a hit, then you zero in on it for further real-time analysis and possible human interception. All the calls can be recorded, cached, and further examined later for items that may have been overlooked in the real-time analysis.

In a recent press conference, Deputy Director for National Intelligence Michael Hayden said the following (via Defensetech):

And here the key is not so much persistence as it is agility. It's a quicker trigger. It's a subtly softer trigger. And the intrusion into privacy -- the intrusion into privacy is significantly less. It's only international calls. The period of time in which we do this is, in most cases, far less than that which would be gained by getting a court order.

This sounds pretty much like what I've described above. And yes, this kind of real-time voice recognition, crude semantic parsing and pattern matching is doable with today's technology, especially when you have a budget like the NSA.

The "softer trigger" here is a phrase that's on a watch list, or a call with an abnormally high volume of a certain type of vocabulary. The "agility" bit is a reference to the technology's ability to move from call to call, taking small slices. That's also probably what's behind the claim that the technology is less intrusive than a traditional wiretap, because the time slices are very short.

Finally, I know a lot of people are bringing up Echelon here, and Echelon is indeed relevant. However, the relevance isn't in how the NSA program is connected to Echelonit probably isn'tbut in the fact that Echelon is yet another example of a government-run, high-volume, automated intelligence gathering project that looks for certain words or phrases in samples of electronic communications.

Addendum: Looking back over this post and discussing it with some non-techie friends of mine, I can see that folks who aren't regular Ars readers may mistake this for some sort of defense of the administration's activities. For the record, I'm one of those "privacy nazis" that people like Michelle Malkin love to hate, and I view any kind of domestic spying--automated or not--with hostility.

But aside from my general (and genuinely conservative) knee-jerk anti-government reactions against this program, there's an even deeper criticism that can be leveled against casting such a wide, computer-automated net. I've posted this criticism as a followup post, so read on for more.