Thursday's revelation that US and British intelligence agencies are able to decode most Internet traffic was a transforming moment for many, akin to getting definitive proof of intelligent extraterrestrial life. It fundamentally changed the assumptions that many of us have about the tools hundreds of millions of people rely on to shield their most private information from prying eyes. And it challenged the trust placed in the people who build and provide those tools.

But the reporting from The New York Times, ProPublica, and The Guardian was short on technical details about exactly how cryptographic technologies such as virtual private networks and the secure sockets layer (SSL) and transport layer security (TLS) protocols are bypassed. As stated recently by Edward Snowden, the former National Security Agency (NSA) contractor who leaked highly classified documents leading to the reports, "Encryption works. Properly implemented strong crypto systems are one of the few things you can rely on." How is it, then, that agents from the NSA and its British counterpart, known as the Government Communications Headquarters (GCHQ), are reportedly able to bypass the crypto protections provided by Internet companies including Google, Facebook, Microsoft, and Yahoo?

The short answer is almost certainly by compromising the software or hardware that implements the encryption or by attacking or influencing the people who hold the shared secrets that form one of the linchpins of any secure cryptographic system. The NYT alludes to these techniques as a combination of "supercomputers, technical trickery, court orders, and behind-the-scenes persuasion." The paper went on to refer to technologies that had been equipped with backdoors or had been deliberately weakened. Snowden put it slightly differently when he said: "Unfortunately, endpoint security is so terrifically weak that NSA can frequently find ways around" encryption. Exploiting the implementations or the people behind these systems can take many forms. What follows are some of the more plausible scenarios.

Can’t you hear me knocking?

Backdoors are among the easiest ways to bypass encryption, and they can take many forms. Most often, they're considered to be hidden code that allows an outsider surreptitious access to privileged information or functions without a password or other official credential. But backdoors can just as easily be vulnerabilities that are inserted into source code or designs, or are allowed to remain there after being discovered. The NYT specifically mentioned backdoors placed in micro chips used for encryption, and it also alluded to crypto standards that were manipulated in ways to make them easier to exploit.

One such way would be to tamper with pseudo random number generators used to create strong keys. An NSA-controlled flaw that made these numbers easy to predict would provide agents with a covert and easy-to-use method to extract a key protecting a target's communications. Given the staggering volume of data that the NSA wants the capability of reading, it's reasonable to assume analysts want techniques that work across huge swaths of the Internet. To make the backdoor exploitable on a mass scale, the flaw would have to be present in a widely used design, say, in the cryptographic libraries included in Microsoft's Windows or Web server software, or the OpenSSL package that enables cryptographic functions in Apache and other Web servers.

Rumors of backdoors placed in popular crypto standards at the behest of the NSA have existed since at least 2007. Similar theories surfaced again in 2008 following the discovery of an almost catastrophic vulnerability in the Debian distribution of Linux. It also involved random numbers and caused vulnerable machines to generate dangerously weak cryptographic keys. I used to dismiss those kind of thoughts as conspiracy theories that bordered on paranoia. After all, crypto is hard, and it's painfully easy to make honest mistakes. Now, I'm not so sure.

Stealing (or asking for) the keys

Another way to easily break encryption is to obtain the keys that encrypt and decrypt data. The easiest way to get the keys is to simply ask for them, and if that doesn't work, one could use a combination of court orders, persuasion, or threats to coerce them out of the holder. Barring any of those methods, the feds might hack into the servers of large companies and steal them. This method has a few inefficiencies to it. For one, under some versions of this scenario, the feds must obtain a different set of keys for each service they want to monitor, making this method less scalable. And for another, in theory at least, it wouldn't be practical against sites such as Google that have implemented perfect forward secrecy into their cryptographic protections. That's the property that blends private keys held by both the website and an end-user to create a new temporary key that changes all the time. Unless the feds know of a flaw in the Diffie-Hellman key exchange process at the heart of this scheme, it wouldn't be enough to simply obtain the private key of Google or other sites that use perfect forward secrecy.

The feds might also hack or coerce one of the many certificate authorities who validate SSL and TLS keys into providing a master certificate that would work across one or more Internet addresses. While not impossible, this method also seems impractical. First, such certificates would be useful only if the NSA was able to impersonate the website in what's known as an active man-in-the-middle attack, which can make the attack less scalable and harder to pull off. That forecloses the possibility of a passive eavesdropping, in which the NSA simply monitors and decrypts traffic passing between a website and a target. More importantly, the technique is easily detected through what's known as certificate pinning that's built into Google's Chrome browser, dedicated Twitter apps, and some security software.

The take away

One of the more frustrating aspects to the reporting on the Snowden leaks is the lack of specifics. If we don't know exactly how the NSA bypasses Internet crypto it's hard to take any action to prevent it. That said, crypto and security expert Bruce Schneier has compiled a list of concrete things readers can do to at least make intelligence agency surveillance harder. The measures include the use of the Tor anonymity service; the use of software such as GPG, TextSecure, RedPhone, TrueCrypt, OTR, SilentCircle, and BleachBit to encrypt messages, calls, and files; and a robust operations security regimen to lock down endpoints, including the use of air-gapped computers when working with truly sensitive data.

Snowden and Schneier have both counseled people to trust the math that underlies cryptography. Of course, the challenge is ensuring that the software, hardware, or people implementing that math haven't been compromised, and that's becoming increasingly hard to gauge in this post-Snowden era.