Ladar Levison founded secure e-mail service Lavabit in 2004. His company garnered international attention when it was revealed to be the preferred provider for Edward Snowden, a former NSA contractor and whistleblower now living in exile in Russia. In July 2013, the American government ordered Lavabit to hand over the SSL keys to the entire website, which would have allowed them to read every single user's e-mail—not just Snowden's. Levison complied with the order by printing the keys on paper in a tiny font, which gave him enough time to shut down the service. He is actively fighting the government in court, and he recently joined forces with another related company, Silent Circle, to create the forthcoming Dark Mail Alliance.

Lavabit was designed to protect the privacy of e-mail by allowing users to encrypt messages stored on the Lavabit servers. Once encrypted, an e-mail could only be decrypted with a user’s password. The system was made to protect messages on Lavabit’s servers from prying eyes. Quite simply, the goal was to remove Lavabit from the surveillance equation.

In response to the recently announced Dark Mail Alliance, famed security researcher Moxie Marlinspike penned an op-ed in which he makes a number of interesting points. His arguments are well-reasoned and his contributions to the community are worthy of note, so I feel compelled to respond to his critique of Lavabit’s design.

Marlinspike’s arguments carry weight because his facts are largely correct: the Lavabit servers took plaintext in and sent plaintext out. The system relied upon SSL to protect the plaintext in transit. This is where Marlinspike goes astray, though:

The ciphertext, key, and password are all stored on the server using a mechanism that is solely within the server's control and which the client has no ability to verify. There is no way to ever prove or disprove whether any encryption was ever happening at all. Whether it was or not makes little difference… The cryptography was nothing more than a lot of overhead and some shorthand for a promise not to peek.

Marlinspike is assuming that the Lavabit system was designed to be a substitute for the security provided by end-to-end encryption systems like PGP. It was not. Lavabit’s encrypted storage feature was designed solely to protect e-mails at rest.

Why protect the data at rest? To limit the data Lavabit could access and therefore turn over to the government. When the government seeks a specific user’s data, it is given what is stored on disk, e.g., any metadata found in a server’s log files and any e-mail content the provider has access to. Since I find the prospect of violating a user’s privacy to be unethical, I designed Lavabit to minimize the amount of information I could turn over, at least in a useable form. For years, Lavabit was successfully removed from the surveillance equation.

An improvement, not a fix

I wasn’t trying to fix e-mail’s overall security problem. I was only trying to improve privacy and security in a way that was completely transparent to users. My system did nothing to protect e-mail messages before they arrived at Lavabit or after they were downloaded onto the user’s computer.

Let’s dig a little deeper into how things worked. For starters, Lavabit didn’t record any personal information in its log files. If the feds subpoenaed a user’s metadata from our log files, we had nothing to surrender. If the account was using the secure storage feature, any e-mails stored on the server were encrypted. This left nothing of value to turn over. In other words, I could not access or capture any of the plaintext e-mail content and surrender it because the only place user e-mails were available in their plaintext form was in memory or inside the SSL tunnel while traveling down the wire.

Marlinspike claims that if an attacker gained access to a Lavabit server, they would be able to get users' e-mails in their unencrypted form by extracting the data from memory. With most systems he’d be correct. The Lavabit mail daemon, however, was designed to resist efforts to access the memory allocated to it by the operating system. Specifically, the daemon allocated a region of “secure” memory (an area of memory designed to resist snooping) and used it for storing sensitive data such as passwords and encryption keys. Without getting into all the details (See the NIST and CERN secure coding guidelines for additional information), I’ll hit the main points.

Secure memory is immediately wiped when the data being stored is no longer needed. To protect against sensitive data being leaked via a system fault, the daemon catches fatal errors and wipes the secure memory before terminating and potentially dumping the current state to a core file on disk. The daemon will also detect and reject attempts to attach a debugger to the process (when compiled for production use). I won’t naively claim it’s impossible to extract sensitive data from memory, but I will say that it would have been very difficult. If an attacker with access to the server wanted to intercept user data, they would probably need to replace the production binaries with a trojaned version—a difficult task for your average attacker who lacks access to the source code. (Naturally it’s easier to modify the daemon if the code is available as free and open-source software.)

If Marlinspike's hypothetical attacker is the feds, they could have forced Lavabit to modify the code and deploy it—it has been done before. Congress has passed laws requiring companies like Lavabit to modify their software to meet new legal requirements. But whether a court can compel such a change in order to facilitate surveillance remains an open question; some products and services are exempt from the Communications Assistance for Law Enforcement Act (CALEA), an American wiretapping law.

I maintain that our government can’t compel such modifications, but I always realized that assumption might need to be defended in court. If the battle was lost, I decided long ago that I would shut down Lavabit. That said, it was the possibility of forced modification that led me to write on the secure page Marlinspike referenced:

If you’re intent on hiding your communications from the government, we recommend you investigate systems that secure messages throughout the entire e-mail system and not just at one particular point along that journey.

It never occurred to me that the feds might demand Lavabit’s SSL key. It simply wasn’t part of my threat model. If I were to highlight one of my personal failings in this ordeal, it would be that oversight.

A changed landscape

When I was designing the Lavabit encrypted storage feature in 2004, it simply wasn’t possible for an attacker to intercept and decipher a large number of SSL connections in real time. This assumption was presumed true even if an attacker managed to gain access to the SSL private key. The situation has obviously changed. Network tools now decipher SSL connections efficiently, and servers are fast enough to make this attack a reality. A theoretical weakness became practical. I missed that development. More importantly, I failed to update the Lavabit SSL configuration to prefer ciphers that provided perfect forward secrecy.

In my defense, I wasn’t the only person to miss this change in the threat landscape. Almost everyone else did, too. Returning to my particular situation for a moment; the reason I fought so hard to have my case unsealed was so others could learn from my misfortune. The events leading up to the shutdown of Lavabit illustrate in dramatic fashion just how drastically the threats to e-mail security have changed.

From a technical standpoint, this means administrators need to reconfigure servers so they prefer SSL ciphers that provide perfect forward secrecy. Likewise, software developers need to modify their applications and libraries to always use an SSL cipher that provides perfect forward secrecy, if available, even when it’s not the preferred cipher. Forward secrecy would at least limit the damage of a compromised SSL key in the future.

To fix the situation permanently, we’ll need to make end-to-end encryption of e-mail automatic and ubiquitous so non-technical users can benefit. For that to happen, we’ll need to rethink and replace the mail protocols in use today. If support for end-to-end encryption is added to the protocol, clients can handle the encryption process automagically. We have the ability to build messaging systems that look and function like what we have come to expect from e-mail but are in fact completely secure end to end. Building such a system, in short, is the goal of Lavabit’s Dark Mail Initiative. We want to release software into the public domain that is capable of making everyone’s e-mails go dark.

On the legal side, I’m still looking for the Fourth Circuit Court of Appeals (or perhaps even the Supreme Court) to provide relief. The appellate court could quash the subpoena and search warrant that forced me to surrender Lavabit’s SSL keys and compromise the system. A victory would certainly make it difficult for the feds to demand SSL keys in the future.

A victory in court would go a long way, but if the goal is restoring trust in American technology products and online services, we’ll also need a political victory. Only Congress can solve the privacy problem.

Legislation can ensure that the courts don’t have the authority to compel companies to secretly compromise a system’s security for the benefit of law enforcement. Conversely, if companies can be compelled to act against the interests of their users, then Congress should ensure that the limits of that authority are defined by law. Only then can business owners make informed decisions about whether to operate in the United States. Consumers will need to make similar decisions about whether to trust the protection of their private information to companies with physical ties to the United States.

Expanding my viewfinder for a moment, if we learn only one lesson from the Summer of Snowden, it should be how hostile the Internet has become. Media outlets have repeatedly reported on the insatiable appetite the US government has for the private information of citizens.

Reports continue to emerge suggesting that companies are secretly weakening the security of their products and services. Precisely which companies are complicit in the global slurping of private data remains a mystery, but the message is clear. Partial solutions like the Lavabit secure storage system are no longer adequate to protect the privacy of users. What is needed are new protocols that incorporate encryption and function securely by default. If you’re looking for something positive to emerge from something that has been mostly negative, it's that the world may finally be ready to invest the effort needed to go dark.

Listing image by Cyrus Farivar