On both sides of the Atlantic the battle over encryption is hotting up, with the FBI continuing to press its case for access and the British government making noises about its fears of what an encrypted future might mean. The talk is of how ubiquitous encryption will lead to spies and law enforcement "going dark".

In recent years, the state could compel national telecoms providers to give them access to data traffic, which the spies could then read. But those companies are seeing more and more of what passes through their pipes encrypted by service providers. And since Edward Snowden revealed the extent of government surveillance, those providers and other tech companies have come to see offering privacy as a selling point to their customers. But if end-to-end encryption becomes increasingly ubiquitous, is it the end of the line for the spies? History suggests not.

When encryption first became available to the public in the 70s, thanks to the development of public key cryptography, one of its inventors Whit Diffie had a conversation with Arthur Levenson, a senior figure at US spy agency, the NSA. Diffie told Levenson he thought signals intelligence was finished. Levenson was less sure. "Whit, we've heard these arguments before," Levenson (whose experience stretched back to Bletchley Park) replied. Forty years on, as Diffie recalled the conversation with me, he shook his head with a rueful smile. "I was clearly mistaken," he says. When it comes to signals intelligence, "the sources are fragile, but the phenomenon is robust" as Diffie remembers one official telling him.


One of the things that became clear to me while writing a book on the history of computers and spies, is that the talk of going dark is not new and the smartest spies know they can adapt. Signals intelligence is an inherently insecure business in which the tiniest change can instantly dry up a valuable stream of intelligence.

When Nazi Germany upgraded its Enigma machine as the second world war started, the head of the Government Code and Cypher School (soon to become GCHQ) said it would be "a waste of time and public money" to even try to crack the new codes. But Alan Turing and others took up the challenge, and proved them wrong. As the Cold War started, Soviet codes proved near impossible to break. But the spies instead carried out massive traffic analysis on the externals of communications to extract useful intelligence. By establishing what normal patterns of Soviet military communications were, GCHQ and NSA would look for any change -- this might be an indicator of troops on the move, and potentially war. This was the real -- and secret -- birth of today's buzz word of "big data". Finally, when fibre-optic cables spread in the 90s, the spies again thought their satellite-based collection model was over, but they adapted (as Edward Snowden soon revealed).

Read next These Chrome extensions protect you against creepy web tracking These Chrome extensions protect you against creepy web tracking

But what does adapting mean today?

The point about end-to-end encryption is that there is still an endpoint where the message is clear and readable. And so the spies at GCHQ and NSA will likely shift towards greater exploiting of target endpoints by what they call Computer Network Exploitation -- what everyone else calls hacking. This may also be done on a much larger scale than in the past, with references in some recent reports to something called "bulk" computer network exploitation. There are indications from the Snowden leaks that the US may be able to pre-install large numbers of implants in computers, ready to be activated. Spies will also do what they have done in the past by looking for weaknesses in encryption protocols (as they discovered with Enigma machines) and for any human failings in implementation. These may offer the chink in the armour which clever mathematicians and machines can together work on, as happened again at Bletchley.

Other forms of surveillance may also play a role -- after all, a covertly placed camera above your PC can catch you type in your password and outwit the very best forms of encryption. This kind of activity needs to be authorised if the state wants to do it, however, and new laws planned for the UK are expected to overhaul the entire surveillance system to make it clearer what can and cannot be done ,and who should sign it off (perhaps soon a judge, rather than a minister).

One of Whit Diffie's reflections about why he was mistaken back in the 70s is that, while much of the emphasis is on what proportion of traffic the state can read, there is another part of the equation for signals intelligence. And that is the overall volume of communications that are out there. The trend over the years has been for almost exponential growth and all the signs are that this will continue as we connect up more and more internet of things devices. And not everything will be properly encrypted. In other words, even if a smaller proportion of the communications is readable, there is still more out there overall.


The connected devices in our household, like our fridges and those that we wear and carry, like our watches, are likely to be potentially highly revealing sources of intelligence. The shift towards encryption may also increase the pressure to carry out traffic analysis and extract meaning from metadata rather than the unreadable content. More will also be made of open source forms of intelligence (information searchable from the web and social media like Twitter which might reveal connections, links or locations) -- this is already proving increasingly valuable.

The smartest spies know encryption is coming -- and that they risk being on the wrong side of the argument if they oppose it, as the public increasingly understands its value in protecting their data from a range of malevolent actors like criminals and foreign hackers. The spread may mean spies have a lean period as existing intelligence flows do, in fact, go dark. But history suggests that if they are as smart in the future as they have been in the past, then they will find new ways to do their job.

Intercept: The Secret History of Computers and Spies by Gordon Corera is published by Weidenfeld & Nicolson