Since the 2013 Snowden disclosures revealed the extent of government surveillance programs it’s been a standard claim by intelligence agencies, seeking to justify their push for more powers, that their ability to track suspects using new technologies is under threat because of growing use of end-to-end encryption by technology companies.

For example, in a speech in fall 2014, FBI director James Comey asserted: “The law hasn’t kept pace with technology, and this disconnect has created a significant public safety problem. We call it “Going Dark”… We have the legal authority to intercept and access communications and information pursuant to court order, but we often lack the technical ability to do so.”

More recently, in the UK, the government has claimed expanded surveillance legislation — including a proposal to record and store details of every website citizens visit for a full year — are necessary to plug so-called “capability gaps” for intelligence agencies. The wording of the draft Investigatory Powers bill even implies that end-to-end encryption will stand outside the law since comms providers will apparently be legally required to hand over data in a legible form.

However a new study, published yesterday, by Harvard University and funded by the Hewlett Foundation, debunks the notion that surveillance agencies are struggling with a data blackout. On the contrary, it argues, the rise of connected devices (the so-called Internet of Things) presents massive opportunities for surveillance, bolstered by technology companies having business models that rely on data-mining their own users — providing an incentive for them not to robustly encrypt IoT data.

As I wrote last year, when it comes to the Internet of Things and privacy, the risk is “that an embedded ‘everywhere Internet’ becomes a highly efficient, hugely invasive machine analyzing us at every turn in order to package up every aspect of our existence as a marketing opportunity”.

Or security expert Bruce Schneier — one of the signatories of the report — writing on IoT and privacy back in May 2013…

In the longer term, the Internet of Things means ubiquitous surveillance. If an object “knows” you have purchased it, and communicates via either Wi-Fi or the mobile network, then whoever or whatever it is communicating with will know where you are. Your car will know who is in it, who is driving, and what traffic laws that driver is following or ignoring. No need to show ID; your identity will already be known. Store clerks could know your name, address, and income level as soon as you walk through the door.

The aim of the 37-page Harvard report, which involved contributions from technical experts like Schneier, along with US government counterterrorism officials, civil liberty advocates and Harvard law academics, is to bring a more balanced perspective to the policy debate around surveillance, according to Harvard’s Jonathan Zittrain (another report signatory), who convened the group, speaking to the New York Times yesterday.

“We managed to do that in part by thinking of a larger picture, specifically in the unexpected ways that surveillance might be attempted,” he said.

Discussing why intelligence agencies seem apparently so blind to the surveillance potential of the IoT, Zittrain told TechCrunch: “I think, as you suggest, it’s a difference between the capabilities they have right now and what they’re likely to have in the future. They’re focused on the here and now. Also, law enforcement is grounded in the old telephony model — one app in overwhelming use that can be regularly gone to for surveillance. Something as unruly as today’s IoT probably doesn’t seem like a good candidate for routine monitoring.

“But that will change as the IoT space likely consolidates, with a few go-to paths emerging for surveillance. I think that’s why it’s important to think through boundaries on that surveillance now — before it becomes routine. And IoT companies need to take seriously the privacy of the telemetry they receive — and what they *could* receive with a remote tweak to their devices.”

One notable signatory on the intelligence agency side is the former director of the National Counterterrorism Center, Matthew G. Olsen. Although two current senior NSA officials — John DeLong, the head of the agency’s Commercial Solutions Center, and Anne Neuberger, its chief risk officer — are also described as “core members” of the group, albeit the NYT notes they were unable to sign the report as they could not act on behalf of the agency or the US government in endorsing its conclusions.

The report asserts that “communications in the future will neither be eclipsed into darkness nor illuminated without shadow”, emphasizing the role played by commercial companies in eroding data privacy.

“Market forces and commercial interests will likely limit the circumstances in which companies will offer encryption that obscures user data from the companies themselves, and the trajectory of technological development points to a future abundant in unencrypted data, some of which can fill gaps left by the very communication channels law enforcement fears will “go dark” and beyond reach,” it adds.

Core findings of the report include:

that end-to-end encryption and its tech ilk are unlikely to be “adopted ubiquitously by companies” — given that the majority of businesses providing such services rely on “access to user data for revenue streams and product functionality, including user data recovery should a password be forgotten”

that the fragmentation of software ecosystems works against widespread and comprehensive encryption, given it would require “far more coordination and standardization than currently exists”

that projected substantial growth in the number of networked sensors/IoT devices has the potential to “drastically change surveillance” — with the report noting “still images, video, and audio captured by these devices may enable real-time intercept and recording with after-the fact access”, offering a workaround for intelligence agencies being unable to monitor a target through an encrypted channel

that metadata is not encrypted, and the report asserts the “vast majority” is likely to remain unencrypted because it is required for systems to operate (e.g. location data from cell phones and other devices, telephone calling records, header information in e-mail). “This information provides an enormous amount of surveillance data that was unavailable before these systems became widespread,” it notes

The report adds that the various trends it has identified raise “novel questions” about how to protect individual privacy and security in the future — a topic that was also worrying the FTC chairwoman at the start of last year, when she called for IoT companies to adopt security by design, engage in data minimization practices, increase transparency and provide consumers with notice and choice for unexpected data uses.

“Today’s debate is important, but for all its efforts to take account of technological trends, it is largely taking place without reference to the full picture,” the report concludes.

That in turn begs the question why governments and intelligence agencies are being so partial in their arguments as they seek to justify expanded surveillance powers. But if the imperative is to landgrab as much access to data as possible then narrowing the debate to focus on specific technologies such as end-to-end encryption makes sense as a way to distract attention from other potential surveillance avenues, such as IoT and location metadata. In other words, it’s pure misdirection.

In any case, whether such incomplete arguments will pass muster with legislators, the judiciary and the general public remains to be seen. But the tug-of-war between technology and politicians will of course continue.