"Metadata is extraordinarily intrusive. As an analyst, I would prefer to be looking at metadata than looking at content, because it's quicker and easier, and it doesn't lie."

Image: Screenshot

In just two sentences, Edward Snowden nailed the hypocrisy — or perhaps it's really just stupidity — at the heart of the Australian government's efforts to sell a mandatory data-retention scheme.

Snowden was appearing via a video link at the Moment of Truth event in New Zealand on Monday night. He was speaking to ardent fans — he scored a standing ovation laced with the tribal whooping of "Yeah!" and "Woo!" before he'd even begun — so he had an easy run. But he also spoke with a clarity that's hard to argue against.

"If I'm listening to your phone call, you can try to talk around things, you can use code words. But if I'm looking at your metadata, I know which number called which number. I know which computer talked to which computer. And yeah, that [capability to access metadata] exists comprehensively for all the Five Eyes analysts," Snowden said.

The signals intelligence agencies of all Five Eyes nations — the US, the UK, Australia, Canada, and New Zealand — have access to the NSA's XKeyscore, a federated search system that deals in metadata captured from the NSA's interception of international fibre links, as well as other sources.

XKeyscore also searches the last three to five days of content data, Snowden said, and that archive is growing — but that's another story.

Snowden isn't the first person to point out that metadata can be more revealing than content. Far from it.

Nigel Phair, who used to head up investigations for the Australian Federal Police's High-Tech Crime Centre, has said exactly the same thing.

"There is a wide range of information that goes under the umbrella of metadata, and I would argue in many instances, it's probably more valuable than the content that comes with it," Phair told ABC Radio National's Sunday Extra last month.

Or, as Stewart Baker, former general counsel of the NSA, said last year: "Metadata absolutely tells you everything about somebody's life. If you have enough metadata, you don't really need content."

Of course metadata tells you everything. Why else would police and spooks want it? A moment's thought reveals why. The content of the phone call may say, "Honey, I'm working late", but the metadata tells you the call is being made from a rent-by-the-hour motel, an hour after a call was made to an escort agency. You join the dots.

As I've said previously , anyone who's still trying to portray communications metadata as less revealing than the content — less deserving of a process like getting a warrant before it can be accessed — is either a fool or a liar.

Which brings me, once again, to Australia's favourite attorney-general, Senator George Brandis QC.

"We want to maintain the sharp distinction between metadata and content," Brandis said in that train-wreck TV interview on August 6, when he discussed the government's plans to retain internet metadata for use by domestic intelligence and law-enforcement agencies.

"That's why we are developing protocols to try and ensure that the integrity of that distinction is maintained," he said.

Now the reason why domestic agencies want to maintain that supposedly sharp distinction is obvious. Under current Australian law, they can access telephone metadata without a warrant. The police, plus a vast array of other organisations, do so half a million times per year. The spooks do it too, but we have no idea what the numbers might be. All of these people want the same warrantless access to internet metadata. And why not? They're busy people. It makes their job a lot easier.

The government seems to believe that the sharp distinction can be maintained by calling the data produced by us "content", and the data produced by the communication protocols and service providers "metadata".

That was an easy distinction with analog telephones. We talk down the wire, and the phone company creates some business records.

But smartphone apps make things a lot more complicated.

Say we ask an app to show us the nearest Italian restaurant. Our location is transmitted to a server somewhere, and results come back. We didn't type it in, but our location is a key part of that communication. Is it content, or metadata? Both? And what about the fact that our location was also sent to an advertising network? Now write that into a law that makes sense.

Yeah, good luck with that.

The problem with trying to maintain this content-metadata distinction, an accident of technological history , is that it completely misses the point.

The real test of whether specific kinds of private data should be accessible with or without a warrant should be the degree to which it's seen as an invasion of our right to a private life — regardless of how the data was generated, regardless of who generated it, and regardless of where it's stored.

The digital world is still evolving rapidly. More data is being collected than ever before, and it's increasing daily. The sheer scale, scope, and potential of digital surveillance represents a massive shift in the power relationship between the watchers and the watched — no matter how much the government pretends otherwise. But our notions of privacy are still being negotiated.

That means this conversation needs to move beyond the narrow circle of government, police, spooks, and, seemingly begrudgingly, ISPs. Snowden was blunt about this.

"When the bulk collection of private citizens' communications — emails, text messages, location data, metadata, calling records, what you order online, what you buy, who you talk to, who you love, what you do — when these things are collected by any arm of government without an individualised, particularised suspicion of wrongdoing on the individual level, that is a violation not just of rights on a national level, but of human rights that are not given to us by government, but are inherent to our nature," Snowden said.

"I think it's wrong of any politician to take away the public's seat at the table of government and say, 'You'll simply have to trust us.'"

Brandis is not only after a "simply trust us" regarding what metadata would be collected — he can't explain it, because he doesn't understand it — he's also after a "simply trust us" regarding what the metadata would be used for. When asked directly whether it'd be solely for terrorism cases, he weaselled it.

"Our primary focus is terrorism, but the fact is that access to metadata is an extremely useful criminal investigative tool," he said.

So the attorney-general wants to introduce a law, but he won't be straight with us about what it covers and when it'd be used — and he himself doesn't even understand the subject area.

Yeah, good luck with that too.

Now, what if Brandis were clear about the government's intentions? Explained clearly what data would be collected? That it'd be made available with a warrant, or perhaps some more efficient process that still included external oversight, for specific stated crimes — maybe terrorism and child sexual abuse? Made it clear that extending the system to other kinds of crime would require coming back to the people? I reckon most Australians would be OK with that.

But stumbling through the biggest shift in that power balance since the commercialisation of the internet a generation ago, or maybe even the invention of the telephone more than a century before that? Stammering in ignorance? Smearing the specifics with weasel words? Nah, not so much.