Last week in the Hague, Barack Obama seemed to have suddenly remembered the oath he swore on his inauguration as president – that stuff about preserving, protecting and defending the constitution of the United States. At any rate, he announced that the NSA would end the "bulk collection" of telephone records and instead would be required to seek a new kind of court order to search data held by telecommunications companies.

This policy change is a tacit admission of what Edward Snowden (and 2001 whistleblower William Binney before him) had been claiming, namely that the warrantless surveillance of US citizens by the NSA and other government agencies does, in fact, violate the constitution of the United States. Obama's announcement looked to some observers as the first crack to appear in the implacable facade of the national surveillance state. This looked promising because, as we know from second world war movies, the first crack is inevitably the harbinger of the eventual total collapse of the dam.

Dream on. The significant thing about Obama's announcement is the two things it left out: surveillance of the internet (as distinct from the telephonic activity of American citizens); and of the rest of the world – that's you and me. So even if Obama succeeds in getting his little policy swerve through Congress, the central capabilities of the national surveillance state will remain in place, untouched and unimpaired.

At the heart of these capabilities is the "bulk collection" (that is, warrantless) collection and storage of communications metadata on an unimaginable scale. Given that metadata in this context is essentially a log of every communicative act that you make in cyberspace – where you went; who you emailed or texted; who emailed or texted you; the URL of every website you visited; a list of every web search you've ever made; and so on – metadata nowadays constitutes information of a very detailed and intimate nature.

It's the kind of data that in a civilised society would only be accessible to state authorities under very strict conditions. Yet we now know, thanks to Snowden, that all of it is freely available to the NSA and its overseas franchises.

This is intolerable, for various reasons. The first and most obvious one is the intimacy of the data that is being collected. What is even more offensive is the speciousness of the rationale that is trotted out by state authorities to "justify" it. This goes back to the era of analogue telephony when the US supreme court decided that the "metadata" of a telephone subscriber consisted of a log of the numbers s/he called, and that this log was the product of the telephone company, not of the subscriber. This was perhaps a not unreasonable judgment in an analogue era, but it is entirely inapplicable in a digital one. Our metadata should belong to us and should only be accessible under judicial supervision.

Equally offensive is the argument, regularly trotted out by government apologists, that "collection" does not mean what any normal person thinks it means. Sure, they say, the metadata may be hoovered up by the spooks' machines, but it isn't actually "collected" until it has been looked at by a human being. This is either breathtaking casuistry or evidence of startling official ignorance of current capabilities in machine learning and pattern recognition. Either way, it's bullshit.

Related to that is the way in which bulk collection of metadata undermines a fundamental principle of any civilised legal system – the presumption of innocence until proved otherwise. Current NSA/GCHQ practice effectively turns every citizen into a suspect to be surveilled, just in case, at some time in the future, the state decides to take an interest in him or her.

Finally, there is the legal issue. Ever since Snowden began telling it like it is, the ultimate fall-back position of the establishment has been that what the spooks are doing is "lawful". Even if we accept that proposition, there is still the problem of bad laws caused by incompetent or dishonest law-making. And what we are now discovering is how flawed the US law-making process relating to warrantless surveillance was. In the UK, the parliamentary process that led to RIPA, the Regulation of Investigatory Powers Act, was also flawed but in a different way. In the US, it was Dick Cheney and the politicians who decided what they wanted the NSA to do and drafted the law accordingly. Over here, the spooks told the politicians what they wanted and the legislators obliged. This is no way to run a democracy.

* This article was amended on 31 March. In the paragraph "This is intolerable…" the author meant to say "perhaps a not unreasonable judgement" rather than "not reasonable".