Believe it or not, journalists can keep secrets. Nor do they keep any old secrets. They keep state and security secrets. In the last six months the journalists to whom the NSA-GCHQ archive was passed by Edward Snowden have kept more secrets safe than the entire Anglo-American intelligence community did in a decade. But then the trouble with a secret is that nobody knows when you keep it, only when you do not.

This week the British judiciary has come face to face with the same problem. Two defendants, AB and CD, are on trial at the Old Bailey as "terrorism suspects". The crown prosecution service has asked the judge to impose anonymity on the defendants, the charges, the evidence and indeed the reporting of the trial. Even the reasons for asking should be secret. "Public interest immunity certificates" have floated in the air like snow as judges, prosecutors and police prepared a witches' Sabbath of secrecy.

Not so fast, said the admirable Mr Justice Sweeney on Monday. He wanted a clear explanation to his court as to why secrecy was necessary. Any exemption from the principle of open justice, he said, "must be very clearly made out after every party [including counsel for the press] has the opportunity to make submissions". All the CPS had offered as justification were "operational reasons" and "to prevent the administration of justice from being damaged". It must now do better.

The argument is a microcosm of the debate consuming the privacy-security nexus across the Atlantic. The state requires confidences to be maintained if it is to work effectively, as does any organisation dependent on human relationships. Cabinets conducted in the public gaze would be not government but performance. State secrets must be guarded against known threats. Even with ordinary policing which embraces counterterrorism, active operations may require temporary blackouts and injunctions.

I may differ from some of my colleagues in accepting that such secrecy may require legal defence. Good governance needs confidentiality. During President Reagan's notorious Iran-Contra affair, the official conduits of diplomacy proved so leaky that private "back channels" took over. The result was disaster. When every participant keeps a diary for publication and every hard drive is evidence, power retreats behind cronyism and sofas, as under Tony Blair's regime. The present inefficiency of Whitehall is the outcome. Nobody trusts anybody with a secret.

In Dave Eggers' new novel The Circle, the mighty digital corporation taunts those holding public office into "going clear". They are wired with cameras and mics so their every movement day and night is broadcast on the web. Any who refuse "must have something to hide". Soon businesses, schools, even partners are drawn into doing likewise. The ruling mantra is, "Secrecy is lies: privacy is theft".

This may be a novelist's exaggeration. Yet it is plainly the direction of travel revealed by Snowden of the wilder shores of the NSA and GCHQ. What made his revelations particularly important was the spectacle of two contrasting forces in ghastly collusion. On the one hand was the move to digital transparency, the erosion of personal privacy through electronic surveillance, drone photography, face recognition and type-sorting. On the other was the hijacking of such power by a secret agency of the state under cover of "the war on terror". The agency sought to penetrate every citizen's life for undisclosed reasons, yet demand immunity from any oversight. This was asymmetrical democracy.

Any claim that "everyone knew these things were going on" is rubbish. Few even within the security realm were aware that servers were being hacked, encryptions corrupted and undersea cables tagged. They did not know because the means put in place to inform them were, as Snowden clearly shows, being circumvented and disobeyed. As Lord Ashdown said today, the nerds were "out of control".

This brings us back to the Old Bailey. What is it the police wish to keep secret and why? I have no problem with keeping names secret in some trials, as with certain sex crimes and family cases. Nor is Sweeney opposing secrecy in the AB/CD case. He merely regards justice as requiring openness prime facie. If it cannot be open, then the reasons for closure must be open. He and his court must be persuaded, not just told.

The days are over when those demanding secrecy for their work can offer the dismissive excuses of the three intelligence chiefs to Westminster's intelligence and security committee earlier this month. This boiled down to "Trust us, we are policemen". How could anyone trust people who delivered Britain's entire wardrobe of state secrets, some 60,000 files, to a potential audience of up to 800,000 Americans, including an honest but appalled private contractor named Snowden – and that after the Manning-Assange revelations. They must have known it would all leak.

British intelligence was saved from catastrophe only by Snowden not dumping his material on the web but giving it to what he regarded as responsible journalists. The newspapers, notably the Guardian, published less than 1% of the material judged as clearly in the public interest. This was after consulting (if not always agreeing with) security authorities on both sides of the Atlantic. British intelligence asserts that national security was endangered and lives put at risk. No American source has repeated that claim, but rather acknowledged a gross intelligence failure badly in need of correction.

The world now faces total electronic penetration, with huge power available to those who control it. The idea that the assurances of a policeman or spy are "good enough for me" has been shown as deluded. No group should be trusted with such unconstrained leverage over others, least of all one revealed as systematically deceiving Congress.

Expecting the west's arcane democratic institutions to police the new digital power of states is to build a sandcastle against a tank. Yet they are all we have. Parliamentarians who discover they have been deceived by the spies must howl blue murder, as they are doing in America but not in Britain. Sweeney must be persuaded for himself how far judicial openness should be suspended to help the police.

The press, showered with leaks, must resort to its own educated judgment in deciding where the public interest lies. Everyone knows secrets must be kept, but keeping them needs a framework built on public trust. That framework must be informed and argued. It can no longer rely on the bark of command and a cringing deference to the gods of security.