Special Report Duncan Campbell has spent decades unmasking Britain's super-secretive GCHQ, its spying programmes, and its cosy relationship with America's NSA. Today, he retells his life's work exposing the government's over-reaching surveillance, and reveals documents from the leaked Snowden files confirming the history of the fearsome ECHELON intercept project. This story is also published simultaneously today by The Intercept. Duncan's Register Christmas Lecture from last year is also, at long last, now available for viewing. In it, he tells the same fascinating story.

Update: Since this piece was published the Intercept have been in touch to say that it is "deceptive" of the Register to suggest that this is a "joint publication". We're very happy to confirm that there has never been any editorial collaboration between the Register and the Intercept.

I stepped from the warmth of our source's London flat. That February night in 1977, the air was damp and cool; the buzz of traffic muted in this leafy north London suburb, in the shadow of the iconic Alexandra Palace.

A fellow journalist and I had just spent three hours inside, drinking Chianti and talking about secret surveillance with our source, and now we stood on the doorstep discussing how to get back to the south coast town where I lived.

Events were about to take me on a different journey. Behind me, sharp footfalls broke the stillness. A squad was running, hard, toward the porch of the house we had left. Suited men surrounded us. A burly middle-aged cop held up his police ID. We had broken "Section 2" of Britain's secrecy law, he claimed.

These were Special Branch, the then-elite security division of the British police.

For a split second, I thought this was a hustle. I knew that a parliamentary commission had released a report five years earlier that concluded that the secrecy law, first enacted a century ago, should be changed. I pulled out my journalist identification card, ready to ask them to respect the press. But they already knew that my companion that evening, Time Out reporter Crispin Aubrey, and I were journalists. And they had been outside, watching our entire meeting with former British Army Signal Intelligence (SIGINT) operator John Berry, who at the time was a social worker.

Aubrey and I were arrested on suspicion of possessing unauthorized information. They said we'd be taken to the local police station. But after being forced into cars, we were driven in the wrong direction, towards the center of London. I became uneasy.

From left, journalists Duncan Campbell and John Crispin Aubrey, with former SIGINT operator John Berry in 1978

(Source: ANL/Re/REX Shutterstock)

It was soon apparent that the elite squad had no idea where the local police station was. They stopped and asked a taxi to lead them there. We were then locked up overnight, denied bail, and sent to London’s Brixton prison.

Aubrey had recorded our interview. During three hours of tapes that the cops took from Aubrey, Berry had revealed spying on Western allies. When the tape was transcribed, every page was stamped "SECRET" in red, top and bottom. Then, with a red felt-tip pen, "Top" was methodically written in front of each "SECRET."

Our discussion was considered so dangerous that we – two reporters and one social worker – were placed in the top floor of the prison maximum security wing, which guards told us had formerly held terrorists, serial murderers, gang leaders and child rapists.

Meanwhile, police stripped my home of every file, every piece of paper I had, and 400 books.

Our case became known as "ABC", named after our surnames Aubrey, Berry and Campbell. We hoped it would end quickly. We knew that the senior minister responsible, Home Secretary Merlyn Rees, had announced three months before that the “mere receipt of unauthorized information should no longer be an offense.”

The day after we were arrested, I was told he was furious to be woken with news that the security agencies had delivered a fait accompli.

Historian Richard Aldrich said the head of MI5 – Britain's Security Service – considered me the person of the greatest interest to see incarcerated.

In 40 years of reporting on mass surveillance, I have been raided three times, jailed once, had television programmes I made or assisted making banned from airing under government pressure five times, seen tapes seized, faced being shoved out of a helicopter, had my phone tapped for at least a decade, and — following my 1977 arrest — faced 30 years' imprisonment for alleged violations of secrecy laws.

And why do I keep going? Because from the beginning, my investigations revealed a once-unimaginable scope of governmental surveillance, collusion and concealment by the British and US governments – and practices that were always as much about domestic spying during times of peace as they were about keeping citizens safe from supposed foreign enemies, thus giving the British government the potential power to become, as our source that night had put it, a virtual “police state.”

Cuffed and booked ... The charges against Aubrey, Berry

and Campbell in the "ABC" trial (click to enlarge)

A decade later, in a parliamentary debate, Foreign Secretary David Owen revealed that he was initially against our being prosecuted, but was convinced to go along after being promised that we journalists could be jailed in secret:

Everybody came in and persuaded me that it would be terrible not to prosecute ... I eventually relented. But one of my reasons for doing so was that I was given an absolute promise that the case would be heard in camera [a secret hearing].

In the face of this security onslaught, the politicians collapsed and agreed we should all be charged with espionage – although there was no suggestion that we wanted to do anything other than write articles. I was alleged to be "a thoroughly subversive man who was quite prepared to publish information which was secret,” my lawyer later wrote in his memoir.

But my lawyer saw it differently himself.

"Campbell is a journalist ... a ferret, not a skunk," he told the magistrates' court in Tottenham, North London. But when he inquired about the possibility of a misdemeanor plea and a £50 ($75) fine, he was cut down: "That course might be acceptable for Berry and Aubrey. But the security services want Campbell in prison for a very long time."

They meant it. In March 1977, one month after our nighttime arrest, we were all charged with breaking Britain's Official Secrets Act, for the "unlawful receipt of information.” Then we were charged with espionage. Each espionage charge carried a maximum of 14 years. I was also charged with espionage for collecting open source information on UK government plans. In total, I faced 30 years.

The interview, and then our arrests, were a first encounter with the power of Government Communications Headquarters – better known by its acronym GCHQ – Britain's electronic surveillance agency.