At the height of the controversy about Jeremy Thorpe and the conspiracy of silence that grew around his apparent involvement in a plot to kill his former lover Norman Scott, we began making a documentary film about the story for BBC television.

Little did I realise I, too, would soon become an unwitting figure in that conspiracy.

Our team had pitched the idea, in 1979, to an editor who later became the Corporation’s powerful Head of News. He agreed to our investigation but demanded to know absolutely everything we were doing. ‘Keep me in the picture, boys,’ he said, amicably, time and again.

Many years later, at his funeral, it emerged that this man had been a colonel in British Military Intelligence in the Territorial Army.

With the benefit of hindsight, it now becomes more than a mere suspicion that we investigative journalists at the cutting edge of the story were being gently manipulated by others in powerful positions — people who were anxious to know how much information there was about the developing scandal, how accessible it was, and what could be done to shut it down.

After all, how much easier to use unwitting real reporters than get spooks to pose as reporters to dig for information.

At the time, former Liberal party leader Jeremy Thorpe was a well-connected and charismatic politician from a social class that dominated the Whitehall establishment and Civil Service in those days back in the Seventies.

I believe I was the first journalist to learn Thorpe may have been involved in a conspiracy to silence one of his former gay lovers, Norman Scott, a stable boy from the West Country, who was needy, self-centred and dangerous.

When Thorpe ended their affair — which had begun many years before he became the Liberal leader — Scott began to blab about it to friends, other politicians and eventually the police. Initially, the allegations resulted merely in an internal Liberal party inquiry, which exonerated Thorpe — although he had been forced to deny that he’d had a sexual relationship with Scott.

I am sure the Special Branch, Devon police and Scotland Yard were well aware of the truth. However, as part of a conspiracy to protect the MP, they instinctively knew — without having to meet in secret in a smoke-filled room in the Reform club — how to cover up such a scandal.

F or the very last thing that Britain’s governing elite needed at the time was a high-profile homosexual scandal. Indeed, if it had been made public, it could have done irreparable damage to Britain’s relationship with our closest ally, America, whose government had just begun sharing nuclear secrets with us.

There had already been the huge embarrassment of Kim Philby, Guy Burgess, Donald Maclean (British members of a KGB spy ring that had penetrated MI6 and then passed vital information to the Soviets).

the Special Branch, Devon police and Scotland Yard were well aware of the truth behind the conspiracy theory that Jeremy Thorpe (pictured) was trying to silence one of his former gay lovers

There had also been the case of John Vassall, a gay Admiralty official who had been compromised by Soviet KGB officers, and the scandal of War Minister John Profumo, who had bedded Christine Keeler while she was also sleeping with a Soviet naval attache. And those are only the cases the public knew about!

The Americans were already nervous that Britain had been so easily infiltrated by the KGB, and would have become still more alarmed to learn that yet another high-profile political figure had a secret life that left him vulnerable to potential blackmail and manipulation.

But if political land mines lay all over Whitehall for my investigative TV team to avoid, there were even more lying hidden in the corridors of the BBC — the beyond reproach ‘queen of broadcasting’ and ‘fountain of truth’. So what if the Daily Express or Daily Mirror had reported the Thorpe scandal, who would care? Such newspapers were considered to be ‘gutter press’ and with no influence.

But if the BBC, with its power and influence, ran a one-hour documentary ... to the Establishment, that would have been unacceptable.

I had been on the story for less than two weeks when I got a phone call from Jo Grimond, one of Thorpe’s predecessors as Liberal leader. ‘What you are doing is outrageous!’ he barked down the line. ‘Unless you stop at once, I’ll have you dismissed within hours by the [BBC’s] Director General, who I happen to know extremely well.’

We then received a torrent of threatening calls from another Liberal MP called Cyril Smith — who, ironically, since his death has been exposed as a predatory paedophile and a serial abuser of young boys, whose activities were covered up years later by the Establishment.

Nevertheless, we remained ‘protected’ from their wrath by an invisible shield within the BBC. I now think I know why.

When Thorpe’s ex-lover Norman Scott (pictured) made a formal criminal complaint about the Liberal MP’s relationship with him, the leading officer didn’t bother to open an investigation

I can’t prove it, but after a lifetime with the BBC — an organisation for which I have undying affection and whose initials remain stamped through my backbone like Blackpool rock — and after 40 years of making numerous films about intelligence and writing a book about the CIA, I do now, sadly, believe I and my team were being carefully manipulated by the state broadcaster.

Early on in our investigation, I flew to California, where I interviewed Thorpe’s closest chum, the Liberal MP Peter Bessell. I cassette-taped three hours of his full, astonishing story.

When I returned to the BBC HQ in London’s Lime Grove, a kindly executive told me I looked jet-lagged and took the cassette from me — ‘for safe keeping,’ he said. ‘I’ll keep it in the office safe.’

Funnily enough, however many times I requested or even demanded the tape’s return, I could never lay hands on it. We ploughed on regardless.

Slowly, we discovered the extent of the Whitehall conspiracy to cover up Thorpe’s behaviour. We learnt that the FBI had warned their Special Branch counterparts in Britain about Thorpe’s predilection for rent boys in Times Square during visits to New York.

As a result, Devon police were asked to make discreet inquiries — and the then Chief Constable, Sir Ranulph Bacon, made damn sure the results remained secret.

As the police/Special Branch file on Thorpe grew, it ‘disappeared’ from the registry in Scotland Yard and was placed where no one could see it — in the safe of the Assistant Commissioner (Crime).

I am indebted to the veteran intelligence historian Nigel West for reminding me there were ‘plenty of very sensitive’ Metropolitan Police Special Branch files that were kept by the Assistant Commissioner. He says: ‘It is no different at MI5, where some files would be retained by the appropriate section or in the MI5 director-general’s personal safe.’ West points out that MI5 was under no obligation to share intelligence about political figures with government ministers. Nor would any information obtained centrally by the police have been passed back to the local forces.

J oin all the dots and it is not surprising that when Thorpe’s ex-lover, Norman Scott, went to Chelsea police station to make a formal criminal complaint about the Liberal MP’s relationship with him, the leading officer, Bob Huntley (later to become head of the Yard’s bomb squad), didn’t bother to open an investigation.

‘Who would believe the word of a queer stable boy against that of Jeremy Thorpe?’ Huntley said when I interviewed him years later. There has never been any explanation of why that criminal complaint was not taken any further in terms of a prosecution. But you can bet a crate of Bollinger champagne that someone made sure Scott’s allegations went straight up to Special Branch.

Even so, Scott’s allegations were causing Thorpe sufficient damage that it was alleged the MP told a colleague: ‘We have to get rid of him.’ In due course, as the Old Bailey trial of Thorpe heard, a plot was devised to hire a hitman using Liberal party funds to solve the problem of Norman Scott.

The contract to murder him, for a fee worth £140,000 in today’s money, went first to a small-time South London villain called Dennis Meighan. However, although he dropped out at the last minute, he remained a key witness and would have been a crucial prosecution witness at any trial of Thorpe.

Tom Mangold had been on the story for less than two weeks when he received a phone call from Jo Grimond, one of Thorpe’s predecessors (right), who claimed he would have the journalist dismissed by the BBC Director General, and another from Cyril Smith (left), who has since been exposed as a paedophile

Quite correctly, Scotland Yard detectives got a damning — and true — statement from him.

But a few weeks later, Meighan took a phone call from an unidentified man who told him to go to Brentford Police Station where he was told he was ‘expected’. When referring to the local police, the caller used the expression ‘wooden-tops’, which was then commonly used by Special Branch in a sneering reference to their ‘lesser mortals’ in uniform.

Meighan went to Brentford, was ushered into an interrogation room, and handed an envelope which contained a new and totally different version of his statement about the plot to kill Norman Scott.

It exonerated Thorpe and the Liberals from any plot and also exonerated Meighan for conspiracy to kill, and possession of an illegally loaded Mauser. Meighan, who himself faced criminal charges, couldn’t believe his luck. He happily signed the false statement — and was never called as a witness to the subsequent Thorpe trial.

So what was behind one of the great cover ups of the 20th century? Jeremy Thorpe’s innocence was regarded as crucial to the national interest. Whitehall did not want another homosexual scandal, conscious of the damage it could pose to Britain’s relationship with the U.S., and also the deleterious effect it would have on the balance between rulers and ruled. But in a democracy, there is only so much a conspiracy of this enormity to protect such a high-profile politician from prosecution could achieve.

Eventually, such was the pressure from the Press, that the Director of Public Prosecutions was finally obliged to bring charges against Thorpe for his alleged role in the plot to murder his former male lover.

I am the opposite of a conspiracy theorist, but at this point I do get conspiratorial.

Sir Ian Trethowan was in charge of the BBC at the time and have a very close relationship with the Security Services

Particularly, there was the profoundly questionable choice of judge to hear the case at the Old Bailey. Not all judges are allocated trials on a simple roster basis. Sometimes, the Lord Chancellor (as the post was back then), who was head of the judicial system and a very powerful political appointment, had an oh-so discreet hand in these matters.

Sometimes, the ‘right’ judge was hand-picked for a sensitive trial … the kind of judge who shared without question the Establishment consensus. The result? The judge in question Sir Joseph Cantley’s summing up in the Thorpe case was a judicial farce. He virtually ordered the jury to find Thorpe not guilty.

A similar thing had happened a decade earlier in the trial which I covered of the society osteopath Stephen Ward, who had introduced Christine Keeler to Tory war minister John Profumo and was subsequently prosecuted for living on the earnings of prostitutes. In this case, though, the jury was encouraged to convict.

By now we had finished our Thorpe documentary — having uncovered overwhelming evidence of just how the Establishment had closed ranks. It was a remarkable snapshot of the period. But Thorpe was found innocent. Not surprisingly, on the night of the end of his trial, the BBC management, rightly, felt that it could not transmit our film, which was based on an assumption of Thorpe’s guilt.

But what happened next was odd.

The man who had commissioned the film sent a despatch rider to my home with a letter ordering me to destroy every existing tape of the film. If I didn’t do so, I would be fired.

And who was the BBC’s director general and chief executive in charge of news and current affairs at the time? None other than Sir Ian Trethowan, who had a very close, and editorially unhealthy, relationship with the Security Services. Three years later, he directly interfered with an investigative — and not always friendly — documentary I was making about MI5, ordering me to give my script to MI5 so they could vet it before transmission.

‘And you are to tell no one of this conversation,’ he instructed as I stood to attention in his office. The MI5 documentary was broadcast, but only after more Trethowan interference.

As BBC director-general at the time, it was in Sir Ian’s power to stop us making our Jeremy Thorpe film. But oddly he never used that power.

Is this because the information we turned up was helping to fatten the files of MI5? Were we unwittingly keeping MI5 in the picture? Ever since, I’ve never stopped wondering. But, deep down, I think I know the truth.

Compared with the Seventies, the BBC has matured and Britain today is a freer, more open, less class-ridden and secret society. Yet there are still justified concerns that when it can get away with it, the Establishment will cover up for its own.

Take the slow boiling scandal involving paedophilia rings and Westminster notables, and, this week, the row over the cover up of the CIA’s torture interrogation techniques — some of which were learned from Britain’s brutal behaviour in Northern Ireland — which were well known to Westminster governments of both stripes at the time.

Which is why I would argue that investigative journalism, for all its occasional dreadful mistakes, bad apples and cock-ups, is still a crucial part of the DNA of our democratic genes in Britain.

When establishments seek to control us, we should recognise that this is the thin end of the wedge. It is important to note that the Leveson Inquiry — set up to look into the ridiculous follies of phone-hackers (they even bugged my phone to find out what Panorama was up to, for Heaven’s sake!) — have led to serious attempts to constrain the Press and introduce an element of political involvement in our editorial freedom.

But this will merely take us back to where we were 40 years ago. I strongly believe there is one thing that we must remember above all. When any citizen wants help to expose corruption, hypocrisy, wrongdoing or corrosive inefficiency, there must be a newspaper, magazine or broadcast office which he or she can contact for help.

The British Establishment’s protective walls of secrecy of the Seventies will never be completely breached, but we can try to remain truly independent, vigilant and deserving tribunes of the people. That is our democratic duty and happens to be what we do best.