Scotland Yard is embroiled in a new corruption crisis after it emerged that senior officers knew for years that criminal private investigators had compromised its highly sensitive witness protection programme – and did nothing about it.

Days after the Metropolitan Police was rocked by incendiary claims that officers took part in a smear campaign against the family of the murdered black teenager Stephen Lawrence, The Independent can disclose that private investigators (PIs) were employed by organised crime gangs to try to intimidate witnesses who had agreed to give evidence in high-profile trials.

Scotland Yard uncovered the shocking intelligence up to 15 years ago but, incredibly, did next to nothing to stop the private detectives, who also worked for the News Of The World. A registered police informant codenamed “Michael Green”, who spent years undercover working with a corrupt firm of PIs, warned his handlers at the Met that his colleagues were trying to locate “supergrasses” under police protection and “actively worked on them to withdraw their damaging allegations”.

Download the new Independent Premium app Sharing the full story, not just the headlines

But, for reasons yet to become clear, the Met failed to charge or even arrest the investigators for intimidating key witnesses. One of the supergrasses who was approached while under police protection later withdrew all of his original testimony, resulting in the collapse of a major criminal trial.

The news comes days after a former Met officer, Peter Francis, claimed he was asked find “dirt” and spy on Stephen Lawrence’s relatives in a bid to undermine the campaign to bring his killers to justice. Details of Scotland Yard’s witness-protection programme being compromised were included in a Serious Organised Crime Agency (Soca) report which was suppressed by the agency but leaked to The Independent.

The same documents led to last week’s revelation by this newspaper that private detectives had been hired by major companies to hack, blag and steal personal information about rival companies and the public. The latest disclosures have heaped fresh pressure on the Met and Soca, who are thought to have withheld crucial details of the criminal world of private investigators from a parliamentary inquiry last year.Keith Vaz, the Labour MP who chairs the House of Commons Home Affairs Select Committee, said: “These claims are absolutely devastating. The committee has agreed to call Soca to give evidence next week and [the Met Commissioner] Sir Bernard Hogan-Howe will be giving evidence on 9 July. It seems they will have some very difficult to questions to answer. We were very clear in our report that the link between private investigators and the police needs to be broken, so they can be properly investigated without fear and favour. However, no action was taken and we are still awaiting a Government response. This industry needs to be brought out of the shadows and be properly regulated.

“I have today written to request all the information Soca has on private investigators and their links with the public and private sector. We need to be certain there are no more skeletons to come out of the closet.”

The eight-page Soca memo referred to intelligence that PIs were employed by the “criminal fraternity” to “frustrate law enforcement”. The Independent understands that the same corrupt investigators have also worked for the News of the World. The Soca report includes intelligence that crime bosses were hiring PIs to access “internal police databases, including those containing serving officers’ private details” and “deleting intelligence records from law enforcement databases”.

The most shocking practice, however, involves attempts to trace protected witnesses. Soca noted that PIs often had an “abundance of law-enforcement expertise either through corrupt contacts or from a previous career in law enforcement”, and they were “attempting to discover location of witnesses under police protection to intimidate them”.

The Independent has spoken to the registered police informant “Michael Green”, who did not wish to be named for fear of reprisals.

Daily coronavirus briefing No hype, just the advice and analysis you need Enter your email address Continue Continue Please enter an email address Email address is invalid Fill out this field Email address is invalid Email already exists. Log in to update your newsletter preferences Register with your social account or click here to log in I would like to receive morning headlines Monday - Friday plus breaking news alerts by email Update newsletter preferences

He infiltrated a team of private investigators who worked closely with a corrupt former Met police officer, who is well-known to Scotland Yard but has never faced any criminal charges. He cannot be named for legal reasons so the informant referred to him as “Mr Brown”.

He said the gang used to “boast” of locating supergrasses in the witness protection programme and “actively worked on them to withdraw their damaging allegations”.

Mr Green said: “Indeed on one occasion I managed to get possession of a CD ROM disc which was a recording of a person in the witness protection programme being interviewed by Mr Brown who had traced him. Basically, Mr Brown wanted him to retract his evidence.

“My handlers wanted to have the disc copied, but I knew I had to hand it back, and I was concerned that if it was copied it might be discovered. I asked them to check with their own experts to see if that was possible.

Mr Green said his handlers told him that forensic experts claimed it would not be traceable. He said: “I persisted and asked for a second opinion which I believe they obtained from the security services. Their experts stated that it might be proven that the disc had been copied. As a result a sound recording was taken and the disc was not used to burn another copy.”

Despite being handed this extraordinary intelligence, the Met took no action against the private investigators or Mr Brown. The supergrass, whose identity is known to The Independent, later dramatically changed his evidence and caused several convictions to be overturned, to the great embarrassment of the Met and the Crown Prosecution Service.

The Met declined to comment and referred enquiries to a SOCA spokesperson who said: “This report remains confidential and SOCA does not comment on leaked documents or specific criminal investigations.”

Disclosures over the Met’s inability to maintain the security of its witness protection programme have also raised fresh questions over the decision by the Leveson Inquiry to ignore the bombshell SOCA report.

The confidential document was offered to the public inquiry into the press and police by Ian Hurst, a former British Army intelligence officer whose computer was hacked by private investigators employed by the News of the World.

However, in an email to Mr Hurst’s lawyer, Kim Brudenell, the solicitor to the Inquiry said: “The Inquiry does not propose to go into further detail or take further evidence regarding these matters and so will not be pursuing the [informant], utilising Mr Hurst’s statement as evidence or calling him to provide further oral evidence.”

Lord Justice Leveson then embarked on a fortnight of hearings dominated by arguably far less evidence from union officials, civilian police workers and press officers from provincial police forces. Mr Hurst told The Independent: “Leveson obviously considered the media officer for Staffordshire police to be far more relevant to his Inquiry than the experiences of a man who had spent years infiltrating a criminal gang with direct knowledge of Metropolitan police and News of the World corruption.

“There is no more sensitive system within the police than the witness protection programme. I would have thought evidence of it being compromised almost at will by corrupt detectives, private investigators who work with newspapers and organised crime syndicates would have been relevant to culture, practice and ethics of the press and police.”

However, the Met’s former deputy assistant commissioner Brian Paddick did manage to publicly reveal that the witness protection programme had been compromised during his evidence to the Inquiry.

He told Lord Justice Leveson: “That is something I would expect the (Met) to take with the utmost seriousness. However, there is nothing in the documents disclosed to me to suggest that anything was done.”

Despite the acute sensitivities and the Met’s bizarre, decade-long inaction to tackle the corrupt private investigators, Lord Justice Leveson barely referred to the matter in his final report, published last November – for fear of compromising ongoing criminal investigations. He said: “Although I understand the concern, it would not be appropriate for me to go further.”

Tom Watson, the campaigning Labour MP, said: “It is absolutely shocking that the media-criminal nexus could have got anywhere near compromising the Met’s witness protection programme.

“These new revelations are the strongest argument I have heard for Lord Justice Leveson conducting Part Two of his Inquiry as soon as the criminal cases are over.”

A Leveson Inquiry spokesperson said: “The terms of reference for the Inquiry were absolutely about the culture, practices and ethics of the press and how they engaged with the public, the police and politicians. Evidence on other issues would have been considered to have been outside those terms of reference.”