The deputy prime minister says the government will co-operate with a Scotland Yard investigation into secret MI6 rendition operations Press Association

Scotland Yard has opened a criminal investigation into secret MI6 rendition operations that resulted in leading Libyan dissidents being abducted and flown to Tripoli where they were subsequently tortured in Muammar Gaddafi's prisons.

The announcement came as police and the director of public prosecutions, Keir Starmer, said there was insufficient evidence to prosecute any individual MI5 or MI6 officers following lengthy investigations into allegations of British complicity in the torture of terrorism suspects in Pakistan and Afghanistan.

The new investigation is to focus on Abdul Hakim Belhaj and Sami al-Saadi, who lodged complaints with the police last November after the chance discovery of a cache of classified documents in an abandoned Libyan government office laid bare the role that MI6 played in their rendition.

Saadi was detained in Hong Kong in 2004 and then forced on to a plane to Tripoli with his wife and four children in an operation that MI6 mounted in co-operation with Gaddafi's intelligence chief, Moussa Koussa. Saadi says he suffered years of torture.

Belhaj was detained in Bangkok along with his pregnant wife after an MI6 tip-off, and allegedly tortured by American agents for several days before being flown to Tripoli where he says he was tortured and detained for several years. His wife, who was detained for several months, has not spoken publicly about the manner in which she was treated.

British officials have not sought to deny the involvement of MI6 in either rendition. Instead, they have stressed that each resulted from what they describe as "ministerially authorised government policy", raising the possibility that the new Yard inquiry will require the questioning of ministers of the last Labour government.

In addition, the police and the Crown Prosecution Service announced that they were establishing a joint panel that would examine other allegations of UK complicity in torture and rendition levelled by a number of former Guantánamo inmates and others detained in the so-called war on terror. These complainants include Shaker Aamer, the last British resident still held at Guantánamo.

The panel will decide whether the allegations should be examined first by the official inquiry that was established by David Cameron 18 months ago, and which is waiting to start hearing evidence, or whether police should investigate immediately.

A statement issued on Thursday by Starmer and Lynne Owens, an assistant commissioner of the Metropolitan police, said the panel would consider "whether there is any significant risk that any available evidence would not be available or would be weakened" if an investigation did not take place immediately, or whether "the allegation in question is so serious that it is in the public interest to investigate it now".

This panel advised on the Libyan cases, and the police decided that "the allegations raised in the two specific cases concerning the alleged rendition of named individuals to Libya and the alleged ill-treatment of them in Libya are so serious that it is in the public interest for them to be investigated now rather than at the conclusion of the detainee inquiry".

Scotland Yard detectives have spent 30 months investigating allegations that the UK's intelligence agencies had become so close to the torture inflicted by overseas governments that their officers had committed serious criminal offences.

One inquiry, codenamed Operation Hinton, focused on the events surrounding MI5's interrogation of Binyam Mohamed in May 2002, several weeks after he had been detained in Pakistan, and later events in Morocco.

Proceedings brought on Mohamed's behalf showed that MI5 knew he was being mistreated before an officer was sent to Karachi to question him.

The CPS decided more than a year ago that that officer – identified only as Witness B – should not face charges, but Operation Hinton continued while detectives pursued what Starmer described as a "wider investigation into other potential criminal conduct". This involved attempting to trace responsibility for Witness B's actions up MI5's chain of command and beyond.

The investigation showed that "members of the Security Service provided information to the US authorities about Mr Mohamed and supplied questions for the US authorities to put to Mr Mohamed while he was being detained between 2002 and 2004", the statement said. However, the CPS has also concluded that there was insufficient evidence to prosecute any individual on the basis that they "knew or ought to have known that there was a real or serious risk that Mr Mohamed would be exposed to ill treatment amounting to torture".

The statement added: "Nothing in this decision should be read as concluding that the ill-treatment alleged by Mr Mohamed did not take place or that it was lawful."

A second investigation, codenamed Operation Iden, concentrated on events in 2002 at Bagram airfield north of Kabul, where the US military had established a prison, and where both MI5 and MI6 officers interrogated a number of suspects after being given written instruction from London that "the law does not require you to intervene to prevent" the mistreatment they were witnessing.

That investigation was triggered after MI6 referred one of its own officers to the attorney general in September 2009.

"The offences considered were aiding and abetting torture, aiding and abetting war crimes, false imprisonment, aiding and abetting assault, and misconduct in public office," the statement said.

That investigation foundered on unsuccessful attempts to take a statement from a particular individual who was said to have been mistreated in the presence of an MI6 interrogator. It is thought that this was because it was not possible to be certain about the identity of the individual.

In addition, US officials who were thought to have been present refused to be interviewed by police.

"On the account that has been given by the member of the Secret Intelligence Services and taking into account all other available evidence, there is insufficient evidence to provide a realistic prospect of convicting him of any criminal offence," the statement concluded.

It is unclear when the official inquiry, chaired by Sir Peter Gibson, a former appeal court judge, will begin hearing evidence. In a statement on Thursday, the inquiry panel said: "The detainee inquiry panel will now carefully consider its next steps and Sir Peter Gibson will make an announcement in due course."

Most major human rights groups are boycotting the inquiry, claiming that it will be too secretive and is insufficiently independent of government, but the Foreign Office is mounting a renewed effort to persuade them back on board.

The head of MI6, Sir John Sawers, said he welcomed the decision not to charge any of his officers at the conclusion of the twin investigations and said that the "courageous individual" at the centre of Operation Iden would now be able to continue his work in support of national security.

He added that MI6 would co-operate with the Libyan investigations. "It is in the service's interest to deal with the allegations being made as swiftly as possible so we can draw a line under them and focus on the crucial work we now face in the future."

Detectives are thought to have already begun examining the documentation that was uncovered in Tripoli last month by an investigator with Human Rights Watch, the New York-based NGO.

The documents will also form the basis of civil claims that Saadi and Belhaj and their families are bringing against the British government.

It is not yet clear which ministers may have authorised the secret Libyan rendition operations in the way that well-placed Whitehall sources have asserted.

After the documents were discovered, Tony Blair, who was prime minister at the time, insisted he knew nothing about them. Similarly, Jack Straw, who was then foreign secretary, said in a radio interview: "The position of successive foreign secretaries, including me, is that we were opposed to unlawful rendition, opposed to torture or similar methods and not only did we not agree with it, we were not complicit in it, nor did we turn a blind eye to it." He added: "No foreign secretary can know all the details of what … intelligence services are doing at any one time."

Shortly after Blair and Straw issued their denials, Sir Richard Dearlove, who was head of MI6 at the time, said: "It was a political decision, having very significantly disarmed Libya, for the government to co-operate with Libya on Islamist terrorism. The whole relationship was one of serious calculation about where the overall balance of our national interests stood."

The year after the joint UK-Libyan operations were mounted, Straw told MPs they must disbelieve allegations of UK involvement in rendition "unless we all start to believe in conspiracy theories and that the officials are lying, that I am lying, that behind this there is some kind of secret state which is in league with some dark forces in the United States".

Asked following Dearlove's statement whether he still maintained that he was unaware of the Libyan rendition operations, and whether he knew which ministers Dearlove could be referring to, Straw said he had no further comment to make.

Blair also declined to make any further comment.