Pressure was mounting for tougher and more effective oversight of Britain's intelligence agencies as the disclosure of secret documents in Tripoli appeared to provoke panic and disarray across Whitehall.

With confidential papers from Libya raising important questions about the conduct of MI5 and MI6, particularly regarding the rendition of prisoners to countries where they faced torture, MPs said the system of scrutiny had to be changed.

A forthcoming inquiry into the UK's involvement in the torture and abuse of detainees, chaired by the retired judge Sir Peter Gibson, said it would investigate the latest allegations, which involve the capture and transfer to Tripoli in 2004 of two Libyan dissidents opposed to the Gaddafi regime. In a Commons statement on Monday, David Cameron welcomed the move to investigate the new "accusations of malpractice" and said ministers from the last government would have to answer for what happened.

Ed Miliband also urged the Gibson inquiry, which has been heavily criticised for lacking teeth, to "get to the bottom of the allegations... no part of the British state may ever be complicit in torture".

The unease in Whitehall over the revelations from Tripoli has been compounded by confusion over whether MI5 and MI6 had provided full disclosures about the operations to MPs, or to Gibson, who has spent a year gathering information from the agencies in preparation for his hearings.

Two principal allegations emerged from confidential papers left abandoned in the offices of former members of the Gaddafi regime. One involves a senior commander of the Libyan rebels, Abdul Hakim Belhaj, who has demanded an apology from the British and American governments for the way he was treated in 2004. Then a dissident living in Malaysia, he was flown back to Tripoli by the CIA, where he says he was subjected to torture over seven years.

In one document, a senior MI6 official, Sir Mark Allen, boasts of the role Britain played in the operation to have him taken to Libya. He also makes it clear the agency wanted information gleaned from him during questioning, almost certainly because of fears in British counter-terrorism circles that some dissident members of the Libyan Islamic Fighting Group had links to al-Qaida.

Other documents suggest that the UK worked with the Libyans to mount their own "rendition'' operation to remove from Hong Kong a man called Abu Munthir, despite the risk that he might be tortured when he returned. Munthir, who was also a senior figure in the LIFG, is thought to be in Tunisia.

Neither case has been made public before, but details should have been given to MPs on the parliamentary intelligence and security committee, which published a report on rendition in 2007 as part of its official oversight of the security agencies. In this report, and numerous others, the ISC has insisted it found no evidence of illegal activity by them. However, the Guardian has been told that the ISC did not know about either of the new cases at the time it wrote the 2007 report, and no evidence has been provided since then.

MPs on the committee say they have been told they cannot speak about this. Richard Ottoway, one of the nine members of the ISC at the time, told the Guardian he had been "advised it would be a criminal offence" if he discussed the matter.

Asked whether he feared the committee had been misled, he said: "Every member of the committee signed up to the Official Secrets Act, so I can't answer that."

The Gibson inquiry, set up last July, should also have been given details of the cases. Early on Monday, his staff indicated that they had not been informed about them. Two hours later they backtracked. A statement said: "As part of its preparation work the inquiry has already been made aware of and has received material relating to these issues and they will be considered by the inquiry."

Andrew Tyrie, the Conservative MP who chairs the all-party parliamentary group on extraordinary rendition, said that he was losing confidence in the ability of Gibson to get to the truth about such operations, echoing the concerns of human rights organisations who say they will boycott the hearings. "Sir Peter Gibson's early decisions – not to appoint an investigator, not to look at detainee transfer in theatre, not to sufficiently engage with the victims – do not inspire public confidence," Tyrie said.

He added that the new documents "highlight once again the shortcomings" of parliamentary scrutiny of the intelligence services, and that the ISC's powers needed to be strengthened following its "damaging" failure to establish the facts about rendition.

Ottoway also conceded that it was high time the ISC was given greater powers to oversee the work of the intelligence agencies. MPs have no way of knowing whether the agencies are giving them all the material they have.

"The committee only has the power to request information. At the moment, the heads of the agencies can decide whether to provide it. In future, decisions like that should be made by the secretary of state, not by the agencies themselves."

Sir Malcolm Rifkind, current chairman of the ISC, said: "The committee needs to be fully informed about the views of the security services on these very serious allegations. I expect to hear from them in the near future." He wanted "clarification" about "the nature and extent of intelligence-sharing with the Libyan security services and the rendition of Libyan nationals".

Whitehall officials defended the actions of the agencies, saying they had acted within the law and in accordance with government policy as laid down by ministers. They defended the policy of co-operating with Gaddafi's security apparatus, saying Britain had not been involved in an illegal rendition operation and had sought assurances from Libya that individuals sent to Tripoli would not be badly treated. Referring to the two cases identified in documents found in the Libyan capital, they said that individuals had been "deported" and sent to Libya by two sovereign governments.

UK security sources argue that Belhaj and Munthir were sent to Libya legally and that assurances were sought about their treatment. "We needed to be convinced", the source added, saying Britain otherwise could have been acting unlawfully. Belhaj has described being tortured by CIA officers who interrogated him in Bangkok before he was flown back to Libya. He has also described being tortured repeatedly after he returned to Tripoli. Sir Mark Allen declined to comment.