Britain’s intelligence agencies engaged in a series of previously unknown joint operations with Colonel Muammar Gaddafi’s government and used the information extracted from rendition victims as evidence during partially secret court proceedings in London, according to an analysis of official documents recovered in Tripoli since the Libyan revolution.

The exhaustive study of the papers from the Libyan government archives shows the links between MI5, MI6 and Gaddafi’s security agencies were far more extensive than previously thought and involved a number of joint operations in which Libyan dissidents were unlawfully detained and allegedly tortured.

At one point, Libyan intelligence agents were invited to operate on British soil, where they worked alongside MI5 and allegedly intimidated a number of Gaddafi opponents who had been granted asylum in the UK.

Previously, MI6 was known to have assisted the dictatorship with the kidnap of two Libyan opposition leaders, who were flown to Tripoli along with their families – including a six-year-old girl and a pregnant woman – in 2004.

However, the research suggests that the fruits of a series of joint clandestine operations also underpinned a significant number of court hearings in London between 2002 and 2007, during which the last Labour government unsuccessfully sought to deport Gaddafi’s opponents on the basis of information extracted from people who had been “rendered” to his jails.

In addition, the documents show that four men were subjected to control orders in the UK – a form of curfew – on the basis of information extracted from victims of rendition who had been handed over to the Gaddafi regime.

The papers recovered from the dictatorship’s archives include secret correspondence from MI6, MI5 reports on Libyans living in the UK, a British intelligence assessment marked “UK/Libya Eyes Only – Secret” and official Libyan minutes of meetings between the two countries’ intelligence agencies.

They show that:

• UK intelligence agencies sent more than 1,600 questions to be put to the two opposition leaders, Sami al-Saadi and Abdul Hakim Belhaj, despite having reason to suspect they were being tortured.

• British government lawyers allegedly drew upon the answers to those questions when seeking the deportation of Libyans living in the UK

• Five men were subjected to control orders in the UK, allegedly on the basis of information extracted from two rendition victims.

• Gaddafi’s agents recorded MI5 as warning in September 2006 that the two countries’ agencies should take steps to ensure that their joint operations would never be “discovered by lawyers or human rights organisations and the media”.

In fact, papers that detail the joint UK-Libyan rendition operations were discovered by the New York-based NGO Human Rights Watch in September 2011, at the height of the Libyan revolution, in an abandoned government office building in Tripoli.

Since then, hundreds more documents have been discovered in government files in Tripoli. A team of London-based lawyers has assembled them into an archive that is forming the basis of a claim for damages on behalf of 12 men who were allegedly kidnapped, tortured, subject to control orders or tricked into travelling to Libya where they were detained and mistreated.

An attempt by government lawyers to have that claim struck out was rejected by the high court in London on Thursday , with the judge, Mr Justice Irwin, ruling that the allegations “are of real potential public concern” and should be heard and dealt with by the courts.

The litigation follows earlier proceedings brought on behalf of the two families who were kidnapped in the far east and flown to Tripoli. One claim was settled when the government paid £2.23m in compensation to al-Saadi and his family; the second is ongoing, despite attempts by government lawyers to have it thrown out of court, with Belhaj suing not only the British government, but also Sir Mark Allen, former head of counter-terrorism at MI6, and Jack Straw, who was foreign secretary at the time of his kidnap.

Facebook Twitter Pinterest Abdel Hakim Belhaj is suing the British government.

Belhaj has offered to settle for just £3, providing he and his wife also receive an unreserved apology. This is highly unlikely to happen, however, as the two rendition operations are also the subject of a three-year Scotland Yard investigation code-named Operation Lydd. Straw has been questioned by detectives: his spokesman says he was interviewed “as a witness”.

Last month, detectives passed a final file to the Crown Prosecution Service. No charges are imminent, however. The CPS said: “The police investigation has lasted almost three years and has produced a large amount of material. These are complex allegations that will require careful consideration, but we will aim to complete our decision-making as soon as is practicably possible.”

The volte-face in UK-Libyan relations was always going to be contentious: the Gaddafi regime had not only helped to arm the IRA, bombed Pan Am Flight 103 over the Scottish town of Lockerbie with the loss of 270 lives in 1988, and harboured the man who murdered a London policewoman, Yvonne Fletcher, four years earlier; it had been responsible for the bombing of a French airliner and a Berlin nightclub, and for several decades had been sending assassins around the world to murder its opponents.

The Tripoli archives show that the rapprochement, which began with the restoration of diplomatic ties in 1999, gathered pace within weeks of the al-Qaida attacks of 9/11. Sir Richard Dearlove, who was head of MI6 at the time, has said that these links were always authorised by government ministers.

The week after the attacks, British intelligence officers met with Moussa Koussa, the head of Libyan intelligence, who offered to provide intelligence from Islamists held in the regime’s jails.

Two months later, British intelligence officers held a three-day conference with their Libyan counterparts at a hotel at a European airport. German and Austrian intelligence officers also attended.

According to the Libyan minutes, the British explained that they could not arrest anyone in the UK – only the police could do that – and that there could be difficulty in obtaining authorisation for Gaddafi’s intelligence officers to operate in the UK. They also added that impending changes to UK law would give them “more leeway” in the near future.

Other documents released under the Freedom of Information Act detail the way in which diplomatic contacts between London and Tripoli developed, with a British trade minister, Mike O’Brien, visiting Tripoli in August 2002, the same month that the dictator’s son, Saif, was admitted as a post-graduate student at the London School of Economics. Blair and Gaddafi spoke by telephone for the first time, chatting for 30 minutes, and in December 2003 the dictator announced publicly that he was abandoning his programme for the development of weapons of mass destruction.

With the war in Iraq going badly, London and Washington were able to suggest that an invasion that had been justified by a need to dismantle a WMD programme that was subsequently found not to exist had at least resulted in another country’s weapons programme being dismantled.

Three months later, in March 2004, the new relationship was sealed by a meeting between Gaddafi and Blair, during which the British prime minister announced that the two countries had found common cause in the fight against terrorism, and the Anglo-Dutch oil giant Shell announced that it had signed a £110m deal for gas exploration rights off the Libyan coast.

However, the Tripoli archive shows that beneath the surface of the new alliance, the Blair government was encouraging ever-closer co-operation between the UK’s intelligence agencies and the intelligence agencies of a dictatorship which had been widely condemned for committing the most serious human rights abuses; MI5 and MI6, and the CIA, would begin to work hand-in-glove with the Libyan External Security Organisation.

Eliza Manningham-Buller, who was head of MI5 during most of the period that the UK’s intelligence agencies were working closely with the Libyan dictatorship, has defended the decision to open talks with Gaddafi on the grounds that it helped to deter him from pursuing his WMD programme. However, when delivering the 2011 Reith Lecture, she added: “There are questions to be answered about the various relationships that developed afterwards and whether the UK supped with a sufficiently long spoon.”

The archive clearly shows that Gaddafi hoped that this intelligence co-operation would result in British assistance in his attempts to round up and imprison Libyans who were living in exile in the UK, Saudi Arabia, Pakistan and Mali. All of these men were members of the Libyan Islamic Fighting Group (LIFG), an Islamist organisation that had attempted to assassinate him three times since its foundation in the early 90s. A largely spent force since the late 90s, many of the members of the LIFG had been living peacefully in the UK for more than a decade, having arrived as refugees. Some had been granted British citizenship. Koussa’s agency asked British intelligence to investigate 79 of these men, whom they described as “Libyan heretics”.

Two weeks before Blair’s visit to Libya, Belhaj and his four-and-a-half-months pregnant wife, Fatima Bouchar, were kidnapped in Thailand and flown to Tripoli. Bouchar says she was taped, head to foot, to a stretcher, for the 17-hour flight.

In a follow-up letter to Koussa, Allen claimed credit for the rendition of Belhaj – referring to him as Abu Abd Allah Sadiq, the name by which he is better known in the jihadi world – saying that although “I did not pay for the air cargo”, the intelligence that led to the couple’s capture was British.

Three days after Blair’s visit, al-Saadi was rendered from Hong Kong to Tripoli, along with his wife and four children, the youngest a girl aged six.

Facebook Twitter Pinterest Libya’s foreign minister Moussa Koussa was head of Libyan intelligence.

Both men say that while being held at Tajoura prison outside Tripoli they were beaten, whipped, subjected to electric shocks, deprived of sleep and threatened.

Belhaj says he was twice interrogated at Tajoura by British intelligence officers. After gesturing that the session was being recorded, Belhaj says he made a number of gestures to show that he was being beaten and suspended by his arms. One of the British officers, a man, is said to have given a thumbs-up signal, while the second, a woman, is said to have nodded.

Belhaj alleges that following one of these encounters he agreed to sign a statement about his associates in the UK after being threatened with a form of torture called the Honda, which involved being locked in a box-like structure whose ceiling and walls could be shrunk, provoking extreme claustrophobia and fear as well as discomfort.

According to the claim being brought against the British government, the attempt to track down other leading members of the LIFG resulted in the intelligence agencies of Libya and the UK throwing their net still wider.

In late 2005, a British citizen of Somali origin and a Libyan living in Ireland were arrested in Saudi Arabia and allegedly tortured while being questioned by Saudi intelligence officers about associates who were members of the LIFG. The men say they were shackled and beaten. The British citizen says he was also interrogated by two British men who declined to identify themselves and who appeared uninterested in his complaints of mistreatment.

Many of the questions put to the two men concerned the whereabouts of Othman Saleh Khalifa, a long-standing member of the LIFG. Khalifa was detained in Mali a few months later and rendered to Libya. The Tripoli archive shows that summaries of his interrogations were sent to British intelligence, and that both MI5 and MI6 submitted questions that they wished to be put to him. A memorandum from MI6 to Koussa’s deputy, Sadegh Krema, was accompanied by questions “which you kindly agreed to pass to your interview team”.

Khalifa says that he was beaten during interrogations for around six months during the second half of 2006 and that he did not see daylight.

The Tripoli archive shows that during the same week that Khalifa was being rendered to Libya, MI5 and MI6 officers met Libyan intelligence officers in Tripoli and informed them that they were to be invited to the UK to conduct joint intelligence operations. The Libyan minutes of the meeting say that MI5 informed them that “London and Manchester are the two hottest spots” for LIFG activity in the country. The aim was to recruit informants within the Libyan community in the UK.

The Libyan minutes of the meeting also say that the British told them: “With your co-operation we should be able to target specific individuals.” The Libyans, meanwhile, said that potential recruits could be “intimidated” through threats to arrest relatives in Libya.

The following August, senior MI5 and MI6 officers and two Libyan intelligence officers met at MI5’s headquarters in London. According to the Libyan minutes, MI5 warned the Libyans that individuals could complain to the police if they believed they were being harassed by MI5, and could also expose the British-Libyan joint operations to the media.

The minutes also state that the British suggested that Libyan intelligence officers should approach potential recruits in the UK, and that if they refused to cooperate, arrangements could be made for the targets to be arrested under anti-terrorism legislation, accused of associating with those same Libyan intelligence officers, and threatened with deportation.

Facebook Twitter Pinterest Sami al-Saadi has been paid £2.23m in compensation.

One of the targets was a 32-year-old Libyan, associated with the LIFG, who had lived in the UK for 10 years and had been a British citizen for six years. The Libyan intelligence officers repeatedly telephoned him, claiming to be consular officials, and he eventually agreed to meet them at the Landmark hotel in Marylebone, London, on 2 September 2006. According to the Libyan notes of this meeting, the British insisted that two MI5 officers, one calling herself Caroline, should be present, so that the target should know that he was the subject of a joint UK-Libyan approach.

The target was told that he was to be given time to think about the approach. In Libya, meanwhile, the target’s brothers, sisters and mother say they were each detained in turn and told that they should persuade him to return to the country.

The Libyan intelligence officers also visited Manchester, calling at the home of another man targeted for recruitment. According to their notes, MI5 warned them not to enter the house but to persuade him to go with them to a public place where they could be photographed together. As he was not at home, the Libyan spies went instead to a mosque in the Didsbury district, where they told the imam that they were importing and exporting books.

On 5 September, shortly before the two Libyan intelligence officers returned home, they had another meeting with their British counterparts. Their notes show that the British warned that steps should be taken jointly to “avoid being trapped in any sort of legal problem [and] to avoid also that those joint plans be discovered by lawyers or human rights organisations and the media”. The Libyans assured MI5 and MI6: “We have effectively reassured them that we will stick by the joint plan to avoid any blame if the operation fails.”

The target says he was approached by “Caroline” and a second MI5 officer on a number of other occasions, but declined to travel to Libya and still lives in west London.

Six Libyan men, the widow of a seventh, and five British citizens of Libyan and Somali origin are bringing a number of claims, which include allegations of false imprisonment, blackmail, misfeasance in public office and conspiracy to assault.

The case is being brought against MI5 and MI6 as well as the Home Office and Foreign Office. Government departments declined to comment on the grounds that the litigation is ongoing.

When making their unsuccessful bid to have the case struck out, government lawyers admitted no liability. They argued that the five claimants who were subjected to control orders were properly considered to pose a threat to the UK’s national security, and denied that the government relied on information from prisoners held in Libya in making that assessment. They also argued that the LIFG had been a threat to the UK. They are expected to appeal Thursday’s high court decision.

Allen has declined to comment on the rendition operations, while Straw says: “At all times I was scrupulous in seeking to carry out my duties in accordance with the law, and I hope to be able to say more about this at an appropriate stage in the future.”