British prosecutors failed to disclose crucial evidence to the courts in the aftermath of the September 11 terrorist attacks in a case that resulted in an innocent pilot being jailed for five months, previously unseen documents reveal.

Lotfi Raissi, an Algerian living in the UK, was the first person in the world to be arrested after the 2001 attacks in New York and Washington DC. Accused of being the "lead" instructor of the 9/11 hijackers, Raissi, 27, was held in Belmarsh high security prison awaiting extradition to the United States.

In a landmark announcement, Jack Straw, the justice secretary, is shortly expected to reveal whether the UK government will accept responsibility for the miscarriage of justice and pay Raissi compensation.

The Guardian has obtained classified documents produced by the FBI and anti-terrorist officials in the UK after the 9/11 attacks which shed new light on how the courts were misled. They include:

• A report by the Crown Prosecution Service (CPS) into the way its staff handled the case, revealing prosecutors made unfounded allegations about Raissi's involvement in 9/11 on the basis of an oral briefing from two FBI agents outside court.

• A confidential letter from Scotland Yard's anti-terrorist branch to the CPS two months before Raissi was released, back-tracking on the key allegation that was being used in court to link Raissi to a senior al-Qaida suspect linked to Osama bin Laden.

• Memorandums from the FBI to anti-terrorist officials in the UK, revealing 9/11 investigators never wanted Raissi to be arrested and were informed about the unreliability of the evidence against him months before the courts were told.

Ministers were forced to consider Raissi's claim for damages after a ruling by the court of appeal last year that found there was evidence that Scotland Yard and the CPS had circumvented "the rule of English law" in what judges believed would amount to a serious abuse of process.

Now 35, Raissi still lives in the UK but says he has been unable to rebuild his life. He has been forced to abandon his promising career as a commercial pilot.

The FBI became interested in Raissi days after the attacks because he trained at the same Arizona flight school as Hani Hanjour, the hijacker who piloted the plane that crashed into the Pentagon.

Despite a specific plea from the FBI not to arrest Raissi but to gather information about him discreetly, anti-terrorist officers from the Metropolitan police stormed his house in Berkshire on September 21 on suspicion of the terrorist attacks 10 days earlier.

Rather than release Raissi when it emerged there was insufficient evidence to charge him, law enforcement officials in the UK colluded with the FBI to obtain a warrant for his extradition. There was no evidence to justify a warrant for terrorism, so Raissi was requested on charges relating to an allegation that he failed to disclose his knee surgery in a pilot application.

In court, the CPS said the pilot application allegations were mere "holding charges", and said he was in fact wanted for his alleged role in a conspiracy to commit mass murder during the 9/11 attacks.

However, as their case for keeping Raissi in Belmarsh began to unravel, prosecutors introduced a new piece of evidence. They relied in successive hearings on an address book which they claimed belonged to Abu Doha, an Algerian terror suspect said to have had personal contact with Bin Laden in Afghanistan.

The address book contained a number linked to an apartment used by Raissi in Arizona, and supposedly connected him to a global terrorist conspiracy. However, two months into his incarceration at Belmarsh, anti-terrorist officers informed the CPS that they no longer believed the address book belonged to Doha, and said it was more likely to be the property of a man called Adam Kermani, who lived in Islington, north London.

Kermani, an ex-boxer, was of so little concern to police that he had never been arrested or interviewed. Kermani's name and Home Office number were written on the front of the address book, which was found in a locked briefcase at his house.

Judges were not informed of this development until February 2002, after which Raissi was released.

The FBI however had been fully briefed months earlier, writing to Scotland Yard to confirm the owner of the address book was "not Abu Doha as originally thought".

His lawyer, Jules Carey, said Raissi's ordeal was one of the most significant miscarriages of justice during so-called war on terror.

"The court of appeal pulled no punches in asserting that there was a considerable body of evidence to suggest that the Met and CPS were responsible for serious defaults that resulted in Raissi's detention at Belmarsh," he said.

"These documents demonstrate, unequivocally, that the blame lies with the British authorities. He has waited seven years for an apology and watched four home secretaries come and go without receiving it. He is hopeful that his wait is finally up."