A US court has narrowly ruled that Binyam Mohamed, the British resident secretly rendered to Morocco by the CIA before being held in the Guantánamo Bay prison camp for four years, cannot sue over his alleged torture in overseas prisons because it would compromise national security.

Mohamed was the lead plaintiff in a case brought by the American Civil Liberties Union (ACLU) on behalf of five former prisoners who claim they were tortured after being transferred to other countries through the CIA's extraordinary rendition programme.

They are fighting for the right to sue Jeppesen Dataplan, a Boeing subsidiary accused of arranging flights for the CIA.

A US court ruled last year that the case could proceed, but the Obama administration appealed and yesterday the court of appeals for the ninth circuit dismissed the case – although the judges were divided by six to five on the decision.

Judge Raymond Fisher said the majority had "reluctantly" concluded that "legitimate national security concerns" meant the case should not be heard.

Although the alleged offences were committed under the Bush government, the decision is a victory for the Obama's administration's aggressive efforts to prevent anything it believes would jeopardise national security reaching the public domain. Earlier this year, after a British court ordered disclosure of a seven-paragraph summary of classified CIA information showing what British agents knew of Mohamed's torture, the White House said it was "deeply disappointed" by the ruling and it could have an impact on intelligence-sharing between the countries.

The ACLU now plans to take the case to the supreme court, which will be called upon to make a crucial ruling on the president's power to restrict litigation that could reveal state secrets.

ACLU lawyer Ben Wizner told the New York Times: "To this date, not a single victim of the Bush administration's torture programme has had his day in court. That makes this a sad day not only for the torture survivors who are seeking justice in this case, but for all Americans who care about the rule of law and our nation's reputation in the world. If this decision stands, the United States will have closed its courts to torture victims while providing complete immunity to their torturers."

The prisoners' rights charity, Reprieve, which has represented Mohamed in the British courts, described the decision as regrettable.

"Yet again, those responsible for torture and rendition have used 'state secrecy' to avoid facing up to their crimes in court," said the executive director, Clare Algar. "The need for an independent inquiry into state involvement in torture has never been more urgent, and, if we are serious about learning from our mistakes, the UK must lead the way. We cannot learn from history unless we know what it is."

Obama had criticised the Bush administration's frequent use of the state-secrets privilege and last year the attorney general, Eric Holder, issued a new policy aimed at avoiding cases being shut down purely to prevent embarrassment or cover up illegality.

The judge said: "The government [in this case] is not invoking the privilege to avoid embarrassment or to escape scrutiny of its recent controversial transfer and interrogation policies, rather than to protect legitimate national security concerns."

Despite rejecting the lawsuit, Fisher urged the government to grant reparations where it could be proved that people had suffered human rights violations at the hands of the CIA.

But dissenting judge Michael Hawkins wrote: "Permitting the executive to police its own errors and determine the remedy dispensed would not only deprive the judiciary of its role, but also deprive plaintiffs of a fair assessment of their claims by a neutral arbiter."

The court ordered the government to pay the plaintiffs' costs despite the fact that Mohamed and the others had lost, and not requested payment.

Mohamed was detained in 2002 in Pakistan, where he was questioned incommunicado by an MI5 officer. The US flew him to Morocco, where he was subjected to more prolonged and brutal torture, including the repeated slashing of his genitals with a razor blade. He was then rendered to Afghanistan and finally Guantánamo. He was released and returned to Britain in February last year.