Europe's highest court today said it would not allow Abu Hamza and three other men held in Britain to be extradited to the United States to face terrorism charges until it is satisfied they will not be treated inhumanely.

The European court of human rights' decision released this morning means it will in effect sit in judgment on parts of the US's criminal justice system.

It will decide if practices such as subjecting prisoners to prolonged periods of isolation and sentences of life without parole fall short of international standards.

The court in Strasbourg has already been considering the cases of the four men for three years, and has asked for more submissions to be made before it reaches a final ruling.

Hamza, the radical cleric who became the face of extremism in Britain, is serving a seven-year sentence in Britain after being convicted in 2006 of inciting hatred. The previous government had wanted him sent to the US before his jail term had been completed, but the extradition was halted after his lawyers went to the European court.

Lawyers for the men argued that if convicted, the conditions they would be held in were so severe they amounted to a breach of article three of the European convention on human rights. It says that no one should be subject to "inhuman or degrading treatment or punishment".

They said the men would be held in a "supermax" prison where inmates are subjected to isolation that critics say can damage their mental health.

Hamza and two other terror suspects face sentences of life without parole if convicted, with the fourth facing a 50-year sentence.

Barrister Julian Knowles, an expert in extradition, said: "What the court will be deciding is whether someone can be extradited to a sentence of life without parole, to be spent in solitary confinement, and whether that is compatible with the European system of human rights."

The court wants the British government to make fresh submissions by September, to which the men's lawyers can then respond. Among the questions it will decide is would the men, if convicted, ever be allowed into a less harsh prison regime, and whether the US constitution which bans "cruel and unusual punishment" sufficiently protects their human rights.

The case involves Hamza and three other British men, including Babar Ahmad, who the US claims ran a website allegedly raising funds for Islamic extremists. The 36-year-old computer expert from south London has been on remand and refused bail since his arrest in August 2004 on a US extradition warrant.

He was awarded £60,000 in compensation because of the violence British police used during his December 2003 arrest, during which he was punched, kicked and throttled. His case has been supported by the former transport minister Sadiq Khan MP, who is a family friend.

The other suspects are Haroon Rashid Aswat and Seyla Talha Ahsan, and the US claim that between 2004-06 all four men were international terrorists, which the men deny.

Gareth Peirce, solicitor for Ahmad, said the European court's decision was a rare test of whether the US was meeting acceptable standards of human rights: "It's a very important test of whether the way the US treats its prisoners meets international standards."

Peirce said it was as serious a case as when the European court decided to prohibit extradition to the US in cases where the death penalty was possible, unless guarantees were given that execution would not be sought.

In the Hamza case, the US has had to given written assurances that it will not impose the death penalty or place the suspects before Guantánamo Bay-style military tribunals.

The US alleges Hamza was in contact with Taliban and al-Qaida terrorists and aided the hostage-taking of 16 western tourists in Yemen in December 1998 that ended in the deaths of three Britons. He is also charged with attempting to set up a training camp for "violent jihad" in Oregon in 1999, along with Aswat.

The family of Ahmad welcomed the ruling and called for the UK government to try him in the British courts instead of allowing him to be sent to the US: "Babar's extradition should be temporarily frozen due to deep concerns it has regarding possible 'cruel and unusual punishment' of Babar if sent to the United States.

"Our solicitor, Gareth Peirce, has made urgent fresh representations to the attorney general, Dominic Grieve, asking the UK coalition government to immediately stop Babar's extradition and put him on trial in Britain so he has the chance to prove his innocence before a jury of his peers."

The home secretary, Theresa May, said: ""We note that the European court of human rights has decided that all the applications are partly admissible.

"We await the court's judgment on the case. In the meantime these individuals will remain in custody."

Explaining its action, the European court said: "The court further declared all four applicants' complaints under Article 3 concerning the length of their possible sentences admissible for further examination of the merits, Mr Ahmad, Mr Ahsan and Abu Hamza facing life sentences without parole and Mr Aswat a maximum of 50 years' imprisonment (meaning he would be nearly 78 before becoming eligible for release)."

The court said Hamza's claim about being held in a supermax prison would not be heard because he would spend little time in one before being transferred to a medical facility because of his ill health.

The judges said they would consider whether imprisonment in solitary confinement for the three Britons amounted to inhumane treatment so serious that they could not be extradited.

The prison is in Florence, Colorado, and is known as ADX Florence for short: "The court considered that Mr Ahmad, Mr Aswat and Mr Ahsan were at real risk of being held at ADX Florence if convicted and that their complaints under Article 3 concerning the stringency of conditions there for what could be the rest of their lives, raised serious questions of fact and law which were of such complexity that the court had to examine them on the merits."