A series of public inquests should be held into the deaths of civilians who are alleged to have been killed unlawfully by the British military following the 2003 invasion of Iraq, the high court has ruled.

In a ground-breaking judgment that could have an impact on how the British military is able to conduct operations among civilians in the future, the court ruled on Friday that up to 161 deaths should be the subject of hearings modelled upon coroners' inquests.

In practice, a series of hearings – possibly amounting to more than 100 – are likely to be held as a result of the judgment, which follows a three-year legal battle on behalf of the Iraqis' families.

Each hearing must involve a "full, fair and fearless investigation accessible to the victim's families and to the public", the court ruled, and should examine not only the immediate circumstances but other issues surrounding each death.

As a first step, the court ordered Philip Hammond, the defence secretary, to announce within six weeks whether any of the deaths are to result in prosecutions, or to explain any further delays over prosecuting decisions.

After years of judicial review proceedings, and in the face of determined opposition from the Ministry of Defence, which appeared anxious to maintain control over any investigative process, the court concluded that hearings modelled upon coroners' inquests were the best way for the British authorities to meet their obligations under article 2 of the European convention on human rights (ECHR), which protects the right to life.

The court rejected calls for a single public inquiry to be held to examine British military detention and interrogation practices during the six-year occupation of the south-east of Iraq, on the grounds that such an inquiry would take too long and cost too much.

The first public hearings are expected to focus on 11 cases in which civilians died in British military custody. These will be followed by public examinations of other incidents in which civilians died – often in circumstances that were unclear or are disputed – but where it is alleged that British personnel used unlawful force.

The court also ruled that this should be just the start of the process by which public hearings will examine the alleged misconduct of some members of the British armed forces who served in Iraq.

Following the completion of the Article 2 hearings – into allegedly unlawful killings – further hearings should be established in order to meet the UK's obligations under Article 3 of the ECHR, the court said. These will inquire into allegations of torture and lesser mistreatment of individuals detained by British troops in Iraq, focusing on a sample of the most serious of the 700-plus cases in which such allegations have been made.

In December last year the MoD said it had paid out £14m in compensation and costs to 205 Iraqis who alleged unlawful imprisonment and mistreatment, and that it was negotiating a further 196 payments. Several hundred more claims were expected to be lodged.

Many of the hearings that examine allegations of torture and mistreatment are expected to shed light on the activities of a British interrogation unit known as the Joint Forward Interrogation Team, whose personnel, drawn from all three branches of the services, appear to have received training in the abuse of prisoners.

The deaths in custody include those of a number of teenagers who drowned after allegedly being pushed into canals.

In another case, an RAF police investigation into the death of a man who was allegedly kicked to death while on board an RAF helicopter failed to establish the cause of death, or even the dead man's name. A Guardian investigation established that there were concerns that MoD officials may have interfered with the initial investigation because they feared a prosecution could lead to the discovery that the dead man – and large numbers of other prisoners – were being ferried to a secret interrogation centre that had been concealed from both British army lawyers and the Red Cross.

A third case concerns a mother who was shot and wounded in November 2006 when troops raided her home. She was certified dead on arrival at a first aid post, and her body was later dumped at the side of a road in a British army body bag.

Baha Mousa. Photograph: Liberty/PA

Several other victims died after being detained for interrogation, and are thought to have been beaten to death in a manner similar to that of Baha Mousa, the Basra hotel receptionist who was murdered by British soldiers in September 2003.

The court said it had examined "allegations of the most serious kind involving murder, manslaughter, the wilful infliction of serious bodily injury, sexual indignities, cruel inhuman and degrading treatment and large scale violation of international humanitarian law".

The judgment from Sir John Thomas, president of the Queen's Bench Division, and Mr Justice Silber, added that there was evidence to support claims that some of the abuse had been systemic, and questioned whether responsibility for poor training and a failure to investigate promptly lay with senior officers and figures in government.

Many of the cases have been under investigation for a number of years by a body known as the Iraq Historic Allegations Team (Ihat), which is made up of retired civilian detectives and service police, answering to an MoD panel.

The court ruled that Ihat was an independent body, and rejected completely claims made by a former Ihat official-turned-whistleblower, who had dismissed it as a face-saving exercise. Nevertheless, the judgment concluded, Ihat was not capable of examining the instructions, training and supervision of troops, and the way in which this is alleged to have led to the abuse.

Phil Shiner, a lawyer for the Iraqis, said: "My clients welcome, at last, the opportunities for accountability flowing from the judgment.

"I trust that the various and troubling systemic issues emerging from these cases will lead to further reforms following the Baha Mousa inquiry report of September 2011. The secretary of state must ensure that UK forces abroad respect and apply the rule of law."

An MoD spokesperson said: "We welcome the court's finding that Ihat is independent and carries out its duties with competence and integrity. We are also pleased that the court has agreed that the secretary of state was justified in concluding that a single comprehensive public inquiry into allegations of abuse in Iraq should not be established. This would have led to unnecessary expense and unacceptable delay.

"Ihat continues to make progress in its task of investigating allegations which have been made against British troops serving in Iraq."