The Ministry of Defence has announced a public inquiry into one of the most notorious episodes involving British soldiers in Iraq: the death of Baha Mousa, a hotel receptionist, and the abuse of other civilians in Basra in September 2003.

The unexpected move was announced in a written statement to the Commons this afternoon by Bob Ainsworth, the armed forces minister.

After months of bitter argument in the courts and a court martial that failed to get close to the full story of what happened, the government has decided the issue will not go away and was persuaded that an open inquiry was the best answer.

"Someone died, we must find out what happened and in what circumstances," a well-placed official told the Guardian.

Mousa, 26, died while being held for a weekend in a British detention centre. He had 93 identifiable injuries on his body and had suffered asphyxiation. Eight other Iraqis were inhumanely treated.

Six soldiers of the Queen's Lancashire Regiment, including Colonel Jorge Mendonca, the commanding officer, were acquitted of negligence and abuse. A corporal admitted inhumane treatment. No one was convicted of killing Mousa.

Des Browne, the defence secretary, admitted in March to "substantive breaches" of the European convention on human rights, specifically articles two and three of the convention, which guarantee the right to life and prohibit torture.

The admissions, which could cost the MoD millions of pounds in compensation, followed a vigorous campaign in and outside the courts by relatives of Mousa and the other Iraqi victims. They and their lawyers argued that the Human Rights Act demanded an independent inquiry into the incident.

Today, Browne told MPs: "A public inquiry into the death of Baha Mousa is the right thing to do. It will reassure the public that we are leaving no stone unturned in investigating his tragic death. The army has nothing to hide in this respect and is keen to learn all the lessons it can from this terrible incident."

The inquiry will investigate not only what happened but why. It will look into why the soldiers and other senior figures in the British army were apparently unaware that five brutal interrogation techniques - including wall-standing, hooding, subjection to noise, sleep deprivation and deprivation of food and drink - were banned.

The methods were expressly prohibited in 1972 by the then prime minister, Edward Heath, after the European human rights court condemned techniques used by the security forces in Northern Ireland.

The court martial heard that British soldiers used "conditioning" techniques to "soften up" Iraqi detainees. It revealed that army officers had ignored the 1972 ban on hooding and other techniques. Signficantly, Brigadier Euan Duncan, the director of the army's Intelligence Corps, told the court that US commanders had criticised British forces in 2003 for failing to extract sufficient intelligence from detainees.

In a report earlier this year, Brigadier Robert Aitken, the army's director of personnel strategy, said British soldiers had not been told about their obligations under international law. Troops were given "scant" information on how to treat civilian detainees and needed "a better understanding between right and wrong", he said. The military's Defence Intelligence and Security Centre did not mention the five techniques banned in 1972, he said.

Phil Shiner, a lawyer for the Iraqis, insisted today that any new inquiry must include other claims of ill-treatment by British troops. "It will not be sufficient if the inquiry has a narrow remit and does not look at all the cases and issues," he said.

"The public, as well as parliament, must be given the opportunity of fully understanding what went wrong in our detention policy in Iraq and what are the lessons to be learned for the future."

He said his clients wanted a single inquiry into the British detention policy in Iraq. "The most serious allegations that could be made about UK forces' behaviour in Iraq include that 20 Iraqis were executed at Abu Naji facility in May 2004, another nine survivors tortured, and that bodies were mutilated," Shiner said.

The investigation announced today will be held under the 2005 Inquiries Act. The act - drawn up after the Bloody Sunday inquiry and the Hutton inquiry into the death of the government weapons expert David Kelly - gave the government new powers to control aspects of the conduct of independent public inquiries.