The Ministry of Defence has paid out £14m in compensation and costs to hundreds of Iraqis who complained that they were illegally detained and tortured by British forces during the five-year occupation of the south-east of the country.

Hundreds more claims are in the pipeline as Iraqis become aware that they are able to bring proceedings against the UK authorities in the London courts.

The MoD says it is investigating every allegation of abuse that has been made, adding that the majority of British servicemen and women deployed to Iraq conducted themselves "with the highest standards of integrity".

However, human rights groups and lawyers representing former prisoners say that the abuse was systemic, with military interrogators and guards responsible for the mistreatment acting in accordance with both their training in the UK and orders issued in Iraq.

The campaigners are calling for a public inquiry into the UK's detention and interrogation practices following the 2003 invasion. An inquiry would be a development the MoD would be eager to avoid.

Payments totalling £8.3m have been made to 162 Iraqis this year. There were payments to 17 individuals last year and 26 in the three years before that.

The average payment to the 205 people who have made successful claims has been almost £70,000, including costs. The MoD says it is negotiating payments concerning a further 196 individuals.

Lawyers representing former prisoners of the British military say that more than 700 further individuals are likely to make claims next year.

Most of those compensated were male civilians who said they had been beaten, deprived of sleep and threatened before being interrogated by British servicemen and women who had detained them on suspicion of involvement in the violent insurgency against the occupation. Others said that they suffered sexual humiliation and were forced into stress positions for prolonged periods.

Many of the complaints arise out of the actions of a shadowy military intelligence unit called the Joint Forward Interrogation Team (Jfit) which operated an interrogation centre throughout the five-year occupation. Officials of the International Committee of the Red Cross complained about the mistreatment of detainees at Jfit not long after it was first established.

Despite this, the interrogators shot hundreds of video films in which they captured themselves threatening and abusing men who can be seen to be bruised, disoriented, complaining of starvation and sleep deprivation and, in some cases, too exhausted to stand unaided.

A former soldier who served as a guard at Jfit told the Guardian that he and others were ordered to take hold of blindfolded prisoners by their thumbs in between interrogation sessions then drag them around assault courses where they could not be filmed.

He also confirmed that the prisoners were often beaten during these runs, and that they would then be returned for interrogation in front of a video camera.

The interrogators were drawn from all three branches of the forces and included a large number of reservists.

During proceedings brought before the high court in London, lawyers representing the former Jfit prisoners suggested the interrogation centre could be regarded as "Britain's Abu Ghraib".

Questioned about the compensation payments, an MoD spokesperson said: "Over 120,000 British troops have served in Iraq and the vast majority have conducted themselves with the highest standards of integrity and professionalism. All allegations of abuse will always be investigated thoroughly. We will compensate victims of abuse where it is right to do so and seek to ensure that those responsible are brought to justice."

Lutz Oette, legal counsel at Redress, a London-based NGO which helps torture survivors get justice, said: "The payments provide a long overdue measure of redress. However, for the victims compensation without truth and accountability is a heavy price to pay. For justice to be done there is a need for a full independent inquiry to establish what happened and who is responsible.

"Looking at the number of claimants and scale of payments, there clearly seems to be a systemic problem. It is high time for this to be fully accounted for, first and foremost for the victims but also the British public, which has an obvious interest to know the truth behind the figures."

Next month, the high court will hear a judicial review of the MoD's refusal to hold a public inquiry into the abuses. Human rights groups and lawyers for the former prisoners say the UK government is obliged to hold an inquiry to meet its obligations under the European convention on human rights – and particularly under article three of the convention, which protects individuals from torture.

After a hearing, the high court highlighted matters supporting the allegations of systemic abuse. These included:

• The same techniques being used at the same places for the same purpose: to assist interrogation.

• The facilities being under the command of an officer.

• Military doctors examining each prisoner at various stages in their detention.

• Investigations by the Royal Military police that were concluded without anyone being held to account.

If the court does order a public inquiry, responsibility for any systemic abuse is likely to be traced up the military chain of command and beyond.

The MoD claims no public inquiry is necessary as it has instituted an investigation body, the Iraq Historical Allegations Team (Ihat), which is examining the abuse allegations as well as a number of prisoner deaths in British military custody.

After Ihat investigators examined the videos shot at Jfit, three interrogators were referred to the Service Prosecuting Authority with a recommendation that war crimes charges be considered.

Prosecutors eventually decided that the matters were insufficiently serious for war crimes charges and that disciplinary charges were unlikely to lead to convictions. They concluded that one soldier had committed offences, but that this was "in accordance with the training that they had been given"; it would be inappropriate to charge him.

Other inquiries have led Ihat to recommend that the MoD makes compensation payments to former prisoners.

But lawyers for the former prisoners believe Ihat is insufficiently independent as it answers to MoD officials. One investigator quit Ihat alleging that the organisation's inquiry is not genuine, but more a face-saving exercise.