The US and British governments never appreciated the hatred with which Iraqis at all levels regarded foreign security contractors.

They were detested as freelance gunmen with licences to kill or maim Iraqis, safe in the knowledge that they had the same immunity from Iraqi law as US soldiers. They often appeared to view Iraqis as a hostile sub-species to be treated with suspicion. The killing of 17 civilians by personnel from Blackwater Worldwide in Nisour Square in west Baghdad on 16 September 2007 might be the most notorious incident but there were many more, some of which remain unpublicised.

In the crowded streets of Baghdad Iraqi drivers would try to keep their distance from contractors’ convoys because the security men would nonchalantly fire into the engine blocks of vehicles they thought had got too close. Sometimes they would simply shoot the driver and his passengers. One such victim was as the dean of al-Nahrain University, Professor Khalid al-Judi. I found him in a hospital in 2004, shot through the abdomen while on his way to degree ceremony by men in a truck wearing flak jackets and carrying American rifles. “We got stuck in a traffic jam,” his Iraqi bodyguard told me. “When they got close to Professor al-Judi’s car one of the men in the GMC opened fire.” The truck drove off so nobody knew who had been doing the shooting.

By late 2007 an estimated 180,000 private contractors worked in Iraq, of whom between 25,000 and 30,000 were armed security men. About half of these worked for the US government; they included 1,500 from Blackwater, Triple Canopy and DynCorp who provided security for the US State Department.

There were standoffs with Iraqi army and police units. In 2006 a drunk foreign security man shot dead an Iraqi protecting the vice-president, Abd al-Mahdi. More than 30 security companies provided about 10,000 guards to the US military, and the majority had mundane duties requiring few skills. Such recruits were hired as cheaply as possible from countries like Peru, Nepal and Uganda, where salaries were low and unemployment high.

The quality of these security men varied greatly, from efficient ex-Gurkha soldiers to bewildered-looking former policemen from Lima. The latter spoke almost no English or Arabic and would sadly show me picture postcards from Peru saying “Machu Pichu good!” and give thumbs-up signs. These men were in fixed though dangerous positions, such as the entrances to the Green Zone.

Foreign ambassadors often gained an exaggerated idea of the undoubted dangers of travelling in Baghdad from their security guards, who lived in a high state of paranoia. The company guarding one West European ambassador demanded 24 hours’ notice before he took even the shortest trip outside the Green Zone so that they could examine every speed bump for explosives. It was never in the interests of a security company to downplay dangers. Ambassadors who relied on military policemen from their own armies were able to get a more balanced view of the local security situation.

Many foreign security companies felt that their reputations were unfairly damaged by Blackwater and others, which they said were atypical. “We nicknamed Blackwater dishwater,” said one. Unsurprisingly, Iraqi politicians were keen to remove the immunity of foreign private contractors during the negotiations with the US over the final Status of Forces Agreement with Iraq, which came into force when the last US troops left in 2011. Iraqi military and police wanted foreign security companies out so they could take contracts themselves.

The outsourcing of military functions, particularly those to do with supply and guard duties, was in keeping with the free market ideology of the US and Britain during the Iraq war. But it had grave drawbacks, such as giving aggressive gunmen the right to shoot first and ask questions afterwards. It also meant that crucial functions, such as driving trucks to outposts under semi-siege, were carried out by men who had not expected to be shot at and were trying to earn a little extra money.

In Afghanistan military supply convoys from Pakistan have to pay off the Taliban to avoid attack, so foreign forces are in effect subsidising the insurgents they are trying to defeat. As in Joseph Heller’s Catch-22, the decision to privatise war means profitability is pursued at the expense of military effectiveness and the door opened for disasters such as Nisour Square.

PATRICK COCKBURN is the author of “Muqtada: Muqtada Al-Sadr, the Shia Revival, and the Struggle for Iraq.