Chinese mandarins in the 19th century were faced with the delicate task of explaining to their emperor the repeated defeats of his armies. To avoid embarrassing questions, they adopted the simple device of describing them all as victories. The British Army has adopted a similar approach in explaining its failure in Iraq and Afghanistan. Self-serving myths are cultivated, putting all the blame on Tony Blair and the Americans for pitching British soldiers into unwinnable wars. Responsibility for the insufficient number of troops deployed or the lack of appropriate equipment is held to lie with the politicians, not the generals.

The result is that there is a lack of appreciation in Britain of the extent of the purely military failure. In Iraq in 2007, four years after British troops invaded, they were in the humiliating position of being largely confined to a camp outside Basra, while Shia militiamen ruled the city. The British contingent did little except defend itself and had become, in the graphic phrase of British soldiers, a "self-licking lollipop".

Failure after British troops were deployed in Helmand province in 2006 came even more quickly. Military intelligence wholly misjudged the danger of sending troops there. Assault troops devoted themselves to trying to destroy the Taliban and, by alienating the local population, acted as its recruiting sergeant. Little account was taken of local reaction to a foreign occupation that brought only death and destruction.

Download the new Independent Premium app Sharing the full story, not just the headlines

I had an early brush with the dysfunctional nature of the British military effort in Iraq in 2003 when six British military policemen were killed in al-Majar al-Kabir on the southern marshes. Its people were smugglers and guerrilla fighters who had fought Saddam Hussein's forces for decades. Somehow, the British Army thought it could freely patrol the town and disarm its people without resistance. The locals were deeply and understandably suspicious, asking why, if the British and Americans were not planning a long-term occupation of Iraq, were they trying to take away the weapons of stalwarts of the anti-Saddam resistance movement? British generals portrayed the killings as an atypical local crisis, but it was symptomatic of much that was to go wrong for Britain in these two wars.

Many journalists, diplomats and soldiers know different pieces of the jigsaw puzzle of misunderstandings, ignorance and institutional failures which produced these debacles. But few have the long and diverse experience of the Army in Iraq and Afghanistan that enabled Frank Ledwidge to produce his superb ground-breaking book, Losing Small Wars: British Military Failure in Iraq and Afghanistan. For 15 years he was a naval reserve military intelligence officer before retiring in 2008. During that time, he was a close observer of how military operations were conceived, carried out and assessed. Such "insider" books can be shrill in tone or overstate their case, but Mr Ledwidge is judicious, sceptical, intelligent and highly informed.

He arrived in Helmand "to take part in what I believed to be a just and winnable war". But he found that the briefings he attended gave a picture of the war the Army would have liked to be fighting, not the one it was actually engaged in. It became "luminously clear from the daily reports that British forces controlled only the area a few hundred metres from the barrels of the machine-guns guarding our beleaguered bases".

This was very much a re-run of the situation in Basra, where the British lost control of the city. The final effort, Operation Sinbad in 2006-07, was renamed Operation Spinbad by British soldiers involved; one described how his battalion had to fight its way into the city, build a playground with slides and swings, and then fight its way out. Insurgents immediately dismantled the playground, retaining only the slides to be used for launching rockets at the departing British.

Intervention in Helmand, about which the British military knew little, was conceived as a way of redeeming the Army's damaged reputation in the eyes of the Americans. Military intelligence downplayed warnings from the British embassy in Kabul that Helmand was likely to be a very dangerous place. A Pakistani colonel once said to me that he thought the British and American armies in Afghanistan had not taken on board the fact that at the heart of Pashtun culture "is a hatred of foreigners".

Mr Ledwidge is particularly good at understanding how the very fact of occupation generates political and military antibodies to repel it. There is nothing peculiar about Afghan society in this. He cites a senior development officer with experience of Afghanistan saying "lots of soldiers talk about what is culturally unacceptable. What is culturally unacceptable is dropping 5,000lb bombs on their towns and raiding their houses, pulling wives and daughters out of bed. Same as in Britain, really."

There were other important reasons for British failure. Brigades were sent on six-month tours, far too short. Commanding officers seemed to want their stints to be marked by a career-enhancing military effort, with little connection to actions before or after. British policy was very much militarised, unlike the North West Frontier during British rule of India where military force was only a last resort. Self-regarding and misleading lessons were drawn from counter-insurgency in Malaya and Northern Ireland, which were wholly different from Iraq and Afghanistan.

A major source of weakness was that the most important reason for the British to deploy the Army in both these wars was to demonstrate to Washington that Britain remained its most important ally. Everything else, including operational success, was secondary to this aim. Both Tony Blair and David Cameron have maintained a pretence that Britain is fighting to maintain in power a democratic Afghan government. The reality is that the representatives of this government are often warlords engaged in extortion, corruption and kidnapping. Afghan police are notorious for stealing money, consuming drugs and raping young men and women passing through their checkpoints.

Four years after the British arrived in Sangin, a local farmer was quoted as saying "the Taliban do not even have a bakery that they can give bread, but still most people support the Taliban – that's because people are sick of night raids and being treated badly by the foreigners".