In its escalating confrontation with Iran, the US is making the same mistake it has made again and again since the fall of the Shah 40 years ago: it is ignoring the danger of plugging into what is in large part a religious conflict between Sunni and Shia Muslims.

I have spent much of my career as a correspondent in the Middle East, since the Iranian revolution in 1979, reporting crises and wars in which the US and its allies fatally underestimated the religious motivation of their adversaries. This has meant they have come out the loser, or simply failed to win, in conflicts in which the balance of forces appeared to them to be very much in their favour.

It has happened at least four times. It occurred in Lebanon after the Israeli invasion of 1982, when the turning point was the blowing up of the US Marine barracks in Beirut the following year, in which 241 US military personnel were killed. In the eight-year Iran-Iraq war during 1980-88, the west and the Sunni states of the region backed Saddam Hussein, but it ended in a stalemate. After 2003, the US-British attempt to turn post-Saddam Iraq into an anti-Iranian bastion spectacularly foundered. Similarly, after 2011, the west and states such as Saudi Arabia, Qatar and Turkey tried in vain to get rid of Bashar al-Assad and his regime in Syria – the one Arab state firmly in the Iranian camp.

Now the same process is under way yet again, and likely to fail for the same reasons as before: the US, along with its local allies, will be fighting not only Iran but whole Shia communities in different countries, mostly in the northern tier of the Middle East between Afghanistan and the Mediterranean.

Donald Trump looks to sanctions to squeeze Iran while national security adviser John Bolton and secretary of state Mike Pompeo promote war as a desirable option. But all three denounce Hezbollah in Lebanon or the Popular Mobilisation Units in Iraq as Iranian proxies, though they are primarily the military and political arm of the indigenous Shia, which are a plurality in Lebanon, a majority in Iraq and a controlling minority in Syria. The Iranians may be able to strongly influence these groups, but they are not Iranian puppets which would wither and disappear once Iranian backing is removed.

The Islamic Republic of Iran 40 years on Show all 20 1 /20 The Islamic Republic of Iran 40 years on The Islamic Republic of Iran 40 years on Women praying during Ashura festivities in Yazd in February Jan Schneider The Islamic Republic of Iran 40 years on Cars cemented in at the Holy Defence Museum-Garden in Khorramshahr – in 1980 occupying Iraqi forces used such formations to deter paratrooper landings Jan Schneider The Islamic Republic of Iran 40 years on Women on the Si-o-se-pol bridge over the Zayanderude river in Esfahan Jan Schneider The Islamic Republic of Iran 40 years on Iranian tourists at Vank cathedral in the Armenian quarter in the city of Esfahan Jan Schneider The Islamic Republic of Iran 40 years on A reddened fountain in Yazd – a national motif recalling the countries ‘martyrs’ Pictures by Jan Schneider The Islamic Republic of Iran 40 years on Prayers before a list naming the fallen of the Iran-Iraq war in Tehran Jan Schneider The Islamic Republic of Iran 40 years on In a museum in central Tehran, a wax figure of Ayatollah Khomeini Jan Schneider The Islamic Republic of Iran 40 years on Carom billiards in Tehran Jan Schneider The Islamic Republic of Iran 40 years on Nearly two thirds of Iran’s population is under-30 Jan Schneider The Islamic Republic of Iran 40 years on Sanctions are exacerbating poverty Jan Schneider The Islamic Republic of Iran 40 years on Young people are adept at catching moments of freedom, such as this one in the capital Jan Schneider The Islamic Republic of Iran 40 years on At an art auction in Tehran, the wealthy turn out Jan Schneider The Islamic Republic of Iran 40 years on Propaganda murals are everywhere Jan Schneider The Islamic Republic of Iran 40 years on Tehranis love their outdoor spaces too Jan Schneider The Islamic Republic of Iran 40 years on The hypersaline waters of Lake Urmia are fast diminishing – Iran faces ecological catastrophe Jan Schneider The Islamic Republic of Iran 40 years on A martyr is depicted outside the huge Beheshte-e-Zahra cemetery in Tehran Jan Schneider The Islamic Republic of Iran 40 years on A guard naps outside Iran’s foremost tourist attraction – the ancient ruins of Persepolis Jan Schneider The Islamic Republic of Iran 40 years on Memorial to Iranian pilgrims who died during a stampede in Mecca 1987 Jan Schneider The Islamic Republic of Iran 40 years on The martyr fixation on display in the southern city of Shush Jan Schneider The Islamic Republic of Iran 40 years on Emamzadeh Saleh mosque in Shemiran, Tehran Jan Schneider

Allegiance to nation states in the Middle East is generally weaker than loyalty to communities defined by religion, such the Alawites, the two-million-strong ruling Shia sect in Syria to which Bashar al-Assad and his closest lieutenants belong. People will fight and die to defend their religious identity but not necessarily for the nationality printed on their passports.

When the militarised Islamist cult Isis defeated the Iraqi national army by capturing Mosul in 2014, it was a fatwa from the Shia Grand Ayatollah Ali al-Sistani that sent tens of thousands of volunteers rushing to defend Baghdad. Earlier in the fighting in Homs and Damascus in Syria, it was the non-Sunni districts that were the strongpoints of the regime. For example, the opposition were eager to take the strategically important airport road in the capital, but were held back by a district defended by Druze and Christian militiamen.

This is not what Trump’s allies in Saudi Arabia, UAE and Israel want Washington to believe; for them, the Shia are all Iranian stooges. For the Saudis, every rocket fired by the Houthis in Yemen into Saudi Arabia – though minimal in destructive power compared to the four-year Saudi bombing campaign in Yemen –can only have happened because of a direct instruction from Tehran.

On Thursday, for instance, Prince Khalid Bin Salman, the vice minister for defence and the brother of Saudi Arabia’s de facto ruler Crown Prince Mohammed Bin Salman, claimed on Twitter that drone attacks on Saudi oil pumping stations, were “ordered” by Iran. He said that “the terrorist acts, ordered by the regime in Tehran, and carried out by the Houthis, are tightening the noose around the ongoing political efforts”. He added: “These militias are merely a tool that Iran’s regime uses to implement its expansionist agenda in the region.”

There is nothing new in this paranoid reaction by Sunni rulers to actions by distinct Shia communities (in this case the Houthis) attributing everything without exception to the guiding hand of Iran. I was in Bahrain in 2011 where the minority Sunni monarchy had just brutally crushed protests by the Shia majority with Saudi military support. Among those tortured were Shia doctors in a hospital who had treated injured demonstrators. Part of the evidence against them was a piece of technologically advanced medical equipment – I cannot remember if it was used for monitoring the heart or the brain or some other condition – which the doctors were accused of using to receive instructions from Iran about how to promote a revolution.

Support free-thinking journalism and attend Independent events

This type of absurd conspiracy theory used not to get much of hearing in Washington, but Trump and his acolytes are on record on as saying that nearly all acts of “terrorism” can be traced to Iran. This conviction risks sparking a war between the US and Iran because there are plenty of angry Shia in the Middle East who might well attack some US facility on their own accord.

It might also lead to somebody in one of those states eager for a US-Iran armed conflict – Saudi Arabia, UAE and Israel come to mind – that staging a provocative incident that could be blamed on Iran might be in their interests.

But what would such a war achieve? The military invasion of Iran is not militarily or politically feasible so there would be no decisive victory. An air campaign and a close naval blockade of Iran might be possible, but there are plenty of pressure points through which Iran could retaliate, from mines in the Strait of Hormuz to rockets fired at the Saudi oil facilities on the western side of Gulf.

A little-noticed feature of the US denunciations of Iranian interference using local proxies in Iraq, Syria and Lebanon is not just that they are exaggerated but, even if they were true, they come far too late. Iran is already on the winning side in all three countries.