The United States did not make the modern Middle East, nor did it create its system of rule.

It did, along with the Soviet Union, provide a stiff prop to it by supporting allied regimes. It backed proxies and played kingmaker in the politics of many states.

Because its chief goal was to prevent the region from falling under Soviet influence it privileged "stability" above all else, including the promotion of human rights and democracy. To its critics this was immoral, but to its supporters it was a prudent pursuit of the national interest.

Almost no US allies in the Middle East shared America's democratic values, although it was often reproached for supporting one country that did — Israel.

In 1991, the United States reached the apogee of its power in the region. When Iraq invaded Kuwait in 1990, the US assembled an international military coalition that even included old adversaries such as Syria. The coalition crushed the Iraqi army, then the region's largest. Less than a year later the Soviet Union collapsed leaving the US as the Middle East's dominant force.

US president George H.W. Bush meets troops serving in the First Gulf War in 1990. ( Reuters )

The peacemaker

Somewhat oddly for a status quo player, the US then chose to remake the region — twice.

It first tried to do this by making peace. After the Iraq War, the administration of George H.W. Bush seated a reluctant Israel at a table in Madrid with most Arab countries as well as a Palestinian delegation.

Palestine Liberation Organization leader Yasser Arafat. ( ABC TV )

The effort stalled but from its frustrations emerged the so-called Oslo Process, which eventually saw the Palestine Liberation Organization recognised by Israel and its leader, Yasser Arafat, returned to the Palestinian territories.

Peacemaking was pursued even more enthusiastically by Mr Bush's successor, Bill Clinton. For eight years, he struggled through successes and setbacks, culminating in two summits. In March 2000, Mr Clinton met with Syrian president Hafez al-Assad in Geneva to broker a peace agreement between Israel and Syria.

Later that year came the more famous summit between Israeli and Palestinian leaders hosted by Mr Clinton at Camp David. Both summits failed.

Even though the responsibility for these failures were not Washington's alone, it damaged US credibility and created an opening for more radical forces. The first Bush and Clinton administrations staked considerable prestige on their promise to bring peace to the Middle East. When this failed, America's opponents in the region could argue that its power was not as great as it seemed and that violence was a better way to achieve their goals.

September 11 revived US influence

As if to prove the point, on September 11, 2001, al-Qaeda struck the US.

The September 11 attacks brought another peak in America's involvement with the Middle East. ( Reuters: Sean Adair )

The irony is that after al-Qaeda's attacks, US influence in the Middle East temporarily revived. Many were sympathetic towards America, although they also feared how it would respond. The US used that renewed authority to depose the Taliban in Afghanistan, and kill, capture or scatter al-Qaeda's jihadists. It then promptly squandered the momentum.

In 2003, the US launched what became its second major effort to remake the Middle East. The Bush administration justified the invasion of Iraq as an effort to remove that country's weapons of mass destruction — weapons that never materialised. But more broadly it described it as an attempt to democratise the region and thereby "drain the swamp" of extremists.

In fact, the Iraq War filled it further as extremists exploited the war's disastrous outcome to press their cause. It eroded faith in American competence in the region (and globally). Strategically, however, the most significant effect was that it increased Iran's influence.

In the 1980s, Iran had promised to export its Islamic revolution. But those ambitions were curtailed by a combination of American containment and Iran's disastrous eight-year war with Iraq.

Iran's Revolutionary Guards commemorate the disastrous eight-year Iran-Iraq war in Tehran. ( Morteza Nikoubazl: Reuters )

Iran's changing role

The US failure in Iraq, however, changed Iran's fortunes. After Saddam Hussein's overthrow, Iran parlayed its long history of supporting Iraq's Shi'a opposition into a decisive role in the country's politics. The rise of Islamic State in 2013 gave it an even greater pretext to intervene on the grounds of protecting Iraq against Sunni jihadists.

Iran exploited opportunities caused by chaos in other parts of the region as well, most notably in Syria where Iran backed a successful defence of Bashar's regime. In Yemen, Iran provided opportunistic support to a rebellion led by the so-called Houthis — a Zaidi Shia Islamist movement — which in the chaotic aftermath of the uprisings of 2011 captured the Yemeni capital.

US allies saw an expanding arc of Iranian influence stretching from Yemen to Lebanon and looked to Washington to roll it back. But after the disaster of Iraq a new president, Barack Obama, made it clear he wanted to end US wars in the region, not start new ones — even if the abrupt emergence of Islamic State in 2014 meant he had to temper that somewhat.

Barack Obama hoped to withdraw from the Middle East, but was hamstrung by the rise of Islamic State. ( Reuters: Larry Downing )

It was also a reflection that the US could no longer afford wars in the Middle East and the American people would no longer support them.

This meant little to US allies in the region. From Jerusalem to Riyadh, America's friends condemned Mr Obama in private, and sometimes even in public.

They were incensed by his response to the Arab uprisings in 2011.

Obama failed to protect long-standing US ally Hosni Mubarak from losing power in Egypt's 2011 revolution. ( Reuters )

In their eyes, he had refused to save key US ally, then Egyptian president Hosni Mubarak, which implied that they, too, might one day be sacrificed.

In 2013, Mr Obama's refusal to enforce his "red line" over the Syrian regime's use of chemical weapons against its own people gave his critics more ammunition.

Fury with Obama's accord

But what enraged many allies was Mr Obama's decision to forge a nuclear accord with Iran in 2015. Most US regional allies were sceptical that any agreement would inhibit Iran's nuclear ambitions. Allies such as Israel and Saudi Arabia wanted Iran contained, not engaged.

Some US allies took matters into their own hands. In 2015 Saudi Arabia and the United Arab Emirates launched a war in Yemen to defeat the Houthis and stem Iranian influence.

Saudi Arabia and the United Arab Emirates have battled Yemen's Houthi rebels since 2015. ( Reuters: Khaled Abdullah )

New alliances within the region were also created, including an increasingly overt relationship between Saudi Arabia and Israel founded on their common antipathy towards Iran. Not that long ago such a connection would have been unthinkable.

External players also seized opportunities. Russia came to the defence of Bashar al-Assad, in part to revive an old Soviet strategic foothold, in part to project Russia as a global player and in part to throw brine in the American eye. More quietly, China expanded its economic and political relationships.

Enter Trump

All this helped explain the warm welcome President Donald Trump received from many regional leaders when he visited in 2017.

Donald Trump is welcomed by traditional dancers to Murabba Palace in Saudi Arabia. ( AP: Evan Vucci )

Despite his anti-Muslim rhetoric at home, allies were heartened by his tough talk on Iran.

Even his decision to recognise Jerusalem as Israel's capital provoked little angst among most Arab leaders.

US military power is still capable of vanquishing any rival in the region, and the US remains a key investor and trading partner. For many leaders in the region, a visit to the White House is still more important than a visit to the Kremlin or Zhongnanhai.

'Stretched and fatigued'

US President Donald Trump with Palestinian President Mahmoud Abbas. ( Reuters: Carlos Barria )

But the trajectory of American dominance in the Middle East remains the same.

The US military still wins its battles but has become stretched and fatigued.

US diplomacy — save one or two exceptions — scores fewer successes.

The decline of American dominance in the Middle East may be very gradual, but it is a decline nevertheless.

The US will not disappear from the region as it still has interests to defend. But the stiff prop it once provided to the old system of rule will continue to bow, especially as the weight on it increases.

Anthony Bubalo is a principal at Nous Group and non-resident fellow at the Lowy Institute. This is an edited extract from Remaking the Middle East — How a troubled region may save itself, a Lowy Institute Paper published by Penguin Books Australia.