Overt political debate in the Middle East is hostile to the American occupation of Iraq and dominated by calls for it to end sooner rather than later. No less a figure than King Abdullah of Saudi Arabia, arguably the United States' closest Arab ally, has declared the occupation of Iraq "illegal" and "illegitimate". Real intentions, however, are different. States and local political groups might not admit it - because of public opinion - but they do not want to see the back of the Americans. Not yet.

For this there is a simple reason: while the US can no longer successfully manipulate regional actors to carry out its plans, regional actors have learned to use the US presence to promote their own objectives. Quietly and against the deeply held wishes of their populations, they have managed to keep the Americans engaged with the hope of some elusive victory.

The so-called axis of moderate Arab states - comprising Saudi Arabia, Egypt and Jordan - dreads an early US withdrawal. First, because it would be widely interpreted as an American defeat, which would weaken these pro-American regimes while both energising and radicalising their populations.

Second, if the US leaves, the emergence of a Shia regime in Iraq - in itself an offensive prospect to them - would only be a matter of time. Facing Arab antipathy, this regime would be likely to look eastward and forge close ties with its Iranian co-religionists. In the view of most Arabs, this would present a formidable challenge, setting in motion a series of dangerous events - an Iranian-Iraqi alliance; political and material support from Arab countries being offered to disgruntled Iraqi Sunni groups; retaliation by Iraqi forces; and the threat of broader regional involvement.

Third, a US departure risks triggering Iraq's partition. As some Arabs see it, the occupation is what holds the country together. So long as coalition forces are deployed, a full-blown breakup can be avoided.

In contrast, with the Americans gone, the odds of partition would increase dramatically, presenting a threat to the integrity and security of regional states. Exacerbating dormant, and in some cases not so dormant, secessionist tendencies would be one concern. Perhaps more worrying would be the ensuing challenge to the legitimacy of the fundamental tenets of nationhood, state, and national borders.

Paradoxically, the competing axis of so-called rogue states made up of Syria and Iran also wants the US to stay. So long as America remains mired in Iraq's quicksand, they think, it will be difficult for it to embark on a similar adventure nearby. This is true not only politically - the quagmire standing as a stark reminder of the invasion's failure - but also militarily: US capabilities will remain stretched for as long as the occupation continues.

Moreover, American forces in Iraq present relatively soft targets for retaliation in case Iran or Syria is attacked. In short, whether or not Syria and Iran are correct in their calculations, the occupation of Iraq is seen as the most effective insurance policy against a possible US attack against them.

For Turkey, America's presence ensures that the national aspirations of Iraq's Kurds will not metamorphose into a fully fledged independent state, a strict red line for Ankara, which has its own irredentist Kurdish problem. By containing Kurdish ambitions, the US diminishes the probability of a costly and uncertain Turkish military campaign to thwart them. Nor is Turkey attracted to the prospect of an Iraqi Shia state allied to Iran and tolerant of Kurdish aspirations - an outcome it hopes the occupation will make less likely.

For Israel too, an American withdrawal could spell disaster. Already, nothing has dented Israeli deterrence more than America's performance in Iraq - an inspiration to Israel's Arab foes that even the mightiest can be brought to heel. An early withdrawal, coming in the wake of last summer's Lebanon war, could put Israel in a dangerous position, handing a victory to Iran - the latest putative threat to Israel's existence - and providing a boost to Syria which may be considering military options to recover the Golan Heights.

There are risks for the smaller Gulf states too. With their large Shia communities and heavy dependence on American protection, they would be threatened by an early US departure from Iraq. In Bahrain, home to an unhappy Shia majority, the fallout could be imminent.

Inside Iraq, this is a period of consolidation for most political groups. They are building up their political and military capabilities, cultivating and forging alliances, clarifying political objectives and preparing for impending challenges. It is not the moment for all-out confrontation. No group has the confidence or capacity decisively to confront rivals within its own community or across communal lines. Equally, no party is genuinely interested in a serious process of national reconciliation when they feel they can improve their position later on. A continued American presence is consistent with both concerns - it can keep clashes manageable and be used to postpone the need for serious political engagement.

Shias in government would like the US to stay long enough for them to tighten their grip on the levers of state power and build a loyal military. Those Shias who are not in power would like them to stay long enough to avoid a premature showdown with their rivals. Militant Shia groups can simultaneously blame the occupation forces for their community's plight and attack them to mobilise further support. Pro-Iranian Shias, meanwhile, retaliate against anti-Iranian US moves with attacks on Americans in Iraq.

Al-Qaida and its affiliates arguably benefit most from the occupation. They established themselves, brought in recruits, sustained operations against the Americans and expanded. The last thing they want is for the Americans to leave and deny them targets and motivation for new members. Other Sunni armed groups need the Americans for similar reasons and for protection against Shias. For Sunni politicians, the occupation prevents a total Shia takeover of state institutions and helps increase their influence.

Of all ethnic groups, the Kurds have made best use of the Americans. Protected by the US from their powerful and ruthless historical foes, Arab and Turk, they have built quasi-independent institutions and prospered amid relative security. They have no reason to want this situation to end.

In common with neighbouring states, Iraqi Shias, Sunnis and Kurds are united in being able to use the Americans' presence to pursue separate and often conflicting political agendas. The grand disconnect in the region is between the political sentiments of ordinary people, which are overwhelmingly for an end to occupation, and the political calculations of leaders, which emphasise the benefits of using the Americans and consequently of extending their stay - at least for the time being.

In this grim picture, the Americans appear the least sure and most confused. With unattainable objectives, wobbly plans, changing tactics, shifting alliances and ever-increasing casualties, it is not clear any longer what they want or how they are going to achieve it. By setting themselves up to be manipulated, they give credence to an old Arab saying: the magic has taken over the magician.

· Hussein Agha is a senior associate member of St Antony's College, Oxford

comment@theguardian.com