PRESIDENT George W Bush’s bold decision to order a “surge” of some 20,000 American troops for Iraq has brought the debate over the war to a defining stage. There will not be opportunity for another reassessment.

The Baker-Hamilton commission has powerfully described the impasse on the ground. It is the result of cumulative choices —some of them enumerated by the president —in which worthy objectives and fundamental American values clashed with regional and cultural realities.

The important goal of modernising US armed forces led to inadequate troop levels for the military occupation of Iraq. The reliance on early elections as the key to political evolution, in a country lacking a sense of national identity, caused the newly enfranchised to vote almost exclusively for sectarian parties, deepening historic divisions into chasms. The understandable —but, in retrospect, premature —strategy of replacing American with indigenous forces deflected US forces from a military mission; nor could it deal with the most flagrant shortcoming of Iraqi forces, which is to define what the Iraqi forces are supposed to fight for and under what banner.

These circumstances have merged into an almost perfect storm of mutually reinforcing crises: Within Iraq, the sectarian militias are engaged in civil war or so close to it as to make little practical difference. The conflict between Shias and Sunnis goes back 1,400 years. In most Middle Eastern countries, Shia minorities coexist precariously with Sunni majorities. The civil war in Iraq threatens to usher in a cycle of domestic upheavals and a war between Shia and Sunni states, with a high potential of drawing in countries from outside the region. In addition, the Kurds of Iraq seek full autonomy from both Sunnis and Shias; their independence would raise the prospect of intervention from Turkey and possibly Iran.

The war in Iraq is part of another war that cuts across the Shia-Sunni issue: the assault on the international order conducted by radical groups in both Islamic sects. Functioning as states within the states and by brutal demonstrations of the inability of established governments to protect their populations, such organisations as Hezbollah in Lebanon, the Mahdi army in Iraq and the Al Qaeda groups all over the Middle East seek to reassert an Islamic identity submerged, in their view, by Western secular institutions and values. Any enhancement of radical Islamist self-confidence therefore threatens all the traditional states of the region, as well as others with significant Islamic populations, from Indonesia through India to Western Europe. The most important target is the United States, as the most powerful country of the West and the indispensable component of any attempt to build a new world order.

The disenchantment of the American public with the burdens it has borne alone for nearly four years has generated growing demands for some form of unilateral withdrawal, usually expressed in the form of benchmarks to be put to the Baghdad government which, if not fulfilled in specific time periods, would trigger American disengagement.

But under present conditions, withdrawal is not an option. American forces are indispensable. They are in Iraq not as a favour to its government or as a reward for its conduct. They are there as an expression of the American national interest to prevent the Iranian combination of imperialism and fundamentalist ideology from dominating a region on which the energy supplies of the industrial democracies depend.

An abrupt American departure will greatly complicate efforts to help stem the terrorist tide far beyond Iraq; fragile governments from Lebanon to the Gulf will be tempted into pre-emptive concessions. It might drive the sectarian conflict within Iraq to genocidal dimensions beyond levels that impelled American intervention in the Balkans. Graduated withdrawal would not ease these dangers until a different strategy is in place and shows some progress. For now, it would be treated both within Iraq and in the region as the forerunner of a total withdrawal, and all parties would make their dispositions on that basis.

President Bush’s decision should therefore not be debated in terms of the “stay the course” strategy he has repeatedly disavowed in recent days. Rather it should be seen as the first step towards a new grand strategy relating power to diplomacy for the entire region, ideally on a nonpartisan basis.

The purpose of the new strategy should be to demonstrate that the US is determined to remain relevant to the outcome in the region; to adjust American military deployments and numbers to emerging realities; and to provide the maneuvering room for a major diplomatic effort to stabilise the region.

Of the current security threats in Iraq —the intervention of outside countries, the presence of Al Qaeda fighters, an extraordinarily large criminal element, the sectarian conflict —the United States has a national interest in defeating the first two; it must not involve itself in the sectarian conflict for any extended period, much less let itself be used by one side for its own sectarian goals.

The sectarian conflict confines the Iraqi government’s unchallenged writ to the sector of Baghdad defined as the Green Zone protected by American forces. In many areas the militias exceed the strength of the Iraqi national army. Appeals to the Iraqi government to undertake reconciliation and economic reforms are not implemented, partly because the will to do so is absent but essentially because it lacks the power to put such policies in place, even if the will to do so could suddenly be mobilised. If the influence of the militias can be eliminated —or greatly reduced —the Baghdad government would have a better opportunity to pursue a national policy.

The new strategy has begun with attempts to clear the insurrectional Sunni parts of Baghdad. But it must not turn into ethnic cleansing or the emergence of another tyrannical state, only with a different sectarian allegiance. Side by side with disarming the Sunni militias and death squads, the Baghdad government must show comparable willingness to disarm Shia militias and death squads. American policy should not deviate from the goal of a civil state, whose political process is available to all citizens.

As the comprehensive strategy evolves, a repositioning of American forces from the cities into enclaves should be undertaken so that they can separate themselves from the civil war and concentrate on the threats described above. The principal mission would be to protect the borders against infiltration, to prevent the establishment of terrorist training areas or Taleban-type control over significant regions. At that point, too, significant reductions of American forces should be possible. Such a strategy would make withdrawals depend on conditions on the ground instead of the other way around. It could also provide the time to elaborate a cooperative diplomacy for rebuilding the region, including progress towards a settlement of the Palestine issue.

For such a strategy, it is not possible to jettison the military instrument and rely, as some argue, on purely political means. A free-standing diplomacy is an ancient American illusion. History offers few examples of it. The attempt to separate diplomacy and power results in power lacking direction and diplomacy being deprived of incentives.

Diplomacy is the attempt to persuade another party to pursue a course compatible with a society’s strategic interests. Obviously this involves the ability to create a calculus that impels or rewards the desired direction. The outcome, by definition, is rarely the ability to impose one’s will but a compromise that gives each party a stake in maintaining it.

Few diplomatic challenges are as complex as that surrounding Iraq. Diplomacy must mediate between Iraqi sects which, though in many respects mortal enemies, are assembled in a common governmental structure. It needs to relate that process to an international concept involving both Iraq’s neighbours and countries further away that have a significant interest in the outcome.

Two levels of diplomatic effort are necessary:

(1) The creation of a contact group, assembling neighbouring countries whose interests are directly affected and which rely on American support. This group should include Turkey, Saudi Arabia, Egypt and Jordan. Its function should be to advise on ending the internal conflict and to create a united front against outside domination.

(2) Parallel negotiations should be conducted with Syria and Iran, which now appear as adversaries, to give them an opportunity to participate in a peaceful regional order. Both categories of consultations should lead to an international conference including all countries that will have to play a stabilising role in the eventual outcome, specifically the permanent members of the UN Security Council as well as such countries as Indonesia, India and Pakistan.

Too much of the current discussion focuses on the procedural aspect of starting a dialogue with adversaries. In fact, a balance of risks and opportunities needs to be created so that Iran is obliged to choose between a significant but not dominant role or riding the crest of Shia fundamentalism. In the latter case, it must pay a serious, not a rhetorical, price for choosing the militant option. An outcome in which Iran is approaching nuclear status, because of hesitant and timid non-proliferation policies in the UN Security Council, coupled with a political vacuum in front of it in the region must lead to catastrophic consequences.

Similar principles apply to the prospects for settlement in Palestine. Moderates in both the Arab countries neighbouring Israel and in Israel are evolving compromises unimaginable a decade ago. But if the necessary outcomes are perceived as the result of panic by moderates and an exit from the region by the United States, radicals could raise unfulfillable demands and turn the peace process against the moderates.

In all this, the United States cannot indefinitely bear alone the burden for both the military outcome and the political structure. At some point, Iraq has to be restored to the international community, and other countries must be prepared to share responsibilities for regional peace. Some of America’s allies and other affected countries seek to escape the upheavals all around them by disassociating from the United States. But just as it is impossible for America to deal with these trends unilaterally, sooner or later a common effort to rebuild the international order will be imposed on all the potential targets. The time has come for an effort to define the shoals within which diplomacy is obliged to navigate and to anchor any outcome in some broader understanding that accommodates the interests of the affected parties.

Henry A Kissinger, a former US secretary of state, is considered the architect of US foreign policy during the Cold War