BAGHDAD (Reuters) - Nuri al-Maliki finally bowed to pressure within Iraq and beyond on Thursday and stepped down as prime minister, paving the way for a new coalition that world and regional powers hope can quash a Sunni Islamist insurgency that threatens Baghdad.

Maliki ended eight years of often divisive, sectarian rule and endorsed fellow Shi’ite Haider al-Abadi in a televised speech during which he stood next to his successor and spoke of the grave threat from Sunni Islamic State militants who have taken over large areas of northern Iraq.

“I announce before you today, to ease the movement of the political process and the formation of the new government, the withdrawal of my candidacy in favour of brother Dr. Haider al-Abadi,” Maliki said.

Maliki’s decision was likely to please Iraq’s Sunni minority, which dominated Iraq under Saddam Hussein’s iron rule but was sidelined by Maliki, a relative unknown when he came to power in 2006 with U.S. backing.

Maliki had resisted months of pressure to step down from Sunnis, Kurds, some fellow Shi’ites, Shi’ite regional power Iran and the United States. He had insisted on his right to form a new government based on the results of a parliamentary election in late April.

His stubborn insistence stirred concerns of a violent power struggle in Baghdad. But in recent days, as his support was obviously crumbling, he told his military commanders to stay out of politics.

“From the beginning I ruled out the option of using force, because I do not believe in this choice, which would without a doubt return Iraq to the ages of dictatorship, oppression and tyranny, except to confront terrorism and terrorists and those violating the will and interests of the people,” Maliki said.

On Wednesday, his own Dawa political party publicly threw its support behind Abadi and asked lawmakers to work with him to form a new government. And Iranian Supreme Leader Ayatollah Ali Khamenei, offered his personal endorsement to Abadi, distancing himself from Maliki.

U.S. National Security Advisor Susan Rice commended Maliki for his decision to support Abadi, and she noted a wide range of leaders from across the Iraqi political spectrum had committed to help Abadi form a broad, inclusive government.

“These are encouraging developments that we hope can set Iraq on a new path and unite its people against the threat presented by the Islamic State of Iraq and the Levant,” Rice said in a statement.

U.S. Secretary of State John Kerry described Maliki’s decision as “important and honorable” and said “the United States stands ready to partner with a new and inclusive government to counter this threat” from the Islamic State.

A U.S. official said that once administration officials concluded Maliki had to go, Washington pushed Iraqi politicians to take steps such as ratifying the election results and designating a prime minister but added it had not advocated specific candidates. “It was all teeth-grinding activity,” said the official on condition of anonymity. “While we were pushing the process, they were determining who was going to be in the driver’s seat.

“In the end, it was the weight of the system and the weight of the history that came down, and Maliki just lost all of his support,” he added. The official also said a clear shift last week against Maliki by Iraq’s most influential cleric, Grand Ayatollah Ali al-Sistani, “was a big, big part of everybody accepting that there was no way forward with Maliki.”

Abadi is seen as a moderate Shi’ite with a decent chance of improving ties with Sunnis. But he is faced with halting the advance of the Islamic State, which has overrun large areas of Iraq.

ANBAR AGREEMENT?

Before Maliki’s announcement, a leading figure in the Sunni minority told Reuters he had been promised U.S. help to fight the Islamic State militants.

Ahmed Khalaf al-Dulaimi, the governor of the Sunni heartland province of Anbar, told Reuters his request for help, made in meetings with U.S. diplomats and a senior military officer, included air support against the militants who have a tight grip on large parts of his desert province and northwestern Iraq.

Slideshow ( 11 images )

Such a move could revive cooperation between Sunni tribes, the Shi’ite-led authorities and U.S. forces that was credited with thwarting al Qaeda in Iraq several years ago.

But the U.S. State Department played down Dulaimi’s statement.

“We’ve continued meeting with a range of officials to talk through what the needs might be - the security needs - to fight ISIL across the board,” State Department spokeswoman Marie Harf told reporters in Washington.

Asked if Dulaimi was correct that the United States had made a commitment, Harf said she had no details. “We’re having conversations about what it (any security assistance) might look like in the future, but nothing concrete beyond that,” she said.

Slideshow ( 11 images )

Dulaimi said in a telephone interview: “Our first goal is the air support. Their technology capability will offer a lot of intelligence information and monitoring of the desert and many things which we are in need of.

“No date was decided but it will be very soon and there will be a presence for the Americans in the western area.”

U.S. President Barack Obama said on Thursday that U.S. troops planning an evacuation of refugees further north were standing down as U.S. air strikes and supply drops had broken the “siege of Mount Sinjar,” where thousands of members of the Yazidi religious minority had taken refuge from the militants.

Obama said some of the U.S. personnel sent to draw up plans for the evacuation of the Yazidis would soon leave Iraq.

Disowned by al Qaeda as too radical after it took control of large parts of Syria, Islamic State capitalised on its Syrian territorial gains and sectarian tensions in Iraq to gain control of Falluja and Anbar’s capital Ramadi early this year.

Unlike Osama bin Laden’s al Qaeda, which set its sights on destroying the West, the Islamic State has territorial goals, aims to set up a caliphate and rages against the Sykes-Picot agreement of 1916 between Britain and France that split the Ottoman empire and carved borders across the Arab lands.

Seizing the capital, Baghdad, would be difficult because of the presence of special forces and thousands of Shi’ite militias who have slowed down the Islamic State elsewhere.

But a foothold just near the capital could make it easier for the IS to carry out suicide bombings, deepen sectarian tensions and destabilise Iraq.

On Thursday, Islamic State militants massed near the town of Qara Tappa, 120 km (75 miles) north of Baghdad, security sources and a local official said, in an apparent bid to broaden their front with Kurdish peshmerga fighters.

The movement around Qara Tappa suggests they are becoming more confident and seeking to grab more territory closer to the capital after stalling in that region.

“The Islamic State is massing its militants near Qara Tappa,” said one of the security sources. “It seems they are going to broaden their front with the Kurdish fighters.”