Iran has brokered a critical deal with its regional neighbours that could see a pro-Tehran government installed in Iraq, a move that would shift the fragile country sharply away from a sphere of western influence.

The Guardian can reveal that the Islamic republic was instrumental in forming an alliance between Iraq's Nouri al-Maliki, who is vying for a second term as prime minister, and the country's powerful radical Shia cleric leader, Moqtada al-Sadr.

The deal – which involved Syria, Lebanon's Hezbollah and the highest authorities in Shia Islam – positions Maliki as a frontrunner to return as leader despite a seven-month stalemate between Iraq's feuding political blocs.

It also positions Iran as a potent buffer to US interests at a time when America is looking to change its relationship with Iraq from military overlords to civilian partners.

Senior officials in Iraq have given the Guardian details of the behind-the-scenes Iranian campaign which began in earnest in early September.

At the time the US had only just withdrawn its last dedicated combat units from Iraq but left behind a political vacuum with no government in place after March elections delivered a seemingly irrevocably split parliament.

According to sources the Iranians saw their opportunity.

"The Iranians were holding out until then," said a key source about the timing of the Iranian move. "They were not going to give the Americans the satisfaction of leaving on a good note."

Within days of the withdrawal, Sadr, who lives in self-imposed exile in the Iranian city of Qom, was told by the Iranians to reconsider his position as a vehement opponent of Maliki. Sadr's party in Iraq had won more than 10% of the 325 seats in play at the election making him a powerbroker in the formation of any new government.

The push initially came from the spiritual head of the Sadrist movement, Ayatollah Kazem al-Haeri, who has been a godfather figure to the firebrand cleric for the past 15 years.

"He couldn't say no to him," said the official. "Then the Iranians themselves got involved."

Days after the Iranian move, an Iraqi push followed. Throughout September Maliki sent his chief of staff to Qom along with a key leader in his Dawa party, Abdul Halim al-Zuhairi. They were, according to the Guardian's source, joined by a senior figure in Lebanese Hezbollah's politburo, Mohamed Kawtharani, as well as arch-US foe General Qassem Suleimani, the head of the Iranian Revolutionary Guards' Al-Quds Brigades, whose forces the US military blames for causing more than one quarter of its combat casualties in Iraq throughout almost eight years of war.

In the following three weeks, Iranian president, Mahmoud Ahmadinejad, met Bashar al-Assad at Damascus airport on his way to deliver a speech at the United Nations in New York.

The two-hour meeting was pivotal in changing Assad's view of Maliki. Both presidents had not spoken for 15 months and had withdrawn their respective ambassadors after Maliki accused Assad of harbouring terrorists who destroyed four ministries in Baghdad in a devastating bombing campaign. In return, Assad visited Tehran the day after the Sadrist support for Maliki was announced. Two other Shia Islamic spiritual leaders, Ayatollah Ali Khamenei and Lebanese Hezbollah leader, Hassan Nasrallah, are also believed to have endorsed the Sadrist move.

It is understood that the full withdrawal of all US troops after a security agreement signed between Baghdad and Washington at the end of 2011 was also sought by Sheikh Nasrallah.

"Maliki told them he will never extend, or renew [any bases] or give any facilities to the Americans or British after the end of next year," a source said.

The shape of the future security relationship between both countries is yet to be negotiated and the US is widely believed to be hoping to retain at least one military base in Iraq that it could use as a strategic asset in the region.

US officials have strongly suggested they would scale back their involvement in Iraq if the Sadrists, who have been a key foe throughout the years of war, were to emerge as a significant player in any government.

The revelations come amid sharp criticism of the US diplomatic role in Iraq since the election. The US at first heavily backed Maliki, then changed tack during the summer to demand a powersharing government that empowered rival secular candidate, Iyad Allawi, whose Iraqiya bloc won more votes than Maliki's bloc.

"American policy inside Iraq has facilitated this Iranian takeover," said Allawi's deputy, Osama al-Najaifi. "They are now pulling out of Iraq and it appears their behaviour early in the summer was almost to appease Iran. This will create a disaster in the region, not just for Iraq, but for their interests as well. We have gone from being under US occupation to Iranian occupation."

A senior Obama administration official said: "I would say Iraq is a sovereign government and we are not party to such discussions. With reference to the degree that Iraq's neighbours seek to play a constructive role, that is something we welcome. I emphasise 'constructive'. It is not about interaction with Iraq that matters but the quality of that interaction. If it is destructive, we condemn that."

On the 2011 December withdrawal date, the official said: "Any follow-up engagement with Iraq in relation to troops would be at the request of the government of Iraq. There are no plans to keep troops after December 2011. We are drawing down and all will be out of Iraq."

Although that is the official US line, unofficially Washington expected to retain a force in Iraq after December 2011, as well as bases to protect oil interests, to buttress the Iraqi government in the event of a destabilising uprising and to help contain Iran.

Maliki will arrive in Tehran today for the talks with Ahmedinejad. He visited Syria late last week for a detente with Assad.

• This article was amended on 25 October 2010. The original referred to Ali Khamenei as a grand ayatollah. This has been corrected.