Martin Chulov reports on the elusive Iranian with so much Iraqi influence that Baghdadis believe he is controlling the country

There's a story that the new CIA director, David Petraeus, likes to tell which harks back to his days as a four-star general in Iraq.

Early in 2008, during a series of battles between the US and Iraqi army on one side and the Shia militias on the other, Petraeus was handed a phone with a text message from the Iranian general who had by then become his nemesis.

The message came from the head of Iran's elite al-Quds Force, Qassem Suleimani, and was conveyed by a senior Iraqi leader. It read: "General Petraeus, you should know that I, Qassem Suleimani, control the policy for Iran with respect to Iraq, Lebanon, Gaza, and Afghanistan. And indeed, the ambassador in Baghdad is a Quds Force member. The individual who's going to replace him is a Quds Force member."

Petraeus hardly needed to be told. Much of the US military's work with Iraq's Shia Muslims had been undermined by Suleimani and the client militias of the Iranian general's al-Quds force. So too had US government diplomatic efforts elsewhere in the Middle East, especially in Lebanon.

Petraeus last year told a thinktank, the Institute for the Study of War, about the problem Suleimani created for him: "Now, that makes diplomacy difficult if you think that you're going to do the traditional means of diplomacy by dealing with another country's ministry of foreign affairs because in this case, it is not the ministry. It is a security apparatus."

As he prepared for the job of the US's most senior spy, Petraeus would surely have been preparing for further shadow boxing. Suleimani's reputation as the most formidable operator in the region has not diminished in the past three years. By some measures it has actually increased: Syria now also comes within Suleimani's sphere of influence.

Qassem Suleimani

The strength of the ties between Suleimani and Iraqi legislators has been revealed during weeks of interviews with key officials, including those who admire him and those who fear the man like no other.

Iraq's former state security minister, Sharwan al-Waeli is one who knows Suleimani well. A formal conversation between the Guardian and al-Waeli last year took on a very different tone as soon as Suleimani's name was mentioned.

The Shia legislator was a known ally of Iran, so much so that he was seen by secularists and Sunnis in parliament as someone prepared to do Iran's bidding. He denied Iran played a pervasive role in Iraq until he was interrupted with a question that Iraqi officials have long prefered to ignore: when was the last time Qassem Suleimani came to the Green Zone, the fortified government district in the heart of Baghdad?

Al-Waeli's left hand trembled slightly and his brow furrowed. "You mean Sayed Qassem Suleimani," he said, giving Suleimani an Arabic honorific reserved for the most esteemed of men. He refused to elaborate.

In Baghdad, no other name invokes the same sort of reaction among the nation's power base – discomfort, uncertainty and fear.

"He is the most powerful man in Iraq without question," Iraq's former national security minister, Mowaffak al-Rubaie, told the newspaper al-Sharq al-Awsat in July 2010. "Nothing gets done without him."

Until now, however, few Iraqis have dared to talk openly about the enigmatic Iranian general, what role he plays in Iraq and how he shapes key agendas like no one else.

"They are too busy dealing with the aftermath," said a senior US official. "He dictates terms then makes things happen and the Iraqis are left managing a situation that they had no input into."

Suleimani's journey to supremacy in Iraq is rooted in the Islamic revolution of 1979, which ousted the Shah and recast Iran as a fundamentalist Shia Islamic state. He rose steadily through the ranks of the Iranian military until 2002 when, months before the US invasion of Iraq, he was appointed to command the most elite unit of the Iranian military – the al-Quds force of the Revolutionary Guards Corp.

The al-Quds force has no equal in Iran. Its stated primary task is to protect the revolution. However, its mandate has also been interpreted as exporting the revolution's goals to other parts of the Islamic world.

Shia communities throughout the region have proved fertile grounds for revolutionary messages and have formed deep and abiding partnerships with the al-Quds force. So too have several Sunni groups opposed to Israel – first among them Hamas in Gaza.

But Iraq has been Suleimani's key arena. The last eight years have witnessed a proxy war between Suleimani's Quds force and the US military, the full effects of which are still being played out, as the US prepares for a full departure from Iraq and Iraq's leaders ponder over whether to ask them to stay.

Arabian heartland

At stake is no less than who gets to shape the destiny of the heartland of Arabia. "His power comes straight from (the country's lead cleric Ayatollah) Khamenei," said one of Iraq's three deputy prime ministers, Saleh al-Mutlaq, a secular Sunni. "It bypasses everyone else, including Ahmadinejad.

"There is a saying in Islam that you should never get angry with your father or mother. The [Shia] interpret that as meaning what (Khamanei, via Suleimani) says has to be respected by every [Shia] inside, or outside Iran.

"All of the important people in Iraq go to see him," said Mutlaq. "People are mesmerised by him – they see him like an angel."

A second MP – a senior member of Prime Minister Nour al-Maliki's inner circle who regularly meets Suleimani in Iran – said the general has only travelled once to Iraq in the past eight years. He described him as "softly spoken and reasonable, very polite". "He is simple when you talk to him. You would not know how powerful he is without knowing his background. His power is absolute and no one can challenge this."

Silver-haired, slight and with a perennial serene smile, Suleimani comes across as the most unlikely of warlords. Those who met him during the one time he traveled to Baghdad at the height of the 2006 sectarian conflict say he walked around the compounds of his two key hosts without bodyguards. The Americans did not know he had been in the capital until he was back in Iran and were deeply unhappy to learn that their arch enemy had been among them.

"He is indeed like Keyser Söze," said a senior US official this week – in reference to the legendary villain in the The Usual Suspects, whose ruthlessness and influence terrified everyone. "Nobody knew who he was and this guy's the same. He is everywhere, but nowhere."

The senior Shia MP added: "He has managed to form links with every single Shia group, on every level. Last year, in the meeting in Damascus that formed the current Iraqi government, he was present at the meeting along with leaders from Syria, Turkey, Iran and Hezbollah. "He forced them all to change their mind and anoint Maliki as leader for a second term."

Over the five years that Maliki has been in power in Iraq, all his key advisers have been granted court in Iran by Suleimani. Iraq's president, a Kurd – Jalal Talabani, has also regularly met the general, sometimes along the border separating both countries.

The Syrian uprising has added a new dimension. The al-Quds Force has been involved in suppressing the Syrian uprising, according to multiple sources inside and outside the country.

The US has slapped personal sanctions on Suleimani and two other generals in the Iranian security forces who it accuses of helping orchestrate the crackdown that is believed to have killed more than 1,600 civilians."

Tehran has heavily invested in the survival of embattled Syrian president Bashar al-Assad, whose ruling Allawite clan has links to Shia Islam. Assad's fall would be a serious strategic setback for Iran and Suleimani. It is perhaps the only part of the region where the general's preferred mix of strategic diplomacy with aggressive operations is being strongly tested.

In the meantime, the work of the al-Quds force continues in Iraq. All but two of the US troops killed in June – the highest number in more than two years, were killed by client militias directly under Suleimani's control, the Keta'ib Hezbollah and the Promised Day Brigades.

"It is clear that the al-Quds force is responsible," said the director general of the intelligence division in Iraq's interior ministry, Hussein Kamal. "There has been a systematic flow of weapons into Iraq for the past eight years. Of course they try to say it is not state-sponsored. But when weapons are flowing from the borders of a sovereign state, it is very clear where the blame lies.

"They are destructive weapons and they cannot deny the responsibility for them."

Another Shia MP said he had personally asked Suleimani why his al-Quds force continued to smuggle weapons, many of which are fired into the Green Zone, where he and most of Maliki's inner circle live. "He just smiled and said it is nothing to do with me," the MP said. "He said he had no idea where the weapons were coming from."

Suleimani has been variously described by those who dislike him – Iraq's Sunnis, and those who have spent years trying to get his measure – as a "talented extortionist" and a highly skilled wheeler-dealer.

US officials who have spent years trying to disrupt the work of his loyalists say they would like to meet him, while at the same time being puzzled as to his objectives.

"I would simply ask him what he wants from us," said a senior US military official. In addition to the soldiers killed this year, the US ambassador in Baghdad, James Jeffrey, said last summer that Iranian proxies accounted for roughly a quarter of US combat casualties in Iraq – around 1,100 deaths and many thousands more injuries.

Despite this, the US has landed few public blows on Suleimani's close circle.

In March 2007, the British SAS captured a senior Hezbollah official, Ali Moussa Daqduq, who had allegedly planned an operation that killed seven soldiers in Karbala. The same year, US troops also captured two men in the Kurdish north who they believed were al-Quds leaders. Apart from that, the trophy cabinet remains bare – at least publicly. More troubling than the apparent dearth of tactical victories is how the rest of the year will play out.

The US – and some key neighbouring Sunni states – believe Iran's strategy in Iraq as the conflict winds down is to keep the country in a permanent but manageable state of chaos.

"They keep it on simmer and turn it up and down when they want to," said one Lebanese official in Beirut.

The senior US military spokesman in Iraq, Major General Jeffrey Buchanan agreed. "Their overall strategy has been to keep [Iraq] isolated from the rest of its neighbours and from the US, because that makes it likely that it will depend on Iran. They want Iraq to play a subordinate, weak role."

Only Iraq's lawmakers can stop the master-client relationship from becoming entrenched here. It's a task that Kurdish legislator in the national parliament, Mahmoud Othman, fears may prove to be beyond his colleagues.

"Qassem Suleimani is the key man to every decision taken in Iraq," he said.

"It is a shame to have such a man playing such a role in this country. There should be a relationship between equals like normal relations with normal states."