In the centre of the city, a 63-year-old man divides his time between a windowless office and a heavily guarded villa, shuttling between the two in a black armoured SUV. Here, in the walled-off green zone, the noise of traffic in the battered capital beyond the security perimeter is barely audible.

Nouri Mohammed Hasan al-Maliki has spent decades trying to change the power dynamics of his country. In recent years, he has achieved some long-term aims. This weekend, for the underground activist turned party official turned two-term prime minister of one of the most volatile countries in the world, his failures are more evident.

For there is fear in Baghdad, where Maliki has been based since returning from a 23-year exile in 2003 in the wake of the US invasion, as there is across much of Iraq. The sudden seizure of Mosul, the second biggest city, by several hundred militants from the Islamic State in Iraq and the Levant (Isis) on Tuesday now looks like the start of a campaign, not its end. On Wednesday, the Isis extremists tookcontrol of Tikrit, a provincial capital an hour's drive north of Baghdad. By Friday, there was talk of an Isis push to the edge of the capital itself.

As the United Nations Security Council met to discuss the sudden crisis, Maliki, as dour and impassive as ever, restricted himself to a short public statement blaming the collapse on a "conspiracy" whose participants would be severely punished. He failed to achieve a quorum in parliament needed to impose a state of emergency or the numbers to ask the US for air strikes and drone attacks against the advancing extremists. Few can remember Maliki ever smiling. This weekend, he has less reason than ever.

The question now is whether this sudden unravelling is Maliki's own fault? Or was it always inevitable? Maliki's story, like that of many Iraqis, is woven through the country's violent history. He was born in 1950 near Hilla, in the southern Shia-dominated heartlands of Iraq. The town is close to Kerbala, one of the most holy cities for the 15% of Muslims worldwide and 60% to 65% of Iraqis who follow that minority branch of the faith.

When he was 18, the Iraqi monarchy was overthrown in a coup led by the Ba'ath party. The new regime was secular, socialist, totalitarian, nationalist and focused on Arab identity. Maliki was drawn to the contemporary ideological alternative, Islamism, which gained momentum through the 1970s. In Iraq, the crisis came at the end of the decade, when Saddam Hussein seized power and the Iranian revolution mobilised and radicalised Shias across the world. When war came with Iran, the situation for Iraq's Shias worsened dramatically. After family members were arrested, Maliki, an organiser with the Islamist Dawa party, fled first to Iran and then to Syria.

In 2003, he was back, one of the thousands of exiles looking to rebuild their lives, and gain power and wealth, in post-Saddam Iraq. There was huge opportunity, particularly for the long-marginalised Shias. Saddam was a Sunni and his regime continued the Sunni dominance seen under the Ottoman Turks and bolstered by the British. But the invasion had turned this arrangement upside down. Now, the more numerous Shia had the upper hand.

The Sunni, driven from power and office by the invaders, were unwilling to accept their newly diminished status. These were causes and conflicts barely understood – or at least gravely underestimated – in the White House and Department of Defence. Clumsy US tactics and policies exacerbated a deteriorating situation. By spring 2006, the country was deep in a horrific civil war.

Maliki came to prominence during the darkest days of that conflict. The ill-fated US-led civilian administration had been long gone by the time the 2005 elections came round, but that did not mean US influence had waned. Washington was unhappy with the incumbent prime minister, whom it deemed ineffectual, but there were few attractive options. Maliki's name came up and US diplomats in Iraq manoeuvred parliamentarians to back him.

"I don't think it was wishful thinking. There were some good things about him and it was a pretty shallow talent pool. I think he was as surprised as anyone to find himself prime minister. He certainly didn't ask for it," said one former British diplomat working on Iraq at the time.

At first, the signs were reassuring. Maliki had not spent years in London or Washington and appeared authentically Iraqi in a way that other more polished former exiles did not. The new prime minister made some of the right noises about national unity and, surprising the US, sent ill-prepared Iraqi troops into the southern city of Basra, and then into vast enclaves in the capital itself, which were controlled by a hardline Shia militia.

The targets of the operation were bitter rivals, particularly for working-class Shia support, but the operations reassured Iraq's Sunnis nonetheless. "The perception of Maliki as a decisive leader improved across all spectra of Iraqi society," Ryan Crocker, the then US ambassador to Iraq, told British counterparts, according to a leaked diplomatic cable.

The Sunnis were coming off worst in the civil war and a fortuitously timed "surge" of additional US troops helped reduce violence. But huge problems remained: a fractured country, gutted by repressive regimes, civil strife and incompetent occupation. Iraq needed a statesman with charisma, vision, strength and the backing of great powers, to bring together Shias, Sunnis and the Kurds, who number 15% to 20%, to rebuild. Instead, there was Maliki, a man whose vision of the world had been formed by decades of covert struggle, in the brutal zero-sum world of Middle Eastern revolutionary politics.

"Maliki is a paranoid guy who spent his years in exile plotting to overthrow the regime. In 2010, after Iran assured him a second premiership, he determined to go after his rivals and to consolidate his grip on power," said Emma Sky, a senior fellow at Yale University who previously advised the US military in Iraq.

The result has been an administration tragically reminiscent of former rulers of Iraq. When Maliki moves, he is surrounded by three rings of armed men and dozens of black-windowed SUVs. His loyalists run all major security agencies, which are now greatly feared. With Iraq now the world's seventh-largest oil producer, there is much money and corruption is of an epic scale. Meanwhile, malnutrition is widespread.

This does not necessarily make Maliki unpopular, however. His rejection of demands from Washington for legal immunity for any US troops remaining in Iraq after the bulk withdrew in 2011 resonated with many. Maliki's State of Law coalition gained 92 out of 328 parliamentary seats, winning in 10 of 18 provinces, in elections last month. Iraq's curse is often said to be its oil, which frees rulers from accountability and the need to build or reform.

It may also be that everyone wants to see a "strong leader" who will bring order and security, but vote for those who look after the interests of their own particular community.

Maliki's few friends outside Iraq have pointed to the instability sown by the Arab spring to deflect criticism following the disasters of last week. That the welter of violence in Syria would overflow into neighbouring Iraq was always inevitable. After being forced out of the west of Iraq by local militias recruited among the Sunni tribes, extremists linked to al-Qaida, and then to Isis, based themselves in and around Mosul. They exploited tensions between Sunnis and Kurds, old Ba'athist networks and easy access across the desert to Syria to rebuild weakened networks. A war next door was always going to help them and, predictably, the Syrian chaos has allowed Isis to gain towns and oilfields, as well as weapons and recruits. Eastern Syria now provides a rear base for a campaign in the homeland.

That the group has found such fertile ground in Iraq itself is also Maliki's fault. From reneging on a promise to departing US officials that he would integrate the Sunni militias that had ousted al-Qaida into the Iraqi army, to targeting the most senior Sunni officials in government, to packing Shia into the security services and "special forces" units, Maliki has followed a narrow sectarian agenda.

When, in recent months, anger among the Sunni led to demonstrations, these were put down with a significant loss of life. Some of the protests were by political parties; others involved front groups for Saddam loyalists and extremist Islamists; but the hard line taken by Maliki backfired badly. When Falluja in effect rebelled, Maliki withdrew the army and had the city bombarded. In the ensuing disorder, various groups, including Ba'athist nationalists, tribes and the infamous Isis, filled the vacuum.

The result is the presence of militants less than an hour's drive from Baghdad and in Baghdad's green zone, backed by many sympathisers within the capital. One former associate of Maliki, now based in London, pointed out that few rulers of Iraq leave power peacefully or, indeed, alive. This is something, he says, of which the prime minister is acutely aware.