The high point of George W Bush's "war on terror" came six weeks after the start of the United States-led assault on the Saddam Hussein regime in March 2003, when the then US president made his “mission accomplished” speech from the flight-deck of the USS Abraham Lincoln off the Californian coast. Everything was going well: the response to 9/11 had been appropriate, the Taliban had been defeated, Iraq was in American hands, and al-Qaida was dispersed. The "new American century" was back on track.

Only six months later, however, the US was fighting a bitter urban insurgency in Iraq and had signally failed in its attempt to widen the coalition. A request to India to provide an army division was refused by the BJP nationalist government in New Delhi; prime minister Atul Bihari Vajpayee was in favour but public opinion deeply opposed. With state elections coming later that year and a general election in 2004, the idea was a non-starter (see "Far from home, alone", 16 July 2003).

Washington and its dwindling band of coalition partners were experiencing rising casualties, amid a lack of troops with the right training and equipment. In the circumstances, it seemed absolutely reasonable for the Pentagon to turn to the one ally with the experience and arms industry that could help out: Israel. By the end of 2003 there had been intense discussions between senior officers in the US army's Training and Doctrine Command (Tradoc) and school of infantry, and their counterparts in the Israeli Defence Forces (see "After Saddam, no respite", 19 December 2003).

These different agencies already had close links, but these meetings produced much greater Israeli cooperation in training United States military personnel, the provision of specialist equipment from Israeli arms companies, and even the construction by the US army corps of engineers of a mock Arab town, Baladia, in the Negev desert to train the military in urban warfare (see "A tale of two towns", 21 June 2007).

George W Bush's administration saw all this as a wise response to an unexpected predicament. But for jihadist propagandists it was a heavenly gift. The United States was now occupying a Muslim state in the heart of the Arab world - and more than this, Iraq, whose capital city Baghdad had been the seat of the the Abbasid caliphate, the most renowned example of Islamic governance. The new involvement of the Israelis offered a perfect opportunity to depict the entire war as a Crusader-Zionist conspiracy against Islam itself.

This, together with the neglect of Afghanistan which allowed a Taliban resurgence, were two of the most extreme examples of unforeseen consequences in the decade after 9/11. The al-Qaida movement may have been dispersed - though there continued to be numerous attacks across the world in the post-9/11 years - but the Iraq occupation gave it a new lease of life (see "Al-Qaida, a multiform idea", 8 August 2013).

A decade on, the dynamics are changing once more. This is another period where events may be combining to aid an al-Qaida movement that has morphed from a narrowly hierarchical and centralised paramilitary movement into more of a potent, decentred idea.

A reviving movement

Even two years ago, in mid-2011, events seemed to be moving against al-Qaida. Osama bin Laden had been killed, armed-drone attacks in north-west Pakistan were decimating the middle ranks of the leadership, and the incipient Arab awakening showed that popular non-violent movements could unseat autocratic regimes (see "Al-Qaida and the Arab spring: after bin Laden", 6 May 2011). The latter in particular was a huge problem for al-Qaida, since a peaceful transition to democracy in an Islamic state was the opposite of its aims in terms both of process and result.

Yet to the surprise of many analysts, the al-Qaida idea managed to retain its potency and continue to attract adherents across the middle east and south Asia (see "Al-Qaida: an enduring insurgency", 1 September 2011). Pakistan was no longer the clear centre of its activity; rather, loose affiliates in a swathe of countries and regions (Yemen, Somalia, the Caucasus, Nigeria, Niger and the Maghreb) were all surviving and some thriving, while other groups in Iraq and (especially) Syria were clearly on the rise.

This phase may still represent more of a series of discrete problems than a generic phenomenon with worldwide implications. By contrast, the impact of political developments in the Arab world on the Arab awakening - and in particular the latter's evolving economic context - offers very promising conditions for al-Qaida's regeneration.

Egypt is at the heart of this prospect. Many western governments are quietly relieved that the Muslim Brotherhood has been ousted by Egypt's army (even if they are reluctant to admit it in public, and are anxious to be seen criticising the excess use of force). Other states, including that strange combination of Israel and Saudi Arabia, are quite open in their satisfaction at the military takeover. At the same time, powerful western states also give oblique support to the mix of suppression and concession practiced domestically in Saudi Arabia and Oman, as well as the more overt repression in Bahrain.

In the region as a whole, the Arab awakening has so far achieved little in the way of fairer representation. A partial exception is the slow and still uncertain path to democracy being pursued in Tunisia, and some modest reforms in Morocco. More generally, the lesson for many from the coup in Egypt and the war in Syria is that non-violent change - even if it makes short-term gains - can easily be undermined by powerful old regimes and their foreign friends. The propagandists' message will be that extreme Islamist rule gained by able jihadist paramilitaries may now be evolving in Syria and may be the right way forward in Egypt.

A returning moment

This could in principle be a seductive message at any time. In this decade, an important but neglected factor will give it further boost: namely, the intensifying economic problems that underpin dissent across the region. In the wake of the financial crisis of 2007-08, and resulting price increases, there were food riots and other protests - in retrospect, a premonition of the Arab awakening.

An impressive new analysis by George Joffé emphasises this element (see “The Future of the Middle East and North Africa” Norwegian Peacebuilding Resource Centre [Noref], 20 August 2013). Joffe points out that the long-term effect of the neo-liberal Washington consensus has been to increase marginalisation in many middle-east economies, leaving so many millions of educated young people feeling frustrated and angry. He writes:

“As has proved to be the case with the euro zone crisis, Western governments seem incapable of escaping from the ideological constraints of neoliberal economic theory, despite the fact that it has proved to be a failure, in the developing world, at least, ever since it was introduced in the 1980s, in the wake of the debt crisis then. The outcome, therefore, is ineluctable: within five years, even worse unrest predicated on economic desperation throughout the MENA region is inevitable.”

The Arab awakening at least provided some promise of representative governance that would answer to the needs and demands of marginalised majorities (see "Egypt, and the world's revolt", 4 July 2013). It's now even more evident, as several columns in this series argued at the time, that is going to be a slow and difficult process. The pervasive and entrenched economic policies that are at the root of bitter social divisions will make it difficult for any elected government across the region to meet widepread expectations of a better life. That is a key aspect of the current reality.

A combination of more representative governance and radical economic transformation might just do the job. There is little sign, though, of any government or intergovernmental body recognising that. What is left is an Arab awakening that is at least faltering, and is now in addition having to cope with a deeply flawed economic ideology.

For radical Islamists, this is a singularly positive environment in which to operate. Indeed, the al-Qaida idea may have far more potential in the coming years than has yet been recognised.