The time has long since past when it became fashionable to talk about a new world order. The collapse of the Soviet Union provided an opportunity to fashion one. But instead of using that opportunity to create a new security architecture in Europe, Nato expanded eastwards as the military anchor for democracy promotion. Not content to have seen off one global military competitor in the Soviet Union, the western military industrial complex and the think-tanks they funded scurried around for a worthy replacement. When 11 September happened, they thought they were in business again. For a brief moment, al-Qaida seemed to fulfil some of the characteristics of communism: it could pop up anywhere in the world; it was an existential enemy, driven ideologically and uncontainable through negotiation; and it was potentially voluminous. Neither the doctrines of the pre-emptive strike, nor attacking a foreign country abroad to ensure security at home, were new. Swap the domino theory of the Vietnam era for the crescent of crisis of the Bush and Obama eras, and you had the same formula for a foe that hopscotched across the globe.

But here's the curious thing. Al-Qaida failed, not by being bombed out of the tribal areas of Pakistan or by losing its video-hugging leader. It failed as an ideological alternative, in its own terms and for its own people. It failed in Egypt, the country that mattered most to its chief thinker, the Egyptian-born doctor Ayman al-Zawahiri. When the opportunity arose for millions of Muslims to shed their brutal Arab yoke (this was supposed to be the fourth phase in the construction of the Caliphate, to be accompanied by physical attacks against oil suppliers and cyber ones on the US economy), nothing of the sort happened. Islam is indeed winning the day, but it is political rather than military. It seeks alliances with the apostate and says it is committed to democratic partnership and the rule of law.

Al-Qaida's failure was all the more significant because the western response, the interventions in Afghanistan and Iraq, also failed. Not because the enemy was especially daunting, but because the mission was impossible to start with. Mission creep started with democracy promotion, continued as state-building, and ended with withdrawal at any cost, by the appointed date. The quality of life in the country US combat troops left behind – most likely one that in Iraq will break up into a loose federation on sectarian lines – became less important than the fact of departure itself. Military ceremonies proclaiming victory in the war in Iraq had as much sense of reality as Kim Jong-il's funeral. This is the next feature of the world we live in. It is an age of the self-defeating intervention. The quests through military means to build stable states out of a dictatorship in Iraq or a failed state in Afghanistan did not and are not failing at the hands of a conventional enemy. They implode. They self-destruct.

Military overreach and serial economic crises have bequeathed us a generation of small leaders who battle with events that outsize them. They have stopped trying to fashion them, but appeal instead to a defensive desire. Protectionism not internationalism rules the day. The Middle East has been transformed from a zone of allies to one in which Washington has been reduced to the role of spectator. It is now largely a taker of Middle Eastern policy, not one of its makers. There are other parts of the globe where US power projection finds natural allies, such as the Pacific, where China's rise is feared. So the paradox is that while US military power retains global reach (it is working on supersonic cruise missiles, and long-range drones) its stewardship as world leader, as a generator of the next big idea, is gradually ending. There may come a time when international institutions are rebuilt to fill this vacuum. But that time is not yet. Until then, a new world disorder would be nearer the mark.