In the immediate aftermath of the September 11 attacks the then national security adviser, Condoleezza Rice, called in her senior staff and asked them to think seriously about "how [to] capitalise on these opportunities". The primary opportunity came from a public united in anger, grief and fear which the Bush administration sought to leverage to maximum political effect. "I think September 11 was one of those great earthquakes that clarify and sharpen," Rice told the New Yorker six months afterwards. "Events are in much sharper relief."

Ten years later the US response to the terror attacks have clarified three things: the limits to what its enormous military power can achieve, its relative geopolitical decline and the intensity of its polarised political culture. It proved itself incapable of winning the wars it chose to fight and incapable of paying for them and incapable of coming to any consensus as to why. The combination of domestic repression at home and military aggression abroad kept no one safe, and endangered the lives of many. The execution of Osama bin Laden provoked such joy in part because almost every other American response to 9/11 is regarded as a partial or total failure.

Inevitably, the unity brought about by the tragedy of 9/11 proved as intense as it was fleeting. The rally around the flag was a genuine, impulsive reaction to events in a nation where patriotism is not an optional addendum to the political culture but an essential, central component of it. Having been attacked as a nation, people logically felt the need to identify as a nation.

But beyond mourning of the immediate victims' friends and families, there was an element of narcissism to this national grief that would play out in policy and remains evident in the tone of many of today's retrospectives. The problem, for some, was not that such a tragedy had happened but that it could have happened in America and to Americans. The ability to empathise with others who had suffered similar tragedies and the desire to prevent further such suffering proved elusive when set against the need to avenge the attacks. It was as though Americans were unique in their ability to feel pain and the deaths of civilians of other nations were worth less.

It's a narcissism best exemplified by former vice-president Dick Cheney's answer when asked just last week on what grounds he would object to Iran waterboarding Americans when he maintained his support for America's right to use waterboarding. "We have obligations towards our citizens," he said. "And we do everything to protect our citizens."

However perverse that seems now such views had great currency at the moment, following the attacks, when many of the mistakes that would shape US foreign policy for the next 10 years were made. Terrorism will do that. "Terror is first of all the terror of the next attack," writes Arjun Appadurai in Fear of Small Numbers. If nothing else the Bush administration had fear on its side. "The next time the smoking gun could be a mushroom cloud," said Rice. "They only have to be right once. We have to be right every time."

The trouble is they got very little right. Broad sweeps of people from predominantly Muslim countries resulted in the "preventive detention" of 1,200 people; voluntary interviews of 19,000; and a program of special registration for more than 82,000 – but not a single terrorism conviction. A decade on the US ability to crush al-Qaida still depends almost entirely on its ability to negotiate with Pakistan and doing a deal with the Taliban in Afghanistan, where last month there was the highest US military death toll since the war began. And that's before we get to Iraq.

An effective response to 9/11 that would have truly satisfied the American public in that moment probably did not exist. A combination of diplomatic pressure, targeted intelligence-led operations and a more enlightened foreign policy was what would have been and has proved to be most successful. But following the attacks, when declarative sentences were the only ones heard and those who urged caution and restraint were compared to Neville Chamberlain, something more urgent, punitive and impressive was insisted upon.

Even now, the case against bombing Afghanistan is often met with the question: "So should we have done nothing?" As though anything short of a military response does not qualify as a response at all, and as if doing something that did not work and left untold innocents dead is better than doing something that would have been more effective but less dramatic.

Dissent to this logic in the US was initially was just that: dissent – minority views dismissed, ridiculed or even vilified by the mainstream. Shortly after the attacks ABC news anchor Ted Koppel introduced Arundhati Roy, Indian novelist and opponent of the Afghanistan war, thus: "Some of you, many of you, are not going to like what you hear tonight. You don't have to listen. But if you do, you should know that dissent sometimes comes in strange packages …"

But as time went on the number of dissenters started swelling. The most important single factor that shapes American attitudes to any war is whether they think they will win, explains Christopher Gelpi, professor of political science at Duke University who specialises in public attitudes to foreign policy. As the Iraq war floundered unity gave way to the acrimony, mistrust and mutual recrimination that characterises US politics today.

The response to 9/11 did not create these divisions – a year before the attacks the presidential election was decided by the courts – but it deepened, broadened, sustained and framed them for more than half a decade before the economic collapse. It was the central issue in the 2004 election and cast the 2008 election in terms of hope – Obama – against fear, McCain and Palin. Internationally Obama's victory marked the country's belated, more nuanced, more enlightened response to 9/11, signalling America's readiness to meaningfully re-engage with the rest of the world and the treaties that govern it.

Sadly that change in tone, style and to some extent substance has also proved inadequate. True, Obama killed Bin Laden, and his administration plans to draw down troops in Iraq and Afghanistan and has retired the phrase "war on terror". But they have maintained many of the most problematic elements of that war, including Guantánamo Bay, extraordinary rendition and military commissions, while intensifying the war in Afghanistan.

Meanwhile on the right, the hubris displayed by Rice that America could simply bend the world to its will and whim has since given way to denial and occasional bouts of impotent rage. Islamaphobia is on the rise, Muslim has become a slur and Iraq, apparently, was a success.

In 2004 a Bush aide (widely believed to be Karl Rove) chided a New York Times journalist for working in the "reality-based community", meaning people who "believe that solutions emerge from your judicious study of discernible reality … That's not the way the world really works anymore. We're an empire now, and when we act, we create our own reality. And while you're studying that reality – judiciously, as you will – we'll act again, creating other new realities, which you can study too, and that's how things will sort out. We're history's actors … and you, all of you, will be left to just study what we do".

But that's never been how the world works. And over the last 10 years reality has caught up with the rhetoric.