More than three years ago, I sat in an overflow room in Washington, DC's Willard Hotel listening to General David Petraeus explain (pdf) how the only solution for the failing war in Afghanistan was a "comprehensive counterinsurgency (COIN) strategy", modeled after the one that had allegedly achieved so much success in Iraq.

Petraeus's speech came at the annual meeting of the Center for New American Security, a DC-based thinktank that had become a locus of COIN thinking in DC. And Petraeus was at the peak of his power and acclaim – heralded by both Democrats and Republicans as the man responsible for saving the Iraq war.

The four-star general's in-depth powerpoint presentation (pdf), with its discussion of securing and serving the population, "understanding local circumstances" separating irreconcilables from reconcilables and living "among the people" was the apogee of COIN thinking, which dominated national security debates in Washington in 2008 and 2009. But, like Petraeus's career, COIN and its usefulness as a tool for US military planners now lies in tatters.

With last week's revelations that Petraeus was having an affair as director of the CIA with his biographer Paula Broadwell, this tawdry story is likely to become the most glaring black mark on Petraeus's career. But while his behavior was reckless, arrogant and, frankly, just plain stupid, it's ironic that Petraeus is likely to be remembered more for that one personal act rather than his most grave professional mistake – namely, that same counter-insurgency strategy in Afghanistan for which he was one of Washington's most influential proponents.

The event at CNAS was the quintessential example of the blinders and hubris that were so pervasive among COIN boosters and, in particular, Petraeus. They were convinced that the surge in Iraq and the use of counterinsurgency tactics there had turned the tide. But as we know now – and should have even been aware then – the reality was far more complicated.

In truth, a number of key social and political shifts occurred in Iraq in 2006 and 2007, which coincided with the US surge. There was the decision by Sunni militias to turn against al-Qaida in Iraq, a group that was responsible for initiating much of the country's horrifying violence in 2006; there was the ethnic cleansing and enclaving that took place in Baghdad, which turned a once Sunni-dominated city into one controlled by Shiites, and gave both sides in the civil war fewer individuals to seek out and slaughter; there was the mass exodus of refugees out of the country; and later, there was the Sadr ceasefire.

What's more, those who pushed the Iraq surge narrative suggested that a more humane and civilian-focused approach there had brought success. In reality, the number of civilians killed by US airstrikes had increased nearly four-fold in Iraq; the number of Iraqis in detention jumped 50%. This is not to suggest Petraeus deserves no credit; he smartly took advantage of these larger shifts in Iraqi society to seek an endgame to the conflict.

But the reality is that much of the decline in violence attributed to the actions of US forces was the result of decisions and actions taken by the Iraqis themselves. The US role was important, but hardly decisive.

This, of course, was a much more complicated explanation for what happened in Iraq – and one far less gratifying to US policy-makers. This more nuanced reality did little to prevent Petraeus and his acolytes from not simply taking a victory lap but far worse, using the supposed "lessons of Iraq" to justify a similar course of action in Afghanistan.

Indeed, around the same time as Petraeus's speech, COIN boosters were regularly arguing that the key to success in Afghanistan was reducing civilian casualties – and that such a goal could be achieved by the application of counter-insurgency tactics.

In reality, the assumptions of COIN advocates were badly flawed and based on unrealistic views of what the US could accomplish. It failed to take account the key ways in which Afghanistan differed from Iraq: the resilience of and public support for the Taliban insurgency; the presence of safe havens across the border; the incompetence of the central government in Kabul; the delusion that US soldiers could be turned into miniature anthropologists with the wherewithal to have a full appreciation of Afghan cultural idiosyncrasies; and finally, an abject refusal to factor in the lack of political support in the United States for a drawn-out counterinsurgency campaign.

Worst of all, COIN advocates committed the cardinal sin of believing that a shift in military tactics or a new commander would be enough to win a military conflict in which the US was engaged. As the great Chinese war philosopher once wrote, "tactics without strategy is the noise before defeat". That is a lesson that Petraeus, among others, simply forgot. Indeed, it's worth remembering that when asked by President Obama, point blank, if a surge of troops to Afghanistan could turn things around in 18 months, Petraeus responded:

"Sir, I'm confident we can train and hand over to the ANA [Afghan National Army] in that time frame."

Petraeus was wrong – badly wrong. And more than 1,000 American soldiers, and countless more Afghan civilians, have paid the ultimate price for his over-confidence in the capabilities of US troops. And it wasn't as if Petraeus was an innocent bystander in these discussions: he was working a behind-the-scenes public relations effort – talking to reporters, appearing on news programs – to force the president's hand on approving a surge force for Afghanistan and the concurrent COIN strategy.

But when he took over as commander of the Afghanistan war in 2010, Petraeus adopted the harsh military strategy that he'd claimed the new, more civilian-focused COIN military plan would eschew. He ramped up airstrikes, which led to more civilian deaths. He increased the use of special forces operations. Perhaps worst of all, he sought to hinder the implementation of a political strategy for ending the war, seeking, instead, a clear military victory against the Taliban.

The greatest indictment of Petraeus's record is that, 18 months after announcing the surge, President Obama pulled the plug on a military campaign that had clearly failed to realize the ambitious goals of Petraeus and his merry team of COIN boosters. Today, the Afghanistan war is stalemated with little hope of resolution – either militarily or politically – any time soon. While that burden of failure falls hardest on President Obama, General Petraeus is scarcely blameless. Yet, to date, he has almost completely avoided examination for his conduct of the war in Afghanistan.

In an age in which military officers are practically above public reproach – glorified and exalted by politicians and the media – the repeated failures of our military leaders consistently escape analysis and inquiry. This can have serious national security implications. As Joshua Rovner, associate professor of strategy and policy, US Naval War College, said to me in an email conversation, this lack of scrutiny has had grave consequences:

"[W]e have misunderstood our recent history in Iraq and Afghanistan; we have created new myths about strategy that will persist for many years despite their manifest flaws; and we may make bad decisions about intervening in other civil wars based on these myths."

The wars in Iraq and Afghanistan were more than just bad strategy; they reflected poor military tactics and generalship. Self-interested and incomplete interpretations of what happened in Iraq led to predictably disastrous results in Afghanistan.

Perhaps we should spend a bit more time looking at that issue, rather who was sleeping with whom.