Former Defense Secretary Leon Panetta is lashing out at President Obama’s inner circle for failing to secure a 2011 deal to leave U.S. troops in Iraq, effectively accusing the White House of sabotaging the talks – in turn, opening the door for the region to become a haven for the Islamic State.

Panetta, who served as CIA director and then Defense secretary during those negotiations, aired his complaints in his forthcoming memoir, “Worthy Fights.” Excerpts on the Baghdad talks were published by Time.

In them, Panetta explained that Iraqi leaders privately wanted some U.S. forces to stay behind after the formal 2011 withdrawal, though they would not say so publicly. The former secretary, though, said the U.S. had “leverage” to strike a deal, and the Defense and State departments tried to do exactly that.

“But,” he wrote, “the President’s team at the White House pushed back, and the differences occasionally became heated. … and those on our side viewed the White House as so eager to rid itself of Iraq that it was willing to withdraw rather than lock in arrangements that would preserve our influence and interests.”

He said the negotiations with then-Prime Minister Nouri al-Maliki went down to the wire in December 2011, but the White House never stepped up.

More On This...

“To my frustration, the White House coordinated the negotiations but never really led them,” Panetta charged. “Officials there seemed content to endorse an agreement if State and Defense could reach one, but without the President’s active advocacy, al-Maliki was allowed to slip away.”

The account from Panetta challenges the notion that the Obama administration would have left some troops behind – as U.S. military advisers wanted – if only the Iraqi government had been more willing to negotiate. While Panetta lays some blame at the feet of the Iraqis, he also argues that the White House never seized the chance at a deal.

Panetta claims that a residual troop presence like he and others had advocated could have made the difference.

“To this day, I believe that a small U.S. troop presence in Iraq could have effectively advised the Iraqi military on how to deal with al-Qaeda’s resurgence and the sectarian violence that has engulfed the country,” he wrote.

Panetta also warned that the rise of the Islamic State “greatly increases the risk that Iraq will become al-Qaeda’s next safe haven.”

Gen. John Campbell, commander of the International Security Assistance Force in Afghanistan, was asked Thursday about Panetta’s comments, but said “we absolutely left [the Iraqis] in the best possible condition militarily that we could.”

He put to onus on the Iraqi government.

“Things that were done by the government did not bring all the different factions in Iraq together was not something that … the U.S. military could have done or changed once we left there in 2011,” he said.

Asked again whether leaving a force in Iraq could have helped, he said: “I think any military guy is going to tell you if you could leave a force, you'd always leave a force.”

Sens. John McCain, R-Ariz., and Lindsey Graham, R-S.C., though, seized on Panetta's comments -- as well as similar remarks by former Iraq Ambassador Ryan Crocker that the U.S. "could have gotten that agreement" if officials had been more persistent.

"The latest statements by two of the most respected national security officials to serve under President Obama definitively refute the falsehood that this Administration has told the American people for years about their efforts to leave a residual force in Iraq," the senators said in a statement. "As we have said all along ... the Obama Administration never made a full effort to leave a residual force in Iraq."