Story highlights Fmr. Defense Secretary Leon Panetta says Obama should have left troops in Iraq

Panetta says other military commanders shared his views

Despite this, Panetta says U.S. troops should not be sent back to Iraq

Panetta served as Obama's Secretary of Defense from 2011 until 2013

President Obama's former defense secretary, Leon Panetta , says if the White House had listened to his advice on U.S. troop levels in Iraq, the country's security situation may not have unraveled.

"To this day, I believe that a small U.S. troop presence in Iraq could have effectively advised the Iraqi military on how to deal with al-Qaeda's resurgence and the sectarian violence that has engulfed the country," Panetta writes in his upcoming autobiography, due out next week.

President Obama announced in October 2011 that virtually all of the 39,000 U.S. troops would leave Iraq by the end of that year. Obama had said that it was a fulfillment to his 2008 campaign promise to bring troops in Iraq home. But at the same time, talks to get the Status of Forces Agreement, which provided immunity from prosecution to those troops, were at a standstill due to the internal politics of Iraq.

Panetta, who said he argued behind the scenes and publicly for a small amount of troops to be left for additional training for Iraq's military, was frustrated with White House negotiations on the deal. Under Secretary of Defense Michele Flournoy led the administration's efforts in the heated debate.

"Those on our side viewed the White House as so eager to rid itself of Iraq that it was willing to withdraw rather than lock in arrangements that would preserve our influence and interests," he said in excerpts of the book released by Time Magazine.

JUST WATCHED Should Leon Panetta have criticized Obama? Replay More Videos ... MUST WATCH Should Leon Panetta have criticized Obama? 02:43

JUST WATCHED Can Iraq's military handle ISIS? Replay More Videos ... MUST WATCH Can Iraq's military handle ISIS? 02:21

JUST WATCHED France: Young Jihadist joining ISIS The question is why... Replay More Videos ... MUST WATCH France: Young Jihadist joining ISIS The question is why... 03:37

He writes that his views were shared by other military commanders in the region and the Joint Chiefs of Staff but he believes that the Commander-in-Chief could have done more.

JUST WATCHED Iraqi refugees flock to Jordan Replay More Videos ... MUST WATCH Iraqi refugees flock to Jordan 02:44

"Officials there seemed content to endorse an agreement if State and Defense could reach one, but without the President's active advocacy, al-Maliki was allowed to slip away. The deal never materialized," writes Panetta.

President Obama told CBS's "60 Minutes" that former Iraqi Prime Minister Nouri al-Maliki had "squandered" the opportunity to be a thriving democracy with a strong military because he "was much more interested in consolidating his Shia base and very suspicious of the Sunnis and the Kurds" than creating a united government.

The president said that sending U.S. troops back into Iraq is not the solution.

"We cannot do this for them, because it's not just a military problem It's a political problem," he said on Sunday. "And if we make the mistake of simply sending U.S. troops back in, we can maintain peace for a while. But unless there is a change in how, not just Iraq, but countries like Syria and some of the other countries in the region, think about what political accommodation means. Think about what tolerance means."

But Panetta worries that in the meantime the deterioration of Iraq's security has potentially wiped out gains in the fight against terrorism.

"The news from Iraq bothered me to no end. In my view, the ISIS offensive in 2014 greatly increases the risk that Iraq will become al-Qaeda's next safe haven. That is exactly what it had in Afghanistan pre-9/11," he writes. "After all we have done to decimate al-Qaeda's senior leadership and its core, those efforts will be for naught if we allow it to rebuild a base of operations in the Middle East."

Panetta served as Secretary of Defense from 2011 until 2013. He returned to his role as director at the Panetta Institute for Public Policy, which he founded with his wife in 1997. His new book, "Worthy Fights" is due out October 7th.