The US defense secretary has been fired after less than two years in office as the White House reorders a national security strategy upended by the Islamic State (Isis).

Chuck Hagel, Barack Obama’s third Pentagon chief and a former Republican senator, will leave the Department of Defense just weeks after his spokesman said Hagel was looking forward to serving “for the remainder” of the Obama administration.

Confirming his departure during an awkward White House ceremony, Obama said Hagel would stay in post until a successor was confirmed by the Senate, though the president was less effusive than he had been announcing the departure of his friend and closer political ally, the attorney general Eric Holder.

Obama went out of his way to list Hagel’s achievements, but avoided dwelling on US engagement in Iraq and Syria, merely praising Hagel’s work “helping build the international coalition” against the Islamic State.

“Thanks to Chuck, our military is on a firmer footing engaged in these missions and looking ahead to the future,” added Obama, who hinted at his heavy involvement in Hagel’s decision to resign.

“Last month, Chuck came to me to discuss the final quarter of my presidency, and determined that having guided the department through this transition, it was an appropriate time for him to complete his service,” said the president.

Though the two men had served together previously in the Senate, Obama seemed short of anecdotes, celebrating Hagel’s career that spanned six decades by pointing out that soldiers had felt comfortable enough in his presence to ask which football team he supported.

An emotional Hagel briefly fumbled his response, in a confused opening statement that appeared to pay tribute to Vice-President Joe Biden, who stood next to them.

Facebook Twitter Pinterest Chuck Hagel gave his warm thanks to Joe Biden. Photograph: Pablo Martinez Monsivais/AP

“I have always valued and will continue to value into my – not old, but my – longtime dear friend Vice-President Biden, who I have always admired and respected, and both the president and I have learned an awful lot from the vice-president over the years. Thank you,” said Hagel.

But Hagel went out of his way to stress he had resigned and remained proud of his achievements in office.

“As the president noted, I have, today, submitted my resignation as secretary of defense,” said Hagel.

“It’s been the greatest privilege of my life … to serve with the men and women of the Defense Department and support their families. I am immensely proud of what we’ve accomplished during this time.”

The first national security casualty of Obama’s midterm elections defeat was one who, despite his Capitol Hill pedigree and Republican registration, never won the confidence of the congressional GOP, who considered him a water-carrier for the administration.

Before Obama’s announcement, a senior administration official praised Hagel as “a steady hand”, and said Hagel had been speaking with Obama in October about leaving “given the natural post-midterms transition time”. Hagel’s spokesman, Rear Admiral John Kirby, told Pentagon reporters on 7 November that Hagel expected to stay on.

Hagel was out of step with the administration on Isis, having urged the White House to clarify its stance on ushering the Syrian dictator, Bashar al-Assad, out of power and bizarrely inflating the threat Isis posed, calling it “an imminent threat to every interest we have” in an August press conference. While the administration has publicly ruled out using US ground forces in combat in Iraq, Hagel and particularly the chairman of the Joint Chiefs of Staff, General Martin Dempsey, floated precisely that as an option in testimony earlier this month.

A man who never quite found his footing as Pentagon chief, Hagel also testified that the US strategy against Isis – which focuses on Iraq primarily and Syria peripherally – was working, even as it undergoes frequent adjustment and revision.

Yet the strategy has come under criticism from hawks as well as doves. Hawks want a deeper US commitment of air as well as ground forces to beating Isis back, while doves are alarmed at the shifting of US war aims and commensurate resources. The next chairman of the Senate armed services committee, the Arizona Republican John McCain, wants a more forceful US response to Isis and had long fallen out with his former friend Hagel.

In the five months since Isis seized Mosul, Obama has authorized 3,000 new troops to advise and train Iraqis, and expanded an air war into Syria. Pentagon efforts to field a Syrian proxy force have barely begun and are expected to take a year before yielding the first capable units.

Hagel, a Vietnam combat veteran and a noncommissioned army officer, was not expected to be a wartime defense secretary, instead brought in to manage the downsizing of US ground forces and shore up the administration’s at-times uneasy relationship with the military. His Senate confirmation hearing saw the former senator rambling and unfocused; he mischaracterized the administration’s position on Iran. Among Hagel’s more forceful positions early in office was to warn against US involvement in the Syrian civil war.

Several oft-mentioned names to replace Hagel have already surfaced. The former defense policy chief Michèle Flournoy, a figure deeply identified with the troop surge in Afghanistan, would be the first woman to run the Pentagon. The Times reported that the Rhode Island Senate Democrat Jack Reed is in the running, as is Ashton Carter, a senior official noted for his management and budgetary skills who was Robert Gates’s acquisitions chief and Leon Panetta’s deputy secretary.

The senior official said a successor would be nominated in “short order” and Hagel will serve until his successor’s confirmation.