(CNN) -- In his official farewell to outgoing Defense Secretary Robert Gates, U.S President Barack Obama honored the plainspoken Kansas native on Thursday with the Presidential Medal of Freedom, America's highest civilian honor.

"I'm deeply honored and moved by your presentation of this award," Gates told Obama during a tribute in front of the Pentagon, punctuated by brass band music.

"It is a big surprise. But we should have known a couple of months ago that you're getting pretty good at this covert ops stuff," he said, possibly referring to a secret U.S. raid Obama authorized which killed al Qaeda leader Osama bin Laden.

Gates -- whose four decades of public service spanned eight U.S. presidents -- is succeeded by former CIA Director Leon Panetta.

Considered the ultimate Washington insider, Gates -- himself a former CIA chief -- replaced Donald Rumsfeld in 2006, and served as secretary of defense in the administrations of both George W. Bush and Barack Obama.

He joined the CIA in 1966 and later worked at the National Security Council and at the White House, making him the only career officer in CIA history to rise from entry-level employee to director.

Once the president of Texas A&M University, Gates described Thursday the transition between the Bush and Obama teams as "the first of its kind" during a war in over 40 years.

Following Obama's 2008 election, he reached out to Gates in a secret meeting at a fire station at Reagan National Airport outside of Washington and convinced the outgoing Pentagon chief to stay in office.

Obama called Gates "one of the nation's finest public servants," noting his "profound sense of duty" that led him to continue serving in the Obama administration despite his desire to return to civilian life.

Gates once had a "countdown clock" that ticked down the days until the end of the Bush administration, indicating the date when he could retire. But within weeks of Obama's election, the clock had become obsolete.

"I've thrown away the clock because it was absolutely useless," Gates said in December 2008.

Obama said Gates "challenged conventional wisdom" to reduce wasteful military spending and save "hundreds of billions," while helping to refocus U.S. attention on the treatment of wounded soldiers.

Shortly after replacing Rumsfeld in the Pentagon's E-Ring office suite, Gates was confronted with a scandal at Walter Reed Army Medical Center, the military hospital in Washington in which wounded troops from Iraq and Afghanistan are treated.

When a Washington Post investigation unearthed months of substandard care there, the man in charge of the hospital was fired and the secretary of the Army resigned.

"I am disappointed that some in the Army have not adequately appreciated the seriousness of the situation," Gates said when announcing Secretary Francis Harvey's resignation. "Some have shown too much defensiveness and have not shown enough focus on digging into and addressing the problems."

During his tenure, Gates implemented the "troop surge" in Iraq, and helped prosecute the U.S.-led war in Afghanistan.

"Today we see the lifesaving difference he made," Obama said Thursday.