Former Defense Secretary Robert Gates said the Obama administration’s “micromanagement” of national security and defense remains a problem.

“I will say, first of all, that I had my issues with the White House but my decision to leave was, I mean, I stayed longer than I intended by a long shot,” he said during a discussion at the Council on Foreign Relations.

Gates recalled President Obama asking him to stay in the position through the end of his first term.

“By that time I’d done the job for four and a half years. That was longer than all but four of my predecessors and I was exhausted. We’d been four and a half years at war and I was ready to leave,” he said. “In the four and a half years since I left, as you point out, there have been three secretaries.”

Citing the freedom-of-navigation exercise in the South China Sea, Gates said one of the problems in the Obama administration is “micromanagement” directly from White House.

“I don’t know this for a fact, but I’d be willing to wager that I’m pretty close to being right. I think the White House was very much involved in the delays and in the handling of the freedom-of-navigation exercise in the South China Sea. We should have sent a ship into those waters long before we did,” he said.

Some have speculated that the White House urged the Pentagon not carry out certain aspects of the operation, which occurred in October 2015. According to several reports, the operation was supposed to feature a Navy vessel trailed by U.S. surveillance planes sailing close to China’s man-made islands in the South China Sea.

“And I would bet anything that the White House said ‘here are the rules of engagement’ that made it look more like innocent passage rather than a freedom-of-navigation exercise,” he added.

Gates also said there was a direct line to a White House staffer when he visited a Special Operations command center in Afghanistan.

“I stood there and I said, ‘you rip that out while I’m standing here and watching’ and I told our commanders, I said, ‘If you get a call from a White House staffer or an NSC staffer, you tell them you’re not authorized to take the call and refer them to my office and tell them I said to go to hell,’” he said.

Gates, who has worked under Henry Kissinger, Zbigniew Brzezinski, and Brent Scowcroft, told the audience he would have been fired if he had called a four-star commander as a National Security Council staffer.

“I’d have been fired instantly. So this micromanagement is a problem and, I mean, my view—and frankly, one of the main lessons in the book – is you have to choose your subordinates carefully, but then you have to empower them, hold them accountable, and if they don’t do the job, get rid of them. But above all, you can’t micromanage them,” he said.

A World War II draftee asked Gates what needs to be done to address the strain the ongoing war on terrorism has put on the U.S. military.

The former CIA director pointed out that most military activities today are not in combat but rather deterrence.

“Where we are in combat today is in very limited numbers in dealing with the lingering extremist problem in Afghanistan and the problem of ISIS that is mainly located in Syria and in certain parts of Iraq,” he said.

“The numbers of our forces that will be engaged directly in those conflicts I think will, in the foreseeable future at least, be a pretty limited number of people but there is no denying the enduring threat that these extremists pose to the United States and to Europe and to others.”

Gates compared the terrorist threat to a public health problem or crime that the U.S. would have to battle on an enduring basis.

“There won’t be a moment when any of these groups say we surrender or we quit, and so we’re going to have to deal with this, I believe, on a long-term basis,” he said.

“I think we have to be very careful about drawing this line between exercising our leadership in the world and leading from the front and having a credible deterrent, and having our allies and friends know that we are a dependable ally,” he added.

Gates said the U.S. government must be able to draw the line between accomplishing its objectives and becoming the world’s policeman.

“It was one of the reasons why I opposed the intervention in Libya. I did not see our interests directly engaged and I said in the Situation Room — I said, can I just finish the two wars I’m already in before you go looking for another one?” he recalled.