Interior Secretary Ryan Zinke said Monday that almost one-third of career bureaucrats at his department are “not loyal to the flag,” and not in lockstep with him and President Trump.

Zinke, a former Navy SEAL, took over the 70,000-employee department in March, and has since been working to change the department’s regulatory culture to be more business-friendly.

“I got 30 percent of the crew that’s not loyal to the flag,” Zinke said. “We do have good people, but the direction has to be clear and you’ve got to hold people accountable.”

Zinke made the remarks in a speech to an oil industry group and compared his department to a pirate ship that captures “a prized ship at sea and only the captain and the first mate row over” to finish the mission.

“There’s too many ways in the present process for someone who doesn’t want to get (a regulatory action) done to put it a holding pattern,” Zinke said, noting he is pursuing a major reorganization that would push much of the agency’s decision-making outside of Washington -- and move several agencies like the Bureau of Reclamation and Bureau of Land Management to Western states.

Details of the move have yet to be revealed, but Zinke has a strategy.

“Push your generals where the fight is,” Zinke said in his speech to the National Petroleum Council, an advisory committee that includes leaders of the oil and gas industry. “It’s going to be huge. I really can’t change the culture without changing the structure.”

Zinke also said he wants to work to speed up permits for oil drilling, logging and other energy development that now can take years.

“The president wants it yesterday,” Zinke said, referring to permits for energy development. “We have to do it by the law.”

Zinke pivoted to other topics, such as the Endangered Species Act, which he said has been “abused” by bureaucrats and environmental groups and needs to be reformed to be less “arbitrary.”

Zinke also offered a quirky defense of hydraulic fracturing, a drilling technique also known as fracking that has led to a years-long energy boom in the U.S. with sharply increased production of oil and natural gas.

“Fracking is proof that God’s got a good sense of humor and he loves us,” Zinke said.

The Associated Press contributed to this report.