Former Defense Secretary Robert Gates said Sunday it was a “big mistake” for President Donald Trump to exclude the chairman of the Joint Chiefs of Staff and the Director of National Intelligence from National Security Council meetings.

Speaking on ABC’s “This Week,” Gates weighed in on a memorandum signed by Trump Friday that reorganized the National Security Council, elevating White House Strategist Steve Bannon as a member but declaring that the DNI and Chairman of the Joint Chiefs “shall attend where issues pertaining to their responsibilities and expertise are to be discussed.”

Gates, who also served as Director of National Intelligence as well as headed the CIA, said “adding people … never really bothers me,” but removing members was his “biggest concern.”

“My biggest concern is there are actually, under the law, only two statutory advisers to the National Security Council and that's the Director of Central Intelligence, or the DNI, and the Chairman of the Joint Chiefs of Staff. I think pushing them out of the National Security Council meetings, except when their specific issues are at stake, is a big mistake,” he said. “I think that they both bring a perspective and judgment and experience to bear that every president, whether they like it or not, finds useful.”

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

Trump has had a fraught relationship with the intelligence community, accusing them of leaking information to the media about the president’s intelligence briefings in “one last shot at me,” tweeting at one point, “are we living in Nazi Germany?”

His team was reportedly considering ways to reign in the DNI throughout the transition period, in part, sources have said, because of the history Trump’s National Security Adviser, retired Lt. Gen. Mike Flynn, has with the intelligence community. He clashed with former DNI James Clapper and was ultimately driven out of his post as director of the Defense Intelligence Agency.

White House Chief of Staff Reince Priebus said during his interview on NBC’s “Meet the Press” that the executive order did not limit access of the DNI and the chairman of the Joint Chiefs.

“If you read the order they’re invited as attendees to the Security Council at any time,” he said.

While the order affirms that the two are members of the NSC, it states that with respect to the Principals Committee — “the Cabinet-level senior interagency forum” to consider policy issues affecting national security — “the Director of National Intelligence and the Chairman of the Joint Chiefs of Staff shall attend where issues pertaining to their responsibilities and expertise are to be discussed.”

Sen. John McCain, chair of the Senate Armed Services Committee, criticized what he characterized as Trump's decision to elevate Bannon and minimize the role of Gen. Joseph Dunford, the chairman of the Joint Chiefs of Staff, on CBS' "Face the Nation."

“I am worried about the National Security Council. Who are the members of it and who are the permanent members? The appointment of Mr. Bannon is something which is a radical departure from any National Security Council in history," McCain said.

McCain added: "The role of the chairman of the Joint Chiefs of Staff has been diminished, I understand, with this reorganization. The one person who is indispensable would be the chairman of the Joint Chiefs of Staff, in my view.So it’s of concern, this 'reorganization.'"