The morale in the intelligence community is "not a good situation" because of a "lack of trust" between it and the White House, former CIA Director Leon Panetta said Sunday.

"Every time he demeans the intelligence community or accuses it of leaks or accuses it of doing things that it's not doing, that obviously impacts on the morale of that institution," Panetta said on NBC's "Meet the Press" with Chuck Todd.

"Look, these are good people. They're not Republicans or Democrats. They're good patriots who are trying to do their job. It's a tough job. They've got to put their lives on the line in order to provide information to the president. The last thing they need is to have a president who questions their patriotism to this country and to him," he continued.

Trump has repeatedly clashed with the intelligence community since taking office, including accusing members of the group of leaking information to the press. As a result, reports have surfaced indicating that intelligence officials are not providing all information to the president.

Panetta urged CIA Director Mike Pompeo and Director of National Intelligence Dan Coats to work to rebuild trust with the intelligence community, saying that that could also stop any possible leaks coming from there.

"If you establish that sense of loyalty then he won't have to worry about leaks," he said

Even if intelligence officials are wary of the president, Panetta said withholding information from the commander-in-chief if "absolutely" a fireable offense. But he said he doesn't believe it's going on.

"I have never had intelligence officers who have a responsibility to provide full information to the president and to other leaders in the country ever withhold a piece of intelligence. So it's something that certainly hasn't occurred when I was there, and I doubt whether it's happened this time," he said.

Trump also butted heads with the intelligence community over the level of Russia's involvement in hacking the election. While intelligence officials said they were certain Russia was behind the acts, Trump, ahead of the inauguration, questioned whether that was really the case.

But Panetta said that the additional briefings and information Trump has access to as president should have changed his mind.

"I think if he listens to the people that are closest to him now, and that are responsible for our foreign policy and our defense policy, that he'll understand that Russia is an adversary, it's not a friend," he said. "So the president, I think ultimately, must recognize that his first responsibility is to protect this country. And one of those you have to protect this country from is Russia."