President Trump's critics have made it clear they probably won't be happy regardless of what happens at his historic, imminent one-on-one meeting with North Korean leader Kim Jong Un, former national security official Michael Anton said on Fox News' "Special Report."

Anton, the former Deputy Assistant to the President for Strategic Communications on the United States National Security Council under Trump, agreed with the president that he "can't win either way."

Earlier Monday, Trump tweeted his frustration at the "haters & losers" who claimed that sitting down to meet with Kim was bad for the U.S., and touted his own successes so far on the Korean Peninsula.

"We have our hostages, testing, research and all missle launches have stoped, and these pundits, who have called me wrong from the beginning, have nothing else they can say! We will be fine!" Trump wrote.

In May, the Trump administration secured the dramatic release of three American detainees it called "hostages" in North Korea, after Secretary of State Mike Pompeo met with Kim.

WATCH: FREED HOSTAGES THANK TRUMP FOR RELEASE FROM NORTH KOREA

Trump's anger is understandable, Anton said, given that "the same people who are criticizing him for being too harsh in one breath, turn around and criticize him for being too soft and giving too much away.

"I think that criticism is inconsistent," he continued. "And I also agree with the president that I don't really see the downside to talking."

Late Monday, just hours before the historic summit, House Democratic Leader Nancy Pelosi and ranking Democrats released a statement saying they are "deeply concerned" that the White House has not confirmed an ambassador to South Korea, or assistant secretary for Asian affairs.

The Democrats insisted on "congressional oversight" of any deal with North Korea, and took the opportunity to decry Trump's "behavior at the G7 summit" that "alienated our closest friends."

The only way that Trump could hand his critics a genuine win and foul-up the critical summit, Anton said, would be if he went too easy on the autocratic regime.

"The downside to talking only occurs if the United States ... gives away too much," Anton said. "The administration appears to me to be completely aware of that danger and determined to avoid it."

The priority in the summit is to denuclearize North Korea, but critics shouldn't pounce even if Trump cannot immediately secure that outcome during the summit, Anton said.

"This is something that I am confident the administration will keep pressing for even as it keeps its eye on the priority, which is denuclearization -- the immediate priority," he added.