Defense Secretary James Mattis wasn't involved in the discussions Wednesday, though Trump said that he called Mattis about it Thursday morning.

But it was a second round of calls early Thursday, between 7 a.m. and 9 a.m., according to senior White House officials, that convinced Trump to walk away from the summit. His letter went to the North Koreans at 9:43 a.m.

The decision occurred so abruptly that the administration was unable to give congressional leaders and key allies advance notice. The letter went out while more than two dozen foreign journalists, including several U.S. citizens, were inside North Korea, where they had gone to witness a promised dismantling of a nuclear test site. At 8:20 a.m., the State Department sent a note to reporters touting the positive discussions that Pompeo was having with Asian counterparts in preparation for the summit.

The move exposed significant disagreements among the president's top advisers. Several administration officials said Pompeo, who has taken the lead in negotiating with the North Koreans, blamed Bolton for torpedoing the progress that had already been made. Pompeo flew to Pyongyang, the North Korean capital, twice; met personally with Kim; and helped secure the release of three Americans who had been held there. Bolton, a longtime national security hawk who has publicly advocated for regime change in North Korea, was integral, these officials said, to convincing Trump to back out of the summit.

North Korean leader Kim Jong Un. KCNA via AP

But another senior official pushed back on any tension between Pompeo and Bolton, describing the four principals as a "pretty tight-knit group" on this topic.

Sen. Cory Gardner, R-Colo., told reporters on Thursday, after a meeting at the White House, that Trump dictated the letter directly to Bolton.

One person close to Trump said that the president was unhappy with Pence for public remarks he made earlier this week that appeared to threaten Kim with the prospect of regime change if North Korea didn't meet America's terms to rid itself of nuclear weapons. "This will only end like the Libyan model ended if Kim Jung Un doesn't make a deal," Pence said Monday.

A senior administration official said that if the president was indeed unhappy with his vice president, he did not express it directly — adding that Pence's "Libyan model" comments should not have come as a surprise to Trump.

Bolton was first to issue such a threat on April 29, when he also referred to Libya, whose former leader, Moammar Gadhafi was ousted, with U.S. assistance, several years after he agreed to give up ambitions of a nuclear weapons program. Such comparisons to Gadhafi have infuriated the North Koreans.

Libyan leader Moammar Gadhafi in 2011. Reuters file

Trump tried to repair the damage on May 17, when he rejected the Libyan comparison and later guaranteed Kim's security. Pence's remarks reignited the North's anger and a senior foreign diplomat responded by issuing a counterthreat of a "nuclear showdown."

One senior administration official said that Bolton and Pompeo, new to each other since Bolton joined the administration less than two months ago, have been at odds about the summit since it was first proposed. The State Department, this official said, had wanted more advance work done with agreements and decisions on achievable goals made before Trump and Kim met.

"This has been like herding cats," the official said.

Bolton, who has led the effort on the White House side, has worked unilaterally to shape the summit. One person familiar with the summit preparations said it was Bolton who drove the decision to cancel and that he had convinced Trump to make the move. Trump then relayed his decision to Pompeo, who felt blindsided, according to multiple officials.

A driving factor for the president was the belief that Kim was heading toward a similar conclusion.

Still, several officials pointed to a door Trump left ajar in his remarks Thursday, when he suggested that a summit may be possible in the future.