President Trump's second summit with Kim Jong Un was cut short on Thursday with the two men failing to reach any agreement on how to keep pushing toward Mr. Trump's stated goal of ridding North Korea of its nuclear weapons. In spite of months of preparation following their first one-on-one discussion last year, Mr. Trump and Kim appeared to have shown up in Vietnam without a clear understanding of the other side's expectations, leaving them to cancel a planned lunch and signing ceremony and head home early.

The talks will continue at lower levels, Mr. Trump said, but the failure to reach any new agreement at all will leave America's allies -- South Korea in particular -- worried that the diplomacy could drag on indefinitely while North Korea continues clandestine work on its nuclear and missile programs.

"Sometimes you have to walk," Mr. Trump told reporters after his meetings with Kim. He said he'd "rather do it right than do it fast."

Get Breaking News Delivered to Your Inbox

"It's a process and it's moving along, but we just felt that it wasn't appropriate to sign an agreement today," Mr. Trump said.

But as CBS News "Face the Nation" moderator Margaret Brennan noted, few expected Mr. Trump to walk away from the meeting in Vietnam quite so empty handed, because "the president's language, up until this point, has been so positive in wanting this deal."

Why the talks fell apart early

According to Mr. Trump, the summit came to a sudden and unexpectedly early halt (the table had already been set for the leaders' now-cancelled formal lunch) because Kim demanded a complete lifting of U.S. sanctions against his regime in exchange for the North Koreans opening up just one key nuclear facility to some degree of inspection.

The Yongbyon Nuclear Research Center has been one of the Kim regimes' primary nuclear sites. It ostensibly exists to generate nuclear electricity, but houses multiple reactors which could be used to produce the raw materials to make nuclear bombs.

It is not the North Koreans' only nuclear facility, however.

As CBS News' Margaret Brennan reported after the summit, there is a uranium processing facility and other sites U.S. intelligence agencies know about that Mr. Trump wants access to before he's willing to ease sanctions on the Kim regime. The White House has long said that sanctions relief can only come after demonstrable steps by the North to denuclearize.

Brennan noted that Mr. Trump's explanation of why he decided "to walk" away from the talks -- the limited offering of Yongbyon by the North Korean side and his refusal of it -- was the most detailed insight into the top-level negotiations offered to date by the White House.

In the end, Brennan said Mr. Trump was leaving Vietnam "with the bare minimum that he set," being North Korea's agreement to continue refraining from nuclear and missile tests.

As CBS News' Asia correspondent Ben Tracy pointed out, however, that is not a concession. Kim unilaterally declared an end to his country's tests last year, before he ever sat down with Mr. Trump, claiming that all of his ambitions had been achieved. While that claim is unproven, and reasonably doubted, Kim nonetheless made the decision to stop missile and nuclear tests of his own volition, long before he met Mr. Trump.

One thing that remained unclear in the wake of the abbreviated summit, noted Brennan, is whether Mr. Trump intends to restart joint military exercises with South Korea. He announced a halt to those long-standing war games after his fist meeting with Kim in Singapore last year, to the great surprise of many in the U.S. and around the world who saw it as a concession by the U.S. without anything given by Kim.

While he wouldn't directly answer a question on Thursday about his plans regarding the military exercises, Mr. Trump appeared to indicate a desire to keep them suspended -- citing the costs. He said last year that the money spent on the drills was a main impetus for shelving them.

"We spent hundreds of millions of dollars on those exercises and I hated to see it. I thought it was unfair," Mr. Trump said in Hanoi, adding that he was "was sort of of the opinion that South Korea should help us with that. You know, we're protecting South Korea. I think they should help us with that. So those exercises are very expensive and I was telling the generals, 'look, exercising is fun and it's nice and they play the war games and I'm not saying it's not necessary, because at some levels it is, but at other levels it's not. But it's a very, very expensive thing and we do have to think about that, too."

That kind of language from the U.S. commander in chief, said Brennan, is "something that really troubles our allies in South Korea and Japan; that the president's language is so focused on the U.S. mainland and not helping to protect them. They do want that readiness to be there."

How will North Korea play it?

Tracy said it was remarkable that the North Koreans came to the table and demanded complete sanctions relief in exchange for access only to Yongbyon.

"You have to assume that they went into that meeting knowing that that was not going to be a concession that the U.S. side could make -- which then raises the question about how serious they are about these negotiations," said Tracy.

If anything, Brennan said Kim would walk away from the abbreviated summit in Vietnam with "one win in his pocket" -- he remains far less isolated with the talks continuing. Kim, unusually answering a question from a Western journalist as he sat across from Mr. Trump at the negotiating table, said he was open to the idea of the U.S. opening a liaison office in North Korea.

Mr. Trump called it "a good idea" for such an office to be opened, and Kim called the notion "welcomable."

Even getting Mr. Trump to show up for a second summit was "a huge moment for him, in terms of his reputation," said Tracy.

Reaction from key allies

The lack of any new agreement between Mr. Trump and Kim at the much-touted Vietnam summit will worry America's closest allies in Asia.

Withstanding the two parties doing the talking, the nation with the most vested interest in the U.S.-North Korea negotiations is by far South Korea. The Demilitarized Zone separating the two Koreas is, ironically, one of the most heavily militarized patches of ground in the world.

While a Korean War armistice in 1953 ended active fighting, no peace treaty has ever been signed, so the nations remain technically at war, with the U.S. backing the South with a significant troop presence.

South Korea's Blue House, the official residence of President Moon Jae-in, released a statement on Thursday calling it "unfortunate" that Kim and Mr. Trump had failed to reach an agreement in Vietnam, but stressing that South Korea expected "active dialogue" to continue between the U.S. and North Korea.

Moon's office said the "long and deep discussions" in Hanoi had at least enabled the two sides to increase their understanding of the other's position, and showed that the talks had reached an "elevated level."

Nonetheless, the collapse of that top-level dialogue will be a huge disappointment to Moon, a moderate South Korean leader who has been a driving force behind the Trump-Kim negotiations.

문재인 대통령은 2월 28일 18:50부터 25분 동안 도널드 트럼프 미국 대통령과 전화통화를 했습니다. 트럼프 대통령은 김 위원장과의 회담 결과를 문재인 대통령과 가장 먼저 공유하고 의견을 구하고 싶었다고 하면서, 회담 내용에 대해 상세하게 설명했습니다. pic.twitter.com/hkjI6FkljL — 대한민국 청와대 (@TheBlueHouseKR) February 28, 2019

The Blue House said Mr. Trump and Moon had spoken on the phone Thursday following the end of the summit in Vietnam. It released a photo of the South Korean leader with a phone to his ear, but offered no immediate description of the conversation.

The government of Japan, which like South Korea is protected from any potential North Korean threat by its firm alliance with the U.S., said Thursday that it fully backed Mr. Trump's decision to walk away from the talks.

"I fully support Mr. Trump's decision not to easily compromise based on his strong determination to realize the denuclearisation of the Korean peninsula, keep on having constructive talks and to encourage North Korea's concrete actions," Prime Minister Shinzo Abe said.

He also suggested, however, that he wanted to meet Kim himself in the wake of the breakdown in Vietnam, possibly hinting at his mounting concern.

Like South Korea, Japan feels directly threatened by North Korea. Before halting such tests unilaterally last year, Kim fired two missiles right over Japanese territory. The missile tests were meant as a warning to the U.S., demonstrating the that American territory of Guam was in reach, but it rattled nerves in Japan for obvious reasons.

Residents on the streets of Tokyo were less guarded in their assessment of the stalled diplomacy on Thursday than was Abe.

"They did (the summit) in a flashy manner, (Kim) travelled all the way to Vietnam, crossing China, and nothing came out of it," Tokyo resident Kayoko Iwase told the Reuters news agency. "It makes me wonder whether it was just a performance."

"I think it's not good to leave the nuclear matter unresolved," Seigo Ogawa said. "Insecurity will affect our lives."