Washington (CNN) As President Donald Trump huddled in the Oval Office on Friday with Kim Jong Un's right-hand man , he was accompanied by only one other US official: Secretary of State Mike Pompeo. Absent was the administration's most vocal North Korea hawk, national security adviser John Bolton.

The White House insists the meeting was always intended to remain small. But the tableau reflected deeper rifts among Trump's national security team and a new reality as the President prepares for his historic meeting with Kim next week in Singapore: the voices advocating most forcefully to tighten the noose on Pyongyang have been sidelined as Trump seeks out a history-making handshake.

Furious after Bolton raised the so-called "Libya" model to describe US intentions in North Korea -- with its suggestion of a grisly death for a dictator who relinquishes his nuclear arms -- Trump has publicly walked back his national security adviser's rhetoric as he works to ensure his summit talks with Kim proceed. In doing so, he's dramatically lowered the bar for his face-to-face session with Kim, who has made no concrete commitments toward abandoning his nuclear arsenal.

Escalating feud

Pompeo told Trump it would be "counterproductive" to allow Bolton to attend the Oval Office meeting with visiting North Korean official Kim Yong Chol, two people familiar with the matter said, citing an escalating feud between the top diplomat and Bolton. The simmering tensions between two of the President's top foreign policy advisers reached a boiling point after Bolton went on television last month and cited the Libya model when talking about North Korea abandoning its nuclear program -- and in doing so, also raising the specter of Libya's subsequent invasion and its leader's brutal murder.

North Korea reacted furiously, lambasting Bolton in a statement. It revived long-held criticism from the regime, most notably in 2003 when North Korean state media described Bolton as "human scum and a bloodsucker" during the Bush administration.

President Trump continues to have broad faith in John Bolton, people familiar with the matter say.

But the remarks about Libya also infuriated Pompeo, who angrily confronted Bolton in a heated conversation at the White House.

"There has been considerable tension between them ever since," a person familiar with the West Wing confrontation said.

Bolton was not alone in infuriating Pyongyang by publicly referencing Libya. Vice President Mike Pence also raised the "Libya model" in an interview with Fox News last month.

"As the President made clear, this will only end like the Libyan model ended if Kim Jong Un doesn't make a deal," Pence had said.

But unlike Bolton's comment, the President knew Pence would offer the remark in his Fox interview, a senior White House official said, and Pence was merely parroting Trump's comment from the previous week.

The vice president had just returned from lunch with Trump when he taped his interview with Fox News, the official said, and the two men had agreed Pence would reiterate Trump's comments warning North Korea of Libya-style devastation if it did not agree to denuclearize.

Still, the comment drew a fiery statement from North Korea which called Pence a "political dummy" and warned of a "nuclear showdown," prompting Trump to briefly cancel the summit. Pence's comment and the ensuing war of words has led to some speculation that Pence was aligning himself with Bolton's more hawkish views. One senior White House official rejected such claims while another senior administration official noted that Pence rarely strays from the President's views on foreign policy -- even in private discussions.

Smoothing the way to the summit

JUST WATCHED Cooper: Trump's eagerness may lead to bad deal Replay More Videos ... MUST WATCH Cooper: Trump's eagerness may lead to bad deal 04:36

Trump has given his secretary of state, whom he views as intelligent and charismatic, considerable leeway on North Korea -- including following his lead and keeping Bolton at arm's length from the negotiations. Even as the Libya comments from Bolton and Pence prompted angry reactions from Pyongyang -- which, in turn, led to Trump canceling the summit -- Pompeo continued on his diplomatic push with the North Koreans, eventually meeting Kim's envoy in New York last week and smoothing the way for the summit to be rescheduled.

White House chief of staff John Kelly has remained in line with Pompeo, and has come to rely on his ability to guide the President, an official said. Kelly greeted Kim Jong Chol at the White House diplomatic entrance on Friday and escorted him to the Oval Office.

Pompeo views Bolton skeptically, two people familiar with their relationship say, and doesn't trust his motives on North Korea. The two men barely knew each other before Bolton's arrival as national security adviser, but Pompeo has grown to dislike his approach and believes he is "trying to advance his own agenda," one official said.

The White House on Monday sought to downplay any rift between Bolton and Pompeo.

"Any rumor of a confrontation between Ambassador Bolton and Secretary Pompeo is categorically false," a spokesman for the National Security Council said.

"Secretary Pompeo has always been the president's lead on the North Korea summit," the spokesman said, adding that Bolton "continues to coordinate and integrate the interagency process and provide the President with national security options."

Trump continues to have broad faith in Bolton, people familiar with the matter said, despite his displeasure at the Libya comments. But on the issue of North Korea, the President is siding for now with Pompeo, who helped launch the diplomatic overture to Pyongyang when he was serving as CIA director. It's an open question, however, how long that may last, given how rapidly aides can rise and fall in Trump's eye and his insistence upon relying on his own instincts in high-stakes negotiations.

In recent days, Trump has told advisers that decades of conventional thinking on North Korea have failed, and that his own gut will guide him as he prepares for the audacious diplomatic opening with the young and unpredictable Kim. He's sought to heighten anticipation around the summit, which he views as a major historic milestone. And he's worked to soften the harsher rhetoric of his aides, including walking back Bolton and Pence's comments about Libya during remarks in the Oval Office last month.

Maximum pressure?

JUST WATCHED Mattis: Sanctions stay until denuclearization Replay More Videos ... MUST WATCH Mattis: Sanctions stay until denuclearization 01:04

Speaking on the South Lawn after Friday's meeting, Trump acknowledged the months-long administration effort to apply "maximum pressure" on North Korea -- touted in public frequently by Pence and other officials -- was on hold. And he revealed he'd called off a new set of sanctions as diplomats prepare for his arrival in Singapore.

"I don't even want to use the term 'maximum pressure' anymore because I don't want to use that term because we're getting along. You see the relationship. We're getting along," Trump told reporters after bidding farewell to his North Korean visitor, a former spy chief and currently the country's chief nuclear negotiator. "So it's not a question of maximum pressure. It's staying essentially the way it is. At some point, hopefully, a deal -- for the good of millions of people, a deal will be worked out."

The administration insists that, for now, sanctions relief won't come until North Korea takes firm steps toward abandoning its nuclear program.

"Our policy hasn't changed," White House press secretary Sarah Sanders said on Monday, citing existing sanctions on North Korea that won't be removed until the country denuclearizes.

A day earlier, Defense Secretary James Mattis warned North Korea will not get any sanctions relief until it has demonstrated "verifiable and irreversible" steps to denuclearization.

"We can anticipate, at best, a bumpy road to the negotiation," said Mattis, who was speaking alongside South Korean and Japanese defense ministers at the Shangri-La Dialogue, an annual security summit in Singapore that draws government officials and academics from around the world.

Reduced expectations

Even as the nuclear talks themselves face a steep climb, however, the initial meeting scheduled for next week now appears to have reduced expectations. Trump last week acknowledged what aides now say is the main intention of his meeting next Tuesday with Kim: to greet the young dictator and size him up.

"I think it's a 'getting to know you' meeting, plus," Trump said. "And that can be a very positive thing."

Indeed, people familiar with the summit planning now say there is little expectation Trump will emerge from his meeting with Kim having secured the type of historic, detailed commitment on denuclearization that officials once said was a prerequisite for the talks.

Instead, Trump and his aides have suggested the most concrete product of the June 12 encounter could be a peace agreement formally ending the Korean War -- a far cry from the commitment to immediate denuclearization that the administration once insisted would be required for Trump to come to the table.

Now, the most some aides are hoping for is a broad declaration from Kim that he's open to getting rid of his weapons, with the work of securing specific commitments from the North Korean regime handed off to aides for a process that experts predict could take years.

"I never said it goes in one meeting. I think it's going to be a process," Trump said on Friday. "But the relationships are building, and that's a very positive thing."