Washington (CNN) President Donald Trump has been flustered by the onslaught of negative coverage generated by his new personal lawyer Rudy Giuliani , who has exacerbated his political troubles in recent days with a series of unscripted interviews, four sources familiar with knowledge of the President's thinking tell CNN.

Giuliani disputes that characterization.

"If he were flustered he would have told me on Sunday," Giuliani told CNN's Dana Bash, referring to a lengthy meeting he had that day with the President at his golf resort.

Giuliani joined Trump's legal team amid a shakeup less than three weeks ago to represent him in the special counsel's investigation into Russian election interference and possible ties to Trump associates. Since he made the bombshell announcement last Wednesday that the President had reimbursed his attorney Michael Cohen for the $130,000 paid to Stormy Daniels, Giuliani has appeared on several television shows and given interviews to more than half a dozen outlets.

Trump has grown irritated by Giuliani's performances and the headaches he has caused as contradictory statements between the two flash across the chyrons on his screen.

The President was initially pleased with his new attorney's brashness, but that has steadily eroded as the parade of interviews has continued. Trump was irked by Giuliani's interview on ABC News Sunday , particularly the clip of Giuliani refusing to rule out him pleading the Fifth Amendment, which has since played on a near-constant loop on cable television.

Giuliani insisted that if the President were not happy with his performance, the President would let him know.

"We have been close friends for 30 years and we can say things to each other that others can't," he told Bash.

"The key people involved in this are very happy," Giuliani said, insisting he is totally on the same page as Jay Sekulow and other Trump lawyers.

He once again pointed to Tuesday's decision on Iran and an upcoming planned summit with North Korea, saying their legal strategy "freed him up to focus and put some wins on the board."

When asked specifically about sources telling CNN that the President was annoyed with Giuliani leaving open the possibility of Trump pleading the 5th Amendment, again Giuliani pushed back.

"It's the perfect way it should be done. It's honest. He wants to testify. We have to figure out what's right," said Giuliani, saying if his answer was "no way" to the question about the 5th Amendment, that it would not be accurate because none of this has been decided.

More broadly, Giuliani defended not coordinating with White House staff.

"(Trump) doesn't let us talk to them about the case. We can talk to the counsel, not people in the White House," Giuliani said.

Longtime friend

JUST WATCHED Giuliani: It's possible Cohen paid off more women Replay More Videos ... MUST WATCH Giuliani: It's possible Cohen paid off more women 00:56

Giuliani was a Trump loyalist during the 2016 presidential campaign who kept his eye on one position -- secretary of state -- but it soon became apparent that he wouldn't get it. A source with knowledge of the President's thinking at the time said Trump passed Giuliani over for the high-profile role because he thought he wasn't as sharp as he used to be.

The former New York City mayor told Trump he wasn't interested in any other Cabinet positions and would instead remain an effective ally of his outside of the administration. The two stayed in contact during Trump's chaotic first year in office.

But more than a year later, the President was growing increasingly frustrated with his legal team, who had promised him for months that the special counsel's investigation would wrap up soon. Instead, it seemed to be expanding.

As Trump looked for a more aggressive approach to the Russia probe, he sought solace in his old friend "Rudy," who had a similarly abrasive and volatile style. But after a string of stunning interviews in recent days raised more questions than answered them, the President began to realize that his hope that Giuliani could end the investigation quickly might have been misguided.

Giuliani, however, has been insistent that he and the President are synced.

"You wont see daylight between me and the President," he told CNN Thursday.

Though the President has been displeased with Giuliani, a source familiar with his thinking said it's not likely he will fire him. Instead, one source predicted, he will leave him on his legal team, but move him to the backburner. Giuliani has given the rest of the legal team pause, two people familiar with the matter say. Since he joined, Giuliani and Trump have maneuvered the strategy separately from everyone else, and are expected to be the two who decide whether Trump will ultimately sit down with Robert Mueller.

The response in the West Wing to Giuliani's arrival has been overwhelmingly negative. White House staffers have been rankled by his media parade, three sources said, with some questioning in private when he last practiced law and remarking that he might be losing his mental faculties. Giuliani has complicated things for West Wing officials, they say, by undermining the defense strategy they have used for the last several months while speaking on behalf of the President without coordinating it with the White House.

Initial strategy for managing Giuliani

The White House's initial strategy for responding to Giuliani's questions has been to refer them back to Giuliani himself. At the briefing Monday, White House press secretary Sarah Sanders said Trump "certainly feels that he's an added member -- added valued member to his outside special counsel." She later fielded several questions to the "outside special counsel."

A Republican strategist close to the White House said if they were on the communications team they would "get used to having my eyes open wide in horror and terror on a regular basis."

"I'd get used to getting more blindsided on a regular basis and not really knowing what the heck is going on and understanding that you are not going to be able to message the good policy things you are doing," the strategist said. "It's really the President taking over. It's the President saying you know what I'm going to do it my way."

"You can plan and plan and plan your short-term and long-term communications strategies, but when you are faced with the reality of this President's mode of operating you might as well just throw the plans out the window," the source added.

Though he is not an official government employee, Giuliani is now being introduced to a similar treatment that other White House staffers -- and former favorites of Trump's -- have faced in the past, where your rank with the President depends on the outcome of the news cycle.

Giuliani's diminished influence on Trump has shown. On Sunday, he told The Washington Post that he and Trump had struck a deal for the President to stay focused on "North Korea, Iran and China" while Giuliani handled the Mueller probe. The next day, Trump tweeted nearly a dozen times. Only two were about the Iran deal.

North Korea remarks

Among the President's national security aides, Giuliani's willingness to freely discuss ongoing negotiations on the Iran deal and US prisoners in North Korea was viewed as deeply problematic, according to multiple officials and others familiar with the situation. Aside from concern over the negotiations themselves, the officials said Giuliani's statements could lead to confusion over who is speaking on behalf of the US government.

One official attributed Giuliani's free-wheeling comments to a misunderstanding of his role -- which is no longer as a "principal," but as a surrogate for Trump. Another official said without any internal White House coordination over Giuliani's television appearances, there was little hope of providing him with accurate information. Giuliani does not work at the White House and does not have a security clearance.

His insistence on Fox News last week that the American detainees would be released within the day was not based on US intelligence or conversations with the President, according to sources. Instead, Giuliani was relaying a loosely sourced story that had been repeated on conservative media outlets with no confirmation from the White House.

US officials have been negotiating the release of the three detained Americans, and it's widely expected they'll go free ahead of Trump's upcoming summit with Kim Jong Un. Giuliani's remarks were viewed inside the White House as detrimental to those efforts, the officials said, since it upped the stakes for the President.

"This isn't how we should be doing this," sighed one US official after Giuliani's appearance on Fox. The official said the administration had hoped any announcement about the Americans' release would come from the President himself.

Officials inside the West Wing, who have been exasperated by Giuliani's string of brazen cable news appearances, have joked among themselves that he has unofficially replaced Michael Anton, the National Security Council spokesman who quit last month.

The former New York City mayor also suggested over the weekend that Trump was in favor of regime change in Iran, which runs counter to official US policy. Officials said the statement came as a surprise and were concerned Giuliani may inadvertently lead people to believe he speaks on behalf of the United States.

On Monday, the State Department made clear he does not.

"He speaks for himself and not on behalf of the administration on foreign policy," Heather Nauert, the State Department spokeswoman, said.

CNN's Jeremy Diamond and Gloria Borger contributed to this report.