House Speaker Paul Ryan told a room of Republican donors and elites in Park City, Utah last weekend that he endorsed Donald Trump to keep the Republican Party together.

That explanation implied that if the highest ranking Republican didn’t support the top of the ticket and went against the legitimate winner of the primary electoral process that the Republican Party would limp forward as a divided party with no chance of electoral success. The Speaker suggested that he put party above personal preference for the sake of unity.

Just days later, party unity appears even more fragile.

Since wrapping up the GOP nomination, Trump has done little to ease the concerns of skeptical Republicans. He has ignored calls to change his tone, refocus his message on policy and ignore personal slights. Instead, he has doubled down on a controversial proposal to ban on Muslims entering the U.S., attacked the heritage of a federal judge and publicly insulted party leaders.

In turn, Republican office holders are keeping their distance – even some who, like Ryan, have endorsed him as the party’s standard-bearer for the November election.

Intraparty fractures deepened Monday after Trump doubled down on his proposal to ban Muslims from entering the U.S. His response to the deadly massacre of 49 people at a LGBT night club in Orlando has been panned or ignored by Republicans.

The speaker of the House told reporters on Capitol Hill Tuesday that he disagreed with Trump’s proposal, saying, “I do not think a Muslim ban is in our country’s best interest.”

When Ryan was asked about it again later in the day, he demurred, saying he will not respond to the machinations of the presidential campaign on a daily basis.

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

Related: Clinton Calls Out Trump's 'Lies' After Orlando Speech

On the other side of the Capitol, Senate Majority Leader Mitch McConnell refused to comment, and avoided even saying Trump’s name.

“I’m not going to be commenting on the presidential candidate today,” McConnell told reporters during his weekly Capitol Hill briefing.

Other than Sen. Jeff Sessions of Alabama, who is a close adviser to Trump, no response about Trump dominated Congressional responses. It was as if his speech didn't happen. Their silence illuminated their unease with their party leader.

The Republican National Convention, which is getting behind Trump, ignored his Orlando speech, too. The RNC sent out a press release attacking presumptive Democratic nominee Hillary Clinton’s refusal to use the term “radical Islam.” And after President Obama used his bully pulpit Tuesday to criticize Trump and his response to the Orlando killings, the RNC didn’t come to Trump's defense. Instead, they focused on Obama and Clinton, saying they want to “take away our (gun) rights.” (Also, Trump is absent from the homepage of the RNC's website.)

Tennessee Sen. Bob Corker, an early supporter of Trump, was one of the few who did respond, but not to Trump's benefit. After saying last week that Trump needs to change his tone because of ongoing attacks against federal Judge Gonzalo Curiel and his Mexican heritage, Corker's criticism was even stronger Tuesday when reporters asked about Trump's Orlando speech.

Related: Fact-Checking Donald Trump's Orlando Speech

"I continue to be discouraged by the direction of the campaign and comments that are made. And I did not think yesterday’s speech was the type of speech that one would give who wants to lead this country through difficult times," Corker told reporters Tuesday.

Division within the party goes far beyond Trump’s Muslim ban and beyond the Republican establishment. The latest NBC News/Survey Monkey Weekly Tracking Poll, which was taken amidst the firestorm over Judge Curiel, shows that Trump is losing support.

Even before Orlando, it was becoming clear that party unity is in peril as discord was on full display at an annual ideas summit in Park City hosted by former Republican presidential candidate Mitt Romney over the weekend. Trump was the political focus of the attendees and not Clinton, the expected target at a confab of politically active Republicans.

Some attendees, who combined have given tens of millions of dollars to Republicans in recent elections, said they couldn’t, at this time, get behind Trump.

“If you ask me today what I’ll do in November,” Lanhee Chen, former policy director to Romney said, “I’ll probably write someone in.”

Meg Whitman, CEO of Hewlett Packard, publicly challenged Ryan over his endorsement in a room of about 250 summit participants. Whitman, who contributed money to the “Stop Trump” movement in the primaries, asked Ryan how he could support a demagogue like Trump, comparing him to Hitler and Mussolini.

Romney didn’t hold back when talking about Trump, either, indicating that while he “respects” Republicans who say they’re going to support the presumptive nominee in November, he has no plans to do so. He called him a misogynist, a racist and a xenophobe and that Trump’s antics are “breaking his heart.” Those aren’t words of a party marching forward in unity.

It’s not just Trump’s policies that are dividing the party, it’s also his temperament. Trump, unable to ignore criticism, used his campaign rally to attack Romney, calling him a “choker” and a “loser.”

And After Ohio Gov. John Kasich told Fox News last month that he has no plans to endorse Trump, Trump suggested Kasich should leave the GOP. It’s a politically risky move on Trump’s party since Kasich is the governor of a critical state in the presidential election. No Republican has won the presidency without winning Ohio in modern-era politics, and the Buckeye State is hosting the Republican National Convention.

Showing more signs of disunity, New Hampshire Sen. Kelly Ayotte, who is facing a tough re-election campaign, did not attend Trump’s speech Monday at St. Anselm College in the Granite State. The other Republican member of New Hampshire’s Congressional delegation, Rep. Frank Guinta, did not attend either.

At Trump’s campaign speech in North Carolina Tuesday night, Congressional allies will also be absent as no Republican members of North Carolina delegation are expected to attend.

While Trump is running as a Republican, he often separates himself from the party, referring to the party as “they” instead of the pronoun most people use - “we” - when part of an organization.

“They have to get their act together,” Trump said at a campaign rally in Tampa Saturday, after calling on the GOP to unify. “The Republican Party has to come together.”