Donald Trump on Tuesday faced a barrage of criticism from within his own party about what many called "racist" comments about the judge in the Trump University lawsuit, prompting Trump to scramble to contain the uprising just when he should have been free and clear to focus his energy on Hillary Clinton.

And while the 2016 presidential race hangs in the balance, Republicans in Congress wasted no time on their first full day back in Washington that there's a limit to what Trump can say and still represent the party.

The uproar was sparked by Trump's insistence that a U.S. judge of Mexican heritage must recuse himself from a Trump University lawsuit, because of his possible bias against the candidate who has pledged to wall off Mexico. After a weekend in which both parties slammed Trump for the remark, and for calling him a "Mexican" judge, two senators said they were done with Trump completely.

"This is the most un-American thing from a politician since Joe McCarthy," Sen. Lindsey Graham, R-S.C., told the New York Times. "If anybody was looking for an off-ramp, this is probably it.

"I have spent my life building bridges and tearing down barriers — not building walls," Sen. Mark Kirk, R-Ill., said in a statement. "That's why I find Donald Trump's belief that an American-born judge of Mexican descent is incapable of fairly presiding over his case is not only dead wrong, it is un-American."

Leaders in both the House and Senate also spoke up. House Speaker Paul Ryan, R-Wis., said Trump's comment was the " textbook definition of a racist comment," and Senate Majority Leader Mitch McConnell, R-Ky., said it was time for Trump to get on message.

"My advice to our nominee would be to start talking about the issues that the American people care about and to start doing it now," McConnell said. "In addition to that, it's time to quit attacking the various people you competed with or the various minority groups in the country and get on message."

Trump has excelled at making controversial comments and winning the aftermath, but the combined weight of the Republican lawmakers forced Trump to respond by the end of the day. Trump didn't apologize, but did say his comments were "misconstrued," and tried to set the record straight in the face of heavy criticism.

"I do not feel that one's heritage makes them incapable of being impartial, but, based on the rulings that I have received in the Trump University civil case, I feel justified in questioning whether I am receiving a fair trial," Trump said.

Trump tried to end it there, but it's not clear it will be that easy. Lawmakers on Tuesday made it clear they were exasperated as they tried to reconcile their support for the GOP nominee, and the need to protect their own reputations.

"I think all of us were disappointed with what he said," Sen. Mike Rounds, R-S.D., told reporters Tuesday.

"His comments are not helping," Sen. Shelley Moore Capito, R-W.Va., said when asked about Trump's effort to keep the judge off his case.

Few GOP lawmakers were saying Trump should drop out, and indicated a grudging need to support the nominee in order to fulfill key Republican objectives, including filling the vacant Supreme Court seat. Republicans also say they want to implement tax reform, healthcare reform and other big policy initiatives next year, and they can't do that if a Democrat wins the White House.

"I think as Republicans, we have a better chance of making the changes at the federal level that we have to with Mr. Trump as opposed to Secretary Clinton," Rounds said.

But GOP senators did not hesitate to condemn Trump's remarks. Sen. Susan Collins, R-Maine, has withheld her support of Trump so far and his latest remarks make her more hesitant, she said.

"It is a deeply troubling comment on so many different levels, and it demonstrated such a lack of respect for the role of federal judges in our system that I find it stunning and appalling," Collins said. "I hope Donald Trump will walk back those comments and apologize to the judge and apologize to the American people."

Rounds said, "I think everybody is frustrated because we are all worried about making the country better." He added that Trump's comments distract from that goal.

Sen. Richard Burr, R-N.C., said he is not willing to drop his support of Trump, despite the urging of Sen. Lindsey Graham, R-S.C.

"Listen, you are asking me a question nobody has an answer to," Burr said when a reporter asked whether Trump has crossed a line with Senate Republicans. "This is one of those things where you'll know it when you see it. But I don't think he'll go there. I don't think he'll get there. You can't forget the fact that he has won the election. He's got the support to win the nomination and right now he runs extremely well nationally."

Reporters asked Burr if he is "frustrated" with Trump and he responded, "I'm frustrated because I think there is a tremendous opportunity being missed."