Washington (CNN) South Carolina Sen. Lindsey Graham blasted President Donald Trump, saying Tuesday he was undermining confidence in both the American democratic system and himself with the continued claims of voter fraud.

"I wasn't there, but if the President of the United States is claiming that 3.5 million people voted illegally, that shakes confidence in our democracy — he needs to disclose why he believes that," Graham told CNN on Tuesday.

Trump has offered no evidence of the alleged voter fraud, and his claim, originally made two months ago, has been repeatedly debunked.

Graham has been one of staunchest Trump critics in the Senate but recently announced he would support Trump's pick for secretary of state, an olive branch to the new President.

Graham cautioned Tuesday that the continued harping from Trump would ultimately undermine his own support among the public.

"I would urge the President to knock this off; this is the greatest democracy on Earth, we're the leader of the free world, and people are going to start doubting you as a person if you keep making accusations against our electoral system without justification," Graham said. "This is going to erode his ability to govern this country if he does not stop it."

Sen. Jeff Flake, an Arizona Republican who scrapped with Trump during last year's election, also chastised Trump Tuesday for reviving the issue.

"That doesn't do anybody any good. That doesn't help him. That doesn't help any of us," Flake said.

Rep. Will Hurd, a Texas Republican, said he saw no evidence of widespread voter fraud.

"I would suggest the White House looks in their windshield and not the rearview mirror. They did something that nobody thought they could do. They should be proud (of) that and not let this be a distraction," Hurd told CNN's Wolf Blitzer on "The Situation Room."

Republican lawmakers dodge another Trump flare-up

Graham's candid broadside stood out Tuesday, as other top Republicans -- some apparently exasperated -- ducked questions on Trump's latest comment.

Sen. John Thune, R-South Dakota, repeatedly said "I don't know the answer," when asked how Trump came up with his numbers. Sen. Tim Scott, R-South Carolina, would only say "it's not important to me" when asked about Trump's comments. And Sen. Marco Rubio, R-Florida, said Trump's claim was "distracting (journalists,) not me."

Senate Majority Leader Mitch McConnell, who was at Monday's dinner at the White House when Trump revived the allegation, deflected a question by saying that voter fraud is real and happens in his home state of Kentucky.

Schumer reacts

Senate Democratic Leader Chuck Schumer, meanwhile, blasted Trump first, and later accused Senate Republicans of not doing enough to deal with the President's wild statements.

"The President ought to realize he's president. Instead of talking about the election, or how many people showed up at the inauguration, he ought to talk about how many new jobs he's created. It's time to be president," Schumer said.

Schumer then pointed the target at Republicans.

"When these falsehoods are told, our Republican colleagues have an obligation to reject them. Not to skirt around them," he said. "The bottom line is simple: you cannot run a government, you cannot help people, you cannot keep America safe if you do not actually admit to the facts."

House Speaker Paul Ryan -- who has previously fought back against Trump's allegations of voter fraud -- largely sidestepped a question about his thoughts when Trump brought the claim up again in their meeting Monday.

"I've already commented on that, I've seen no evidence to that effect. I've made that very, very clear," Ryan said Tuesday.

Trump and Vice President Mike Pence are scheduled to meet with Graham, Ryan and all of the House and Senate Republicans later this week in Philadelphia for their annual retreat.