Washington (CNN) Sen. Lindsey Graham said it's time for Republicans to rebuke presidential hopeful Donald Trump, urging his party to tell Trump to "go to hell."

"You know how you make America great again? Tell Donald Trump to go to hell," Graham said on CNN's "New Day" on Tuesday, picking up on the GOP front-runner's famous slogan, "make America great again."

The South Carolina Republican was responding to Trump's statement released Monday calling for the U.S. to ban all Muslims from entering the country.

"He's a race-baiting, xenophobic, religious bigot," Graham told Alisyn Camerota. "He doesn't represent my party. He doesn't represents the values that the men and women who wear the uniform are fighting for. ... He's the ISIL man of the year."

Graham said Trump's rhetoric benefits ISIS in helping them recruit people to their cause. He said having traveled to the Middle East 36 times as a lawmaker and in the Air Force reserve, he knows the troops and diplomats on the front lines are very concerned.

"What Mr. Trump is doing -- and I don't think he has a clue about anything. He's just just trying to get his numbers up and get the biggest reaction he can," Graham said. "He is helping the enemy of this nation. He is empowering radical Islam. And if he knew anything about the world at all, you would know that most Muslims reject this ideology."

Graham said that what the U.S. needs to do is embrace the "99%" of Muslims who reject radical, violent extremism and have died "by the thousands" trying to fight it, and it needs to invest in the Middle East and giving people and women opportunity there.

"That's how you win the war. A hopeful life versus a glorious death. And what Mr. Trump is doing is undercutting everything we stand for," Graham said. "Going to a military high school, Donald, is not military service. You've never worn the uniform. ... So knock it off. You're putting people at risk."

Trump has been leading the Republican primary in nearly every poll, recently hitting 36% in a CNN survey . Graham has been languishing at the bottom of the race, failing to hit even 1% in some polling.

The senator has become increasingly outspoken about the direction his party is headed, trying to position himself as the voice of reason even as his campaign struggles.

Graham said it's a "dirty little secret" of American history that there has always been an appetite for bigotry and exclusion, but a presidential candidate should "bring us together."

Calling out Cruz, Party

Graham also called out other presidential candidates for not being stronger against Trump, saying the entire party needs to speak out against the mogul.

He specifically spotlighted his fellow GOP presidential candidate, Texas Sen. Ted Cruz, for not condemning Trump strongly enough.

"This is not a policy debate, Ted," Graham said.

On Monday, Cruz said Trump's positions are not his own when asked about the call to ban Muslims and described his own proposals to place a moratorium on refugees entering the country from areas with strong footholds for terrorists.

Graham said it was time for the party to reject Trump even if it hurts them in the long run.

Trump has left the door open to mounting an independent bid for president if he doesn't win the Republican nomination, a possibility that spurred the Republican National Committee to have each candidate sign a pledge to support any eventual Republican nominee.

"I'd rather lose without Donald Trump than try to win with with him," Graham said. "I wish he would leave the party. I don't care if he runs as an independent. If we lose the 2016 election. So be it. ... I want to be in a category of 1% who said 'B.S., this is not who we are at a party, this is not who we are as a nation.' And I'm calling on Ted Cruz, you're trying to have it both ways here."

It's the second time in less than a week that Graham has had harsh words for Cruz, who he also criticized along with Trump at the Republican Jewish Coalition presidential forum in Washington, last week for being too hardline on immigration.

Graham said Cruz is trying to pick up Trump supporters for his own campaign and is ignoring a "moral imperative" to speak out against the rhetoric.

"He's trying to get all the Trump people when Trump falls. He's trying to get all the Carson people when they fall," Graham said. "What I'm trying to do is be a president worthy of a great nation and a great party. So this doesn't cut it for me. This is not a policy debate, Ted. This is about you and us and our character as a party. Up your game. Condemn it, because it needs to be condemned."