Republican Sen. Bill Cassidy is drawing a distinction between sexual assault accusations against conservative Roy Moore and liberal Al Franken.

"I can't get inside Al Franken's mind but he was clearly drummed out. He did not have to quit" the Senate, Cassidy told CNBC's "Squawk Box" on Monday. "There was not due process. Whatever you think about it, you have to have due process in our country. You can't have kangaroo courts. You have to allow people to be presented with evidence to refute it."

But that said, Cassidy last month withdrew his support of Moore, 70, the Republican candidate running against Democrat Doug Jones in Tuesday's special election to fill Alabama's open Senate seat.

As of Sunday, the RealClearPolitics average of major polls still gives Moore, a former Alabama Supreme Court judge, a narrow lead. But the latest Fox News poll shows Jones a full 10 points ahead.

Cassidy of Louisiana has said it's up to Alabama voters, but he explained there's a difference between the allegations against Moore and the allegations against Franken because the accusers of Moore claimed he preyed on them while they were underage.

Moore faces allegations of sexual misconduct with teenage girls, including a woman who claims that he sexually assaulted her nearly 40 years ago when she was 14. Moore has vehemently denied all the allegations, calling them a smear campaign by his opponents.

Meanwhile, Franken, 66, announced he would resign "in the coming weeks" from the Senate floor last week, following multiple women accusing him of groping or forcibly trying to kiss them. The Minnesota Democrat and former comedian has expressed remorse and publicly denied most of the allegations.

"If Al Franken has been involved in this kind of activity as a senator that's problematic," Cassidy told Fox radio last week. "On the other hand, there is a difference between a 14-year-old girl and an adult female, I will say that."

However, if Moore wins in Alabama "the Constitution says he has to be seated," Cassidy said, though he did not mention on CNBC whether or not he thought Moore would or should face an ethics investigation.