Republican lawmakers’ call to replace Donald Trump as their presidential nominee after the release of a damaging audiotape grew Sunday with Utah Sen. Mike Lee saying “there’s still time,” while Trump’s bare-knuckled effort to survive got unintended help from rival Democrats.

Donna Brazile, the interim Democratic National Committee chairwoman, told ABC’s “This Week” that attempts to change November ballots would be "very confusing" to voters and suggested that Democratic Party lawyers would probably fight efforts by states to change the names on the ballot.

Brazile also pointed out that some states have already begun early voting and have mailed absentee ballots with Trump's name on them.

Lee was among the first and most prominent Republicans to call for Trump to quit the race, after the release Friday of the audiotape in which Trump is heard making lewd comments about women.

Lee on Sunday told NBC’s “Meet the Press” said Trump’s “last legacy” should be stepping aside so the party can find a candidate to defeat Democratic rival Hillary Clinton.

“That’s what he has to do,” said Lee, one of the most conservative congressional Republicans. “We’ve got a lot of other candidates that can do this. There’s still time.”

The 2005 audiotape, and an accompanying video tape, released by The Washington Post and NBC, recorded a conversation between Trump and then-"Access Hollywood" host Billy Bush in which Trump described an attempt to have sex with a married woman.

Trump also brags about women letting him kiss and grab them because he is famous.

Political experts say the Republican National Committee can replace a candidate who wins the nomination at the national convention only in extreme circumstances -- like death or if the candidate refuses the nomination.

Trump on Sunday lashed out at the growing list of Republicans abandoning his candidacy, predicting that they're the ones who will lose.

"So many self-righteous hypocrites,” he tweeted. “Watch their poll numbers -- and elections -- go down!"

On Saturday, Trump defiantly vowed to stay in the race -- high-fiving supporters outside Trump Tower and telling fellow Republicans and other detractors on Twitter that he “will never let my supporters.”

The 70-year-old Trump has apologized twice for the comments, saying they “don’t reflect who I am.”

“I said it, I was wrong, and I apologize," Trump said in a video apology, with Election Day about four weeks away and a critical second debate with Clinton scheduled for Sunday.

Also on Sunday, freshman GOP Sen. Kelly Ayotte, in a tough reelection bid in New Hampshire, said she favors Trump quitting the race -- a step beyond her saying Saturday that she wouldn’t vote for him.

Ayotte said Trump’s remarks on the recordings are "fundamentally different" than past, offensive comments. She said Trump in the tape released Friday is advocating assault and that she wants her young daughter to know that she does not support the comments.

Rep. Charlie Dent, R-Pa., chairman of the House Ethics Committee, also said Sunday that his party still has enough time to rally behind an alternative to Trump, with running-mate Indiana Gov. Mike Pence among the names mentioned.

Dent told ABC that he wants House Speaker Paul Ryan and Republican National Committee Chairman Reince Priebus to withdraw their endorsements of Trump, who trails Clinton by 4.6 percentage points, according to the RealClearPolitics polls average.

Dent listed as possible alternatives Robert Gates, defense secretary under Presidents George W. Bush and Barack Obama; Condoleezza Rice, secretary of state under Bush; Ohio Gov. John Kasich, who sought the GOP presidential nomination; or Mitch Daniels, a former Indiana governor.

Fox News' Joseph Weber and The Associated Press contributed to this report.