(CNN) Andrew Puzder has withdrawn as President Donald Trump's choice for labor secretary, a dramatic decision caused by a GOP revolt that claimed the first Cabinet nominee of the new administration.

Puzder, the CEO of the company that owns the Hardee's and Carl's Jr. fast food chains, faced fierce opposition mostly from Democrats in part related to his position on labor issues as well as the fact that he employed an undocumented immigrant housekeeper.

But Republicans too had grown weary of the range of liabilities facing Puzder, and senior GOP officials informed the White House Tuesday night and Wednesday that Puzder lacked a viable path for confirmation.

"After careful consideration and discussions with my family, I am withdrawing my nomination for Secretary of Labor," Puzder said in a statement Wednesday afternoon. "I am honored to have been considered by President Donald Trump to lead the Department of Labor and put America's workers and businesses back on a path to sustainable prosperity.

"I also thank my family and my many supporters -- employees, businesses, friends and people who have voiced their praise and hopeful optimism for the policies and new thinking I would have brought to America as Secretary of Labor. While I won't be serving in the administration, I fully support the President and his highly qualified team."

The decision came as Senate Republicans told the White House he was losing support, a senior GOP source said, adding there were four firm Republican no votes and possibly up to 12.

Sen. Tim Scott, of South Carolina, was one of those Republicans who planned to vote against Puzder, saying he was worried about revelations involving Puzder paying employees in cash and the nominee's comments about some of the American workforce.

In a statement Wednesday afternoon, Scott said he shared those concerns with Senate leadership and Tennessee's Lamar Alexander, chairman of the HELP Committee.

"I look forward to meeting with a new nominee to lead the Department of Labor," Scott said in a statement.

Sen. Johnny Isakson, a Republican from Georgia who was one of the firm "no" votes on Puzder, offered a briefer statement. "I respect Mr. Puzder's decision."

A third senator in the group, Alaska's Lisa Murkowski, said she respects Puzder's decision to withdraw. "This has been a difficult process for him and his family," she said.

Alexander said he was still planning to vote in favor of Puzder before he dropped out. Asked if he felt any of the concerns raised about Puzder were warranted, Alexendar appeared to address Puzder's hiring of the undocumented immigrant.

"He made a mistake," Alexander said. "And in my view, he discovered it, he reported it, he took responsibility for it. He corrected it. And that's about all you can do with a mistake. So I evaluate him and his whole life's work, so I didn't think that mistake should disqualify him from being a cabinet member."

His team subsequently began to pull back from all their plans for the confirmation process. Puzder was scheduled to do extensive preparation Wednesday afternoon for his confirmation hearing before the Senate on Thursday, but canceled the so-called "murder board" after top Republicans spoke to the White House, a source close to Puzder told CNN.

Around the same time, sources on Capitol Hill were told Puzder was expected to cancel his confirmation hearing. And Puzder's team told the White House that he did not want to go forward with the nomination if there was no clear path forward.

White House press secretary Sean Spicer confirmed Puzder's decision, telling reporters simply, "he withdrew."

Senate Democratic Leader Chuck Schumer fanned the flames behind the opposition to Puzder in a statement Wednesday afternoon.

"No matter how you cut it, there is no worse pick for labor secretary than Andrew Puzder, and I'm encouraged my Republican colleagues are starting to agree," the New York Democrat said. "He does not belong anywhere near the Labor Department, let alone at the head of it. Puzder's disdain for the American worker, the very people he would be responsible for protecting, is second to none."

Fingerpointing among Republicans

The fingerpointing has already begun among Republicans.

GOP sources are privately pointing the finger at business groups for not giving political cover to Republicans. They say that they were swamped by labor groups and progressive forces looking to derail Puzder, while the business community sat on the sidelines.

A source said the unions have eaten "their lunch."

"They literally were raising money in like $5,000 increments when millions were being spent against them," one top GOP source says. "His entire support network made one TV ad that looked like a welcome video for new Hardee's employees, which referred to "her" and "she" while praising the boss who hired them. Everyone knew he was in trouble weeks ago and even his strongest supporters and biggest beneficiaries of his point of view as labor nominee would not open their wallets."

Ran into problems immediately

Puzder faced intense criticism since his appointment on December 8. Current and former employees spoke out against him from the start, and Democrats and labor activists attacked his record on worker rights.

The fast food executive has a long history of fighting against government regulation, a $15 minimum wage and the Affordable Care Act.

Decades ago, Puzder's ex-wife, Lisa Fierstein, levied charges of domestic abuse against him, which she later dropped. She also appeared in disguise on Oprah Winfrey's TV show to talk about domestic violence in 1990. In advance of Puzder's scheduled hearing, senators privately reviewed the tape.

During the episode, Fierstein said Puzder told her, "I will see you in the gutter. This will never be over. You will pay for this," according to Politico

Senators were shown the video in the lead up to Puzder's nomination fight.

In a letter to senators obtained by CNN , Fierstein said she regretted accusing Puzder of abuse during a rocky marriage.

Puzder has denied he abused his wife.

That hasn't been the only issue. Last week, he admitted to having employed an undocumented immigrant as a housekeeper. He later said that he and his wife let the housekeeper go when they learned of her immigration status, and he said they paid back taxes on the employee and offered her help getting legal status.

Puzder's own employees were some of the first to cry foul when he was nominated.

Last month, current and former workers at Hardee's and Carl's Jr. announced that they had filed 33 state and federal complaints against their employers. Charges include wage theft, manipulated overtime, sexual harassment and unfair labor practices.

All the complaints pertained to franchised locations that are independently operated. But the workers said they reflected poorly on a chief executive who hoped to lead the Labor Department.

A spokesman for CKE Restaurants said the company does not comment on pending litigation. The company also declined to comment on whether Puzder will stay on as CEO now that he has rescinded his nomination.

Puzder once called some of his Hardee's employees "the worst of the worst," according to reporting by CNN's KFile . He was also criticized for past statements in support of replacing workers with robots.

"They're always polite, they always upsell, they never take a vacation, they never show up late, there's never a slip-and-fall, or an age, sex, or race discrimination case," Puzder told Business Insider in March 2016.

It took weeks for senators to get Puzder's financial disclosure information. He had promised to step down as CEO of the company, CKE Restaurants, if confirmed, and to divest his stake in 200 companies and funds.