House Speaker Paul Ryan visited the White House on Friday to reportedly tell the president the embattled Republican Obamacare replacement bill lacked the votes to pass, setting up a potentially serious blow to the key legislative goal.

Ryan and House leaders scrambled Friday to gather votes for the proposal ahead of a planned vote, which President Donald Trump demanded despite doubts that it could pass. The party has struggled to win over both its conservative and moderate wings for a plan that has proven unpopular so far in opinion polling.

At the White House, Ryan said he did not have the support to push the divisive plan through, according to The New York Times and CNN. Press secretary Sean Spicer said the pair "discussed the way forward," adding that he expects a House vote on the bill around 3:30 p.m. ET.

At least 34 Republicans have said they will vote "no," or are leaning toward voting "no," on the legislation, enough to doom the plan's passage, according to an NBC News tally. That figure could change throughout the day.

Major U.S. stock indexes gave back some of their gains Friday afternoon as concerns grew that the House would approve the bill. Market observers have watched the vote closely, as Trump has said he cannot move on to other parts of his agenda, particularly tax reform, until the GOP passes health-care legislation.

The White House said Trump would move on to other priorities if the vote fails, leaving Obamacare in place.

Lawmakers are debating the plan, called the American Health Care Act, on the House floor before a planned vote later in the day. The bill cleared a key procedural hurdle earlier Friday.

Trump shrugged Friday when asked what would he would do next if the embattled bill fails in Congress.

After shrugging, Trump said: "We'll have to see."

Spicer sidestepped questions about whether Trump thinks the bill will pass, saying the president "is confident that we've done every single thing possible" to ensure its success.

The chances of losing the vote seemed to increase right after Trump spoke when the Republican chairman of the House Appropriations Committee, Rep. Rodney Frelinghuysen of New Jersey, revealed he will vote against the plan.

Republican leaders postponed the House's scheduled vote for Thursday after it became clear the bill would fail. They have tried to balance concerns of both the party's conservative and moderate wings.

Leaders then offered an amendment that would allow states to determine what kind of so-called essential health benefits would be included in health plans, as opposed to the federal government mandating what benefits have to be included.

That change was part of the reason that Frelinghuysen declared himself as a "no" vote on Friday.

Another key moderate Republican — Rep. Barbara Comstock of Virginia — reportedly will vote "no" on the legislation.

— CNBC's Eamon Javers contributed to this report