Washington (CNN) The Republican Party's reputation was on the line, but a group of moderates were prepared to defy not only House Speaker Paul Ryan but President Donald Trump on the GOP's first major legislative priority to repeal Obamacare.

In the end, they didn't have to vote at all.

Just hours before the bill to repeal and replace the Affordable Care Act was pulled from the floor Friday, moderates were still coming out to publicly oppose it, a sign to leadership that some of their more loyal foot soldiers wouldn't be with them in the end.

Newly-minted House Appropriations Chairman Rodney Frelinghuysen's said on Facebook he was against the bill. Rep. Barbara Comstock, a party stalwart, also announced she was opposed Friday.

As it became clearer throughout the day that House leaders didn't have the votes, moderates become less and less willing to walk the plank for a bill they feared would die in the House or be killed in the Senate.

Changes appealed to conservatives, and scared moderates

For some moderates, the last-minute deal hatched between the White House and the conservative House Freedom Caucus Wednesday night to repeal the essential benefits that insurers are required to offer consumers had gone too far.

One GOP source told CNN that while leaders had gotten a few firm commitments coming from conservatives, they'd seen an exodus of moderates in part because of the deal.

For moderates in tough districts back home, the campaign ads would have written themselves. The overhauled GOP repeal bill included a provision that stripped away federal requirements that insurers provide benefits like maternity care, prescription drug coverage and hospital stays and delegated those decisions to the states.

Rep. Leonard Lance, a New Jersey Republican, told CNN Friday morning that without a doubt, the decision to concede the repeal of essential health benefits to the Freedom Caucus had moved some of his colleagues to "no."

"I suspect some became a 'no' because of that," Lance said. "That certainly didn't help."

While so much of the focus over the last week had centered on the louder, more visible House Freedom Caucus, the fear that moderates would abandon House leaders en masse had been a top concern behind the scenes.

Rep. Charlie Dent, a consistent ally for leaders, came out against the bill after a tense meeting in Ryan's office Wednesday night and there were more members who never said publicly where they stood on the bill.

Because Republicans chose to make the change to the benefits so late in the game, moderates feared that they were walking in blind to what the effects of repealing them could be.

"A lot of people don't realize what the implications of that are," one moderate member said of stripping out essential health benefits Thursday. "So we're going to railroad this thing through and there's going to be even more people pissed off -- our constituents, stakeholders."

Members would have been voting without an analysis from the non-partisan Congressional Budget Office of what the budgetary and coverage impact of repealing essential health benefits would have been and moderates had already been spooked by the CBO analysis the first time around, which showed 24 million more people would have been without insurance by 2026 if the GOP leadership's bill had passed.

On Friday morning, just off the House floor, Rep. Mark Amodei, a Nevada Republican who announced his opposition to the bill Thursday, said he had felt "calm" about his plan to vote against the bill. No phone call, leadership pep talk nor Trump strong-arming would have convinced him to get to "yes."

If his voters disagreed with him at the ballot box in 2018, so be it.

"I don't know if they will or they won't. That book is still to be written. But obviously, I'm willing to take that risk," Amodei said. "Listen, I'm taking my refuge in the impacts in my district in Nevada."

Amodei added that he didn't understand the rush to jam the bill through so quickly.

"I don't know that there's a time limit on it," he said. "The same way that some people here want to say it's got to be done by the anniversary date of signing it. Really? Are we in the Hallmark Card Business? Come on."

Ryan met with his rank and file Friday afternoon. A somber line of members trod into the basement conference room where less than 24 hours before they'd been implored one last time to vote for the bill.

They didn't have the votes, Ryan told them.

Moderates who'd been on the fence would never have to decide what to do and those moderates who'd come out as a 'yes' wondered about what would happen to them now.

"I'm really for it," Rep. Mike Coffman, a Republican from Colorado told CNN.

Asked if his public "yes on a bill that never made it to the floor could hurt him, he was frank.

"I have no idea ultimately," he said adding he didn't think it would. "I have a certain style of politics that seems to work for my district. I have a district that Hillary Clinton won by 9 points and I won by 8.5."