Washington (CNN) The House Freedom Caucus -- a group of roughly 40 conservatives -- was a constant annoyance for Speaker John Boehner. They complained Boehner was ignoring the GOP base and constantly capitulating to President Barack Obama and Hill Democrats. One of their members even kick-started the series of events that prompted Boehner to abruptly quit the speakership.

Life for the next speaker won't be any easier.

The conservative lawmakers have the potential to make or break the candidacy of California Republican Rep. Kevin McCarthy or any other would-be speaker. And they know it.

Members are privately discussing a list of demands to make to McCarthy. Conservatives want slots on top committees, assurances from leaders they will get input on bills before they go to the floor, and ‎a promise Republicans will follow through on pledges to dismantle President Barack Obama's policies.

One thing the Freedom Caucus doesn't have: a candidate. So far, none of the roughly 40 conservatives in the group are running to replace Boehner -- or for any other leadership position. And the group has a reputation of not always being as effective as it could be.

"I think most people in the Freedom Caucus realize that one of us would probably be the longest of longshots and so our hope is to work with someone who can actually win to accomplish the agenda we talked about," Rep. Matt Salmon, R-Arizona, a founding member of the Freedom Caucus, told reporters on Tuesday.

Even if they don't get everything they want from McCarthy now, as a voting bloc, they have the potential to torpedo government funding bills, a measure to raise the debt ceiling, or other major must-do items this fall. If McCarthy tries to use Democratic votes to pass bills -- as Boehner had to do at times -- he runs the risk of facing the same uprising that felled the Ohio Republican.

"The same people who were after John Boehner are going to try and fry the next guy," said Rep. Charlie Dent, a moderate Republican from Pennsylvania, who has openly expressed frustration with the power of the conservative bloc over House leaders.

Freedom Caucus member Rep. Raul Labrador, R-Idaho, said the group is focused on the big picture. "Right now the election is not about who's going to be the next speaker of the House," Labrador said Tuesday night. "It really should be about what we're going to do to change the process here, so everybody can feel empowered in the process, and that was the beginning of the conversation."

McCarthy reaching out

Despite being an overwhelming favorite to win the speakership, McCarthy is not taking any votes for granted. If McCarthy loses more than 29 votes on the floor, it could put him under the critical 218 votes to win the speaker's gavel, throwing the GOP leadership race into turmoil.

So immediately after Boehner announced his resignation Friday, McCarthy was on the phone with members of the group. He called up Rep. Mo Brooks on Saturday, but the Alabama Republican was watching college football so he didn't take the call. So McCarthy set up a Tuesday 30-minute sit-down in his suite just off the House floor. The wide-ranging discussion was constructive, Brooks said, but it didn't sell him on McCarthy -- at least not yet.

"I am in a very conservative district from the state of Alabama," said Brooks. "I want things done that will make the people of my district feel more comfortable about him being speaker of the House, given that his voting record, although perhaps arguably conservative in the whole political spectrum, is somewhat less-so if you look at the Republican philosophical spectrum."

Speaking to reporters Tuesday, McCarthy said he was different than Boehner because of his own "bottom-up style" of leadership. And he also added: "I won't be as tan."

What the Freedom Caucus wants

The group was formed earlier this year after breaking off from the Republican Study Committee, another conservative caucus, because its founders wanted to take an even harder line against their party's leadership. While they have frequently been sought to undercut their party's agenda, they have often lacked a clear strategy on how to achieve their goals.

Since the group was formed in early January it has been deeply divided on strategy. Rep. Mark Meadows, R-North Carolina, one of the founding members, freelanced in July when he tried to oust Boehner, but other members of the group chastised him for grabbing the spotlight without a plan to replace the Speaker ready to go.

More than two months later, Boehner suddenly resigned because he did not want to force his colleagues to cast a politically toxic vote to keep him in the leadership chair.

The group, which is secretive and does not publicize its membership, privately met for dinner Monday night at their favorite watering hole on Capitol Hill, Tortilla Coast. And they have yet to come to come to consensus on how to assert their power. The group's rules require that 80% of the members agree on their strategy before it becomes a formal caucus position.

As part of the list of its demands, Rep. Mick Mulvaney, the South Carolina Republican who founded the caucus, said that he wants the GOP leadership to help elevate far-right conservatives on prime committees.

"The population of the committees need to be reviewed," Mulvaney said.

But Mulvaney added that McCarthy "has that potential" to be different than Boehner. "It's up to him to make the case to how he would do that."

Searching for an alternative to McCarthy

On Tuesday, Salmon and other leading members of the Freedom Caucus turned to one of the most popular figures among the conservative base - South Carolina Republican Rep. Trey Gowdy to urge him to jump in and make a run for a leadership slot. But Gowdy told reporters late Tuesday he wouldn't run for leadership.

Pressed why conservatives don't field their own candidate against McCarthy, Rep. Walter Jones, R-North Carolina, another Freedom Caucus member, blamed the current political system for being stacked against outside insurgent candidates. "Everything up here is about money, everything up here is about calculating your next step," Jones said.

Boehner was his party's top fundraiser and pulled in record amounts that helped elect many of the members in 2010 and 2012 who then undermined him. Most of the members in the Freedom Caucus are from solidly red districts so they don't need to worry about raising money to keep their seats. Instead they are more likely concerned about a primary challenge from the right.

Many conservatives are complaining about elevating McCarthy, but they have yet to throw their support behind Rep. Daniel Webster of Florida, a conservative launching an uphill bid to oust McCarthy. Webster challenged Boehner for the top job at the start of the year as well.

"This is an opportunity for our conference to elect a fresh face," Indiana GOP Rep. Marlin Stutzman told CNN. "You know people are very frustrated with John Boehner across the country and I think for us to just go from one person to the next without a very difficult process would be unhealthy for us in the long run."

The first test for conservatives watching various candidates for leadership posts will come on Wednesday, when the House is expected to pass a so-called clean spending bill that funds federal agencies through December 11.

Salmon predicted only about 50 House GOP members would back it because it fails to defund Planned Parenthood and told reporters Tuesday it would be "suicide" for any House Republican competing for a leadership job to back the bill.

But McCarthy will face pressure from House moderates, too. Dent and other Republicans have also grown tired of the group's antics.

Dent wants new House GOP leaders to be prepared to stand with rank and file members on the tough votes - to deal with the debt ceiling this fall and to pass a broader budget deal. "We have a responsibility to govern around here," Dent told reporters. "The worst thing in the world is to be stuck in paralysis and those same leaders will be weakened."