Washington (CNN) Nancy Pelosi is sending a defiant message to House Democrats who are grumbling about her leadership: Come at me.

But while Pelosi's critics are increasingly vocal, they have not yet answered critical questions about who would fill her leadership and fundraising roles if they were to mount a serious challenge for the perch atop the party Pelosi has held since 2002. Nor have they identified a challenger.

At least, not yet.

A dozen House Democrats huddled in the office of New York Rep. Kathleen Rice -- an outspoken Pelosi critic -- on Thursday afternoon, among them Massachusetts Rep. Seth Moulton and Ohio Rep. Tim Ryan. Most notable was the appearance of Louisiana Rep. Cedric Richmond, the chairman of the Congressional Black Caucus.

Those involved in the discussions described them as preliminary among like-minded lawmakers.

"I think there was consensus within the room that there are other members within the caucus who feel just like we do," Texas Rep. Filemon Vela told CNN. He emphasized that the group "was a diverse group from an ideological, geographic and ethnic standpoint."

Increasingly, Democrats concerned about Pelosi's status as a bogeyman on the right -- which Republicans have effectively used to motivate their base voters -- are warning that unless Pelosi is ousted, she could cost the party its shot at winning control of the House in the 2018 midterm elections.

"As long as Leader Pelosi is perceived as the leader of the House Democratic Caucus," Vela said, "Republicans are going to continue to spend millions and millions of dollars in those swing districts to convince those swing voters, those independent voters -- those Republican voters who might go our way -- not to vote for our Democratic candidate because of Leader Pelosi."

Fueling their fears is Democrat Jon Ossoff's loss in a hotly contested House race in the northern Atlanta suburbs.

Ossoff ran on a decidedly moderate platform with millions of dollars to back him up. But his Republican opponent, Karen Handel, as well as a GOP-aligned super PAC and the House Republicans' campaign arm, the National Republican Congressional Committee, spent millions on television ads and mailers latching Ossoff to Pelosi.

JUST WATCHED Dem congresswoman: Time for Pelosi to go Replay More Videos ... MUST WATCH Dem congresswoman: Time for Pelosi to go 01:21

Fundraising and leadership successes

Still, Democrats would have massive gaps to fill if they were to oust Pelosi.

Chief among those is fundraising. Since 2002, Pelosi has hauled in $568 million for House Democratic campaigns -- including $141 million during the 2016 campaign cycle.

Pelosi also has a long history of holding her party together through difficult votes -- enabling former President Barack Obama to shepherd into law a series of Democratic-backed measures in 2009 and 2010 and later forcing Republicans to grapple with the politically damaging divisions within their own ranks.

No other Democrat possesses the stature to match those accomplishments.

That reality was the subtext for Pelosi's taunting remarks Thursday in which she cast her Democratic critics as hungry for attention rather than serious about ousting her.

"When it comes to personal ambition and having fun on TV, have your fun," Pelosi said. "I love the arena. I thrive on competition."

Of the political impact of the Republican attacks: "I think I'm worth the trouble," she said.

But Pelosi's camp signaled she is taking the grumbling seriously. Allies and grassroots supporters were sent suggested tweets Thursday that urged Democrats not to ditch Pelosi in the middle of a high-stakes legislative battle over the future of health care reform.

Democrats supportive of Pelosi have long argued that even if she stepped aside, Republicans would demonize any new leader the party elevated.

There's also an even more basic reality: Showing that the party will dump its leader if Republicans are persistently negative enough about that leader only gives the GOP more incentive to use aggressive tactics -- and puts the next Democratic leader on tepid ground from the outset.

"They always want to choose our leaders," Pelosi said Thursday. "And usually they go after the most effective leaders."

There are, though, reasons to believe Pelosi poses a unique political problem for Democrats.

Her approval ratings are low, and while House Speaker Paul Ryan's are similarly weak, the idea of ousting Ryan doesn't animate Democratic voters the way defeating Pelosi proved to motivate conservatives in the 2010 and 2014 midterm elections, as well as the special election in Georgia.

President Donald Trump earlier Thursday even tweeted he'd like her to stick around.

I certainly hope the Democrats do not force Nancy P out. That would be very bad for the Republican Party - and please let Cryin' Chuck stay! — Donald J. Trump (@realDonaldTrump) June 22, 2017

And Pelosi's fundraising prowess? It comes at a cost: Pelosi is Republicans' biggest fundraising star, too.

"It's fair to say almost every cent has been raised through the prism of, 'we need to hold the Republican majority to prevent Nancy Pelosi from passing her far-left agenda,'" said Jesse Hunt, a spokesman for the National Republican Congressional Committee -- the House GOP's campaign arm.

JUST WATCHED Dem lawmaker: Pelosi is more toxic than Trump Replay More Videos ... MUST WATCH Dem lawmaker: Pelosi is more toxic than Trump 01:31

Thin bench for House Democrats

Pelosi's potential successors have included former Rep. Rahm Emanuel -- but he left the House to become Obama's chief of staff and later to run for mayor of Chicago. Another rising House Democrat, former California Rep. Xavier Becerra, departed the House in late 2016 to accept an appointment as California's attorney general.

The top three House Democrats are now Pelosi, who is 77; Maryland Rep. Steny Hoyer, who is 78; and South Carolina Rep. Jim Clyburn, who is 76.

Those leaders have been entrenched in their current positions for years, limiting younger members' ability to rise to the top of the House Democratic ranks -- though Pelosi has created new junior leadership roles for younger members. Among those: Adding No. 2 roles to be filled by newer members on every House standing committee and adding six regional vice chairs to the Democratic Congressional Campaign Committee.

Pelosi said Thursday that she has "always featured the young 30-somethings."

"So," she said, "we are paving a way for a new generation of leadership."

When the top post among House Democrats might open up, though, Pelosi hasn't said.