Hours after the Massachusetts senator Elizabeth Warren ended her presidential campaign and all but assured the next American president would be a white man in his 70s, a group of influential Democratic women gathered for dinner at a Spanish restaurant in downtown Washington.

The timing was coincidental, but it underscored the urgency of their meeting.

More than a hundred years after American women earned the right to vote; four years after the first female presidential nominee of a major party won the popular vote by 3 million votes and two years after the election of the most female congressional class in history, six diverse women had entered the 2020 Democratic race and didn’t win a single nominating contest.

“Without a doubt, the secret to Democratic victories in the past three years has been women’s leadership. The idea that that wouldn’t translate into the first woman president was heartbreaking for many of us,” said Heather McGhee, a distinguished senior fellow at the thinktank Demos, who supported Warren and helped organize the meeting.

Emotions still raw, the group of women decided to do something about it.

Over tapas, they drew up a list of demands for the party’s next nominee: he would choose a female running mate; women would represent the “governing majority” of a Democratic administration; appointments would reflect “the diversity of our communities” and economic issues focused on issues such as paid family leave would be a priority in the first 100 days.

They compiled the principles into a letter that was signed by dozens of prominent party activists and leaders and sent to the Democratic National Committee.

Days later, at the 11th Democratic presidential debate, the first without a female candidate on stage, Joe Biden explicitly committed to picking a woman as vice-president.

“There are a number of women who are qualified to be president tomorrow,” said Biden, who holds a nearly unassailable lead over his rival, the Vermont socialist senator Bernie Sanders. He also committed to nominating the first African American woman to the supreme court, a gesture that reflects his reliance on the support of black women, a crucial Democratic constituency.

Electing a woman as vice-president would be historic. Only two women have ever been chosen as running mates: the Alaska governor Sarah Palin in 2008 and the New York congresswoman Geraldine Ferraro in 1984. Neither served in the White House.

Sarah Palin during the vice-presidential debate with Joe Biden in St Louis, Missouri, on 2 October 2008. Photograph: Carlos Barría/REUTERS

Far from being a “consolation prize”, McGhee hopes that putting a woman on the ticket will move the country closer to electing the first female president. She envisions an empowered lieutenant in the “Dick Cheney mold – but with very different values”.

“This isn’t just ticking a box,” she said. “This is opening a door that has been closed for too long.”

•••

Biden’s pledge was widely embraced by Democrats eager to elevate a qualified woman to the second-highest office in the land.

But it was also a stark acknowledgment that a party increasingly led by women rejected a historic field of female leaders.

Biden and Sanders, the last men standing, “don’t look the way the party looks”, said Jennifer Lawless, a professor at the University of Virginia and an expert on women in politics. “They don’t even look the way the primary field looked.”

Committing to putting a woman on the ticket directly addresses the “concern among some women that this party continues to marginalize women at the highest levels of leadership”, said Kelly Dittmar, a scholar at Rutgers University’s Center for American Women and Politics. “This is among the least they could do to demonstrate their commitment to gender equality.”

But Biden’s commitment to limiting the prospective pool of candidates by half did not draw universal praise. In some conservative and progressive corners, he was accused by women of “tokenism” and prioritizing inclusion over achievement.

But researchers say reflecting diversity does not sacrifice quality. Studies have found that women often have more relevant experience than their male counterparts when they run for office.

We have more women who are qualified to be vice-president now than we ever have before Jennifer Lawless

“We have more women who are qualified to be vice-president now than we ever have before,” Lawless said.

There are strong political arguments for choosing a female running mate. The 2016 presidential election brought the widest gender gap in a half-century of polling and since then, women’s rage has fueled Democratic victories in every corner of the country. They have led protests, launched political organizations, run for office and voted for Democrats in record numbers. And it is women, far more than men, who most viscerally disapprove of Trump and his policies.

While research suggests running mates do little to persuade undecided voters, putting a woman on the ticket could help generate enthusiasm among Democrats . It could also help draw a contrast with Trump, who has been accused of sexual misconduct by more than a dozen women.

Jennifer Epps-Addison, the network president of the Center for Popular Democracy, which endorsed Sanders and signed the letter, said Biden should consider his choice of running mate as an olive branch to progressives.

“If the next nominee wants our time and our energy, they have to be willing to negotiate the platform, the policies, the transition and what the future of the Democratic party looks like,” she said.

For her part, Clinton said recently she would “love” to have a woman on the ticket, but said the nominee must consider the electoral implications: “What will help him win the electoral college?”

Biden’s announcement has intensified speculation around who he might choose. On Tuesday, the former vice-president told the hosts of The View that he would start vetting a shortlist of candidates “soon” and that it was important his running mate is “simpatico with where I want to take the nation”.

His former rivals, particularly Senators Kamala Harris, Warren and Amy Klobuchar, are seen as top contenders. As far back as last November, Biden floated four possible running mates and all were women, including Stacey Abrams, who ran for governor of Georgia in 2018 and Sally Yates, the former acting attorney general fired by Trump after refusing to defend his administration’s travel ban.

Elizabeth Warren in Manchester, New Hampshire, on 8 February. Photograph: Matt Rourke/AP

While the vice-presidency is a priority, it’s not the only one of many ways the next nominee must demonstrate his commitment to women’s economic and political empowerment, said Sara Nelson, president of the Association of Flight Attendants and another organizer of the meeting.

“It’s about being committed to hearing women, to actively listening to women, to lifting up the voices of women and recognizing that women are great leaders,” she said. “We’re making those demands but we’re also not sitting back and waiting for those demands to be met.”

•••

In many ways, Biden’s pledge reflects the dueling impulses of a Democratic party that prides itself on diversity and inclusion but remains deeply scarred by its loss to Donald Trump in 2016.

Early enthusiasm over the record number of barrier-breaking women and candidates of color in the race did not translate into support as the conversation gave way to concerns about electability.

On the campaign trail, voters who believed sexism played a role in Hillary Clinton’s defeat feared that nominating another woman would deliver the same result. Many Democrats urged the male frontrunners to pick a female running mate, sometimes suggesting candidates like Klobuchar and Harris before they had exited the race.

Kamala Harris and Joe Biden at a campaign rally in Detroit, Michigan, on 9 March. Photograph: Jeff Kowalsky/AFP via Getty Images

“Voters were so afraid to recreate anything that looked or felt like 2016 and that created a unique disadvantage for female candidates who wanted to take on Trump,” Meredith Kelly, who was a top aide to the New York senator Kirsten Gillibrand’s presidential campaign.

During a debate, Klobuchar said it was obvious that the women on stage were “held to a higher standard” than their male counterparts. Their mistakes, it seemed, were punished more harshly by a party that demonstrated a willingness to rally behind men whose success defied political odds.

Biden, who is running on his decades of experience, was first elected to the Senate in 1972, at 29. He has previously lost two presidential elections, and placed fourth and fifth in the first two nominating contests. Sanders fought his way back into contention after a heart attack. Pete Buttigieg, the 38-year-old former mayor of South Bend, Indiana, parlayed national media attention into a first-place finish in the Iowa caucuses.

“Women have to bring the receipts while men get judged by their potential for greatness,” Kelly said. “That was extremely apparent in this presidential race.”

There is no evidence to suggest that a woman would lose to Trump. After all, more Americans voted for Clinton than Trump. The women running attempted to address the issue of electability head-on, but the doubts persisted until last week, when the path for nominating a woman officially closed with the departure of the long-shot Hawaii congresswoman Tulsi Gabbard.

“These were pretty remarkable women who were accomplished, who could raise money, who had national profiles and who were leaders in their own right,” said Randi Weingarten, president of the American Federation of Teachers, another organizer of the dinner. “So we have to ask: is there some misogyny, is there some sexism that serves as a barrier to women running for president?”

The degree to which gender played a role in 2020 will be debated for a long time. Warren herself hadn’t fully grappled with the question when she was asked about it moments after announcing she was ending her campaign.

“Gender in this race?” Warren said, addressing reporters outside her home in Cambridge, Massachusetts. It was a “trap question” that she said forces female candidates to either acknowledge sexism and sound like a “whiner” or dismiss it and deny the experiences of “about a bazillion women” who would ask: “What planet do you live on?”