Four months ago, the woman in charge of recruiting Republicans to run for Congress in the 2020 elections lectured her party about diversity.

“It’s important that we, as a conference, do a better job of looking like America, and better representing the very diverse country that we have,” said Susan Brooks, a four-term representative from Indiana.

At the time, Brooks was one of only 13 women out of 197 Republicans serving in the House of Representatives. But that paltry number could shrink further when the next Congress is sworn in in 2021.

Last week, a sitting Republican congresswoman from Alabama announced her departure. And last month, Brooks undermined her own call for diversity by announcing that she, too, had decided to quit Congress.

“While it may not be time for the party, it’s time for me personally,” Brooks said.

As a party led almost exclusively by white men, represented in Congress overwhelmingly by white men and increasingly dependent on the votes of white men, Republicans have for years paid lip service to the need to “better reflect the makeup of the United States”.

But meanwhile, things keep slipping. On Thursday, the sole African American Republican in the House, Will Hurd of Texas, announced that he would not seek reelection in 2020. Hurd, a former CIA officer once hailed as “the future of the GOP”, tweeted that he was off to “solve problems at the nexus between technology and national security”.

“Hurd leaving is basically an admission that he doesn’t think the GOP is going to change,” tweeted the New York Times columnist Jamelle Bouie. “And as non-whites leave the party, what’s left of the guardrails keeping the party from explicit full white nationalist erode even further.”

The first ballot box test for Republican diversity in the Trump era told a stark story – and a discouraging one

Republican operatives say the party is succeeding in recruiting a diverse class of 2020 candidates. But as things stand, Hurd’s departure would leave only one Republican African American in all of Congress, Senator Tim Scott of South Carolina – out of 250 Republican members. Forty-one out of 53 sitting Republican senators are white men.

Donald Trump, who in his election victory picked up only 6% of the African American vote and 28% of the Hispanic vote, showed that as of 33 months ago, Republicans could win nationally by winning the white vote, which Trump won 54-39, according to a Pew Research study of validated voters.

But Republicans working on down-ballot 2020 races echo Brooks’s view that the GOP has a big, white, male problem on its hands.

“The GOP understands that it’s hit a low point as far as its ability to represent the American public,” Albert Eisenberg, a spokesman for Catalyst Pac, a pro-diversity Republican group, told the Capitol Hill publication Roll Call in May. “Our brand is really toxic.”

Bob Salera, the deputy communications director for the National Republican Congressional Committee, told the Guardian that the group has so far met a diverse batch of potential 2020 recruits, including 196 women and 92 potential candidates who are members of racial or ethnic minority groups.

Will Hurd has said he will not seek re-election. Photograph: Eric Gay/AP

“We’ve been successful in getting these kind of people in races,” Salera said, “and we look forward to having a slate of Republican candidates who reflect the diversity of our country.”

Party operatives have touted 2020 candidates including Wesley Hunt, an African American army veteran running in Texas; Mike Garcia, a former navy fighter pilot running in California; Young Kim, the first Korean American Republican woman to become a state legislator in California, now running for Congress; and Evelyn Sanguinetti, a former lieutenant governor of Illinois of Cuban-Ecuadorian descent now running for Congress.

But previous attempts by Republicans to cultivate diversity in the ranks have failed to bear fruit, and Trump’s racist attacks and broad disfavor among voters of color could damage the party further among non-white voters in 2020, just as Trump has alienated former rising stars in the party who are not white men.

The first ballot box test for Republican diversity in the Trump era told a stark story – and a discouraging one, for party leaders hoping to diversify the brand. Two pictures from new member orientation for the incoming members in 2018 show dozens of white men on the Republican side and a diverse group on the Democratic side.

Republicans elected only one new woman to Congress in 2018, versus 22 for the Democratic caucus. The proportion of white men within the Democratic caucus subsequently dropped from 41% to 38%, while the same percentage rose among Republicans from 86% to 90%.

African American voters remain overwhelmingly Democratic, according to Pew Research, with 84% identifying with or leaning toward the party and just 8% identifying in some way with the Republican party.

Meanwhile, the share of women who identified with or leaned toward the Democratic Party rose from 54% in 2016 and 56% in 2017, while the partisan breakdown of men was relatively unchanged with a 51-47 Republican tilt.