President Donald Trump has unleashed a series of racist attacks in recent weeks, indicating that white grievance could be a key focus of his 2020 campaign strategy. But some Republicans worry that will cost him votes and take focus away from the economy and other issues he should be running on.

“Anything that detracts from a truly remarkable economic renaissance undermines his re-election,” says Republican pollster Frank Luntz. “This is a distraction.”

Over the weekend, Trump wrote a series of racist tweets about Rep. Elijah Cummings and his majority-black Baltimore district, calling it a “rat and rodent infested mess.” Just two weeks earlier, he had unleashed a series of racist criticisms against freshmen Representatives Ilhan Omar, Alexandria Ocasio-Cortez, Ayanna Pressley and Rashida Tlaib, all women of color, saying they should “go back” to where they came from, despite the fact that three of the congresswomen were born in the U.S. and all four are American citizens.

Trump denies that any of these comments were racially motivated. “I’m the least racist person there is anywhere in the world,” he told reporters on July 30.

Still, the rhetoric makes Republican donors cringe. “Trump had some great laps around the track last week: [former special counsel Robert] Mueller’s testimony, wall funding and the budget,” Dan Eberhart, a prominent GOP donor, told TIME in a text. “Rather than have those stories linger, he is effectively driving the car the opposite direction around the race track with his anti-Baltimore feud. As a Republican, all you can do is hope it doesn’t end in a wreck.”

Trump seems to be aiming to animate the white working-class voters in his base. In an interview with TIME in June, when asked whether he would try to reach out to swing voters, Trump said, “I think my base is so strong, I’m not sure that I have to do that.”

Read More: ‘My Whole Life Is a Bet.’ Inside President Trump’s Gamble on an Untested Re-Election Strategy

But polling shows that may not be true. A study funded by Cornell University’s Center for the Study of Inequality and published in the Washington Post found that “the backlash from the president’s racist rhetoric is likely to offset any electoral benefit from Trump’s base.” A recent USA TODAY/Ipsos Poll cited by the study found that a majority of Americans thought Trump’s tweets targeting the four freshmen congresswomen were “un-American”; three-fourths of women polled said the tweets were offensive, and independents, by more than a 2-1 margin, said the tweets were “un-American.”

“Racist rhetoric could backfire not only by motivating Trump’s opponents to turn out to vote, but also by turning away independents,” the Cornell study says.

In the 2018 midterms, Democrats were able to win the House of Representatives largely by focusing on health care and other economic policy issues, a blueprint that the 2020 Democratic presidential candidates are wrestling with whether to follow. The three states that gave Trump his narrow margin of victory in 2016— Michigan, Wisconsin and Pennsylvania—all elected Democratic candidates in the midterms. And there were some indications that white women, a key constituency for Trump, may be beginning to abandon him. About 53% of white women voters cast ballots for Trump in 2016, while 49% of those same voters cast ballots for Republican candidates in the midterms, according to CNN exit polls.

In a series of more than three dozen interviews, the Associated Press found that suburban women — a crucial voting bloc for the next election — often “expressed dismay — or worse — at Trump’s racially polarizing insults and what was often described as unpresidential treatment of people.”

Barbara Perry, director of presidential studies at the University of Virginia’s Miller Center, noted that in 2016 Trump successfully ignored the advice of the Republican National Committee’s 2012 autopsy report, which determined the party needed to reach out to more minorities to win elections in the future after Mitt Romney lost to President Barack Obama. Trump’s focus on racial issues isn’t new: he propounded the false conspiracy theory that Obama wasn’t a real U.S. citizen, and launched his presidential campaign in 2015 by calling some undocumented Mexican immigrants “rapists.”

But Perry says Trump’s dog whistles about race are becoming even more overt. “I’m going to say no, let’s just be racist… because 30%-35% of the American electorate is racist,” Perry says, describing Trump’s strategy. “The other people in the Republican Party and in the conservative electorate are not going to vote for Hillary Clinton, or in 2020, Kamala Harris or Elizabeth Warren. Maybe some would vote for Joe Biden, but they’re probably not going to vote for those two women or Pete Buttigieg, so then you think, they’re going to come on board with me anyway. So I might as well spend most of my time ginning up this 30-35% of my base.”

Trump campaign officials have said the economy would be the top issue for Trump’s campaign messaging. But in a June interview with TIME, Trump said he doesn’t think voters are giving him enough credit on the subject. “I think I’m getting credit,” he said, “but I’m not getting the full credit.”

Asked this week whether Trump’s feuds with the freshmen congresswomen and Cummings distract from the economy and other issues, Daniel Bucheli, deputy press secretary for the President’s re-election campaign, responded, “President Trump isn’t afraid to point out the real problems, and this is why he connects at a different level with all Americans. This is not campaign strategy, but an America First strategy to better all citizens lives, regardless of zip code—and has been since day one.”

-With additional reporting by Alana Abramson/Washington

Write to Tessa Berenson at tessa.berenson@time.com.