A cadre of progressive groups and activists that has elevated leftist candidates and helped shift Democratic policy further left in primary elections across the country this year is now questioning the politics of a potential presidential candidate — Joe Biden.

The activists, some frustrated with how the former vice president has weighed in on several primaries this year and others concerned about some of his past policy positions, are in some cases already speaking publicly to warn Biden and other moderate Democrats mulling presidential bids that they’re going to make a potential centrist’s path to the presidency uncomfortable.

Biden has been explicitly mulling a presidential bid since leaving the White House, and he’s patched together the framework for a potential campaign: forming a think tank to tackle domestic policy, going on an extensive cross-country book tour, and stepping in to endorse some establishment candidates over progressive primary challengers in races across the country. The moves have caught the attention of some in progressive circles, who say that Biden’s centrist political outlook is incongruent with the current political moment.

“Establishment Democrats like Joe Biden could be throwing their muscle behind some of these policies, and maybe without the sort of risk they thought they were under before,” Nasim Thompson, the communications director of Justice Democrats explained about Biden endorsing against candidates who support policies like Medicare for All and getting corporate money out of politics. “If this is the path he’s trying to pave and define the field for his own run, then he’s definitely, with his choices, aligning himself with the sorts of candidates that will leave marks on his record and alienate him from the progressive grassroots.”

Justice Democrats, which has helped boost progressive congressional candidates this year, has taken several shots at Biden on social media, labeling him as part of an “establishment” whose work in the midterms is emblematic of what’s “wrong” with the Democratic Party.

Alexandra Rojas, the codirector of Justice Democrats, said that while the group is primarily focused on electing progressives in the midterm elections and pushing the party further left ahead of 2020, they’ll continue calling out Biden and centrist-leaning Democrats to pressure whoever the eventual nominee might be. “The base right now wants really refreshing candidates, and if we continue to support these people it sets the stage for us to elect a candidate as far left as possible in the primary.”

While some progressive activists have complained about Biden privately, many are hesitant to do so publicly, or they preface their complaints with an appreciation of the Obama administration’s work. The former vice president remains relatively popular in national polling and is a key figure in the Obama administration’s legacy.

“Biden occupies a particularly nuanced position among the grassroots. People sort of see him as this avuncular guy that drives the Trans-Am,” Neil Sroka, the spokesperson for Democracy for America, told BuzzFeed News, pointing out that Biden had also endorsed progressive Maryland gubernatorial candidate Ben Jealous after a contentious primary. He added that while a narrow set of progressive organizers is already paying attention to where Biden is heading politically, progressives more broadly will start paying a more drastic amount of attention to his record should he decide to run.

Biden dipped into a slate of late summer primaries that saw progressive challengers front campaigns against establishment incumbents. In New York, he endorsed Gov. Andrew Cuomo in a speech at the state party’s convention, and he cut an ad for the governor just weeks before his primary against challenger Cynthia Nixon.