In private, according to reports, most Republican lawmakers agree that Donald Trump’s press conference with Russian President Vladimir Putin was a horrifying display of contempt for American institutions. But they won’t speak out. “Most Republican members are willing to admit POTUS doesn’t operate in reality, but know they’re doomed in their next primary if they say so publicly,” says Dave Wasserman of Cook Political Report. “[Republicans] see no upside in speaking out—and fear political suicide if they do,” note Mike Allen and Jonathan Swan for Axios.

Retiring lawmakers like Jeff Flake of Arizona can make impassioned cries against President Trump—“We have indulged myths and fabrications, pretended it wasn’t so bad, and our indulgence got us the capitulation in Helsinki,” he said last Thursday—but few Republicans will back them or take any action to hold the president accountable. (Flake, too, has often failed to offer more than tough talk.) After that speech, for example, Senate Republicans blocked a measure to affirm and support the nation’s intelligence agencies. Fear of the base is just that strong.

What if that fear is misplaced? Yes, polls show high Republican support for President Trump, but those polls don’t measure change in party identification. Most Republicans back Trump, but there might be fewer Republicans. If so, the dreaded GOP base might be less fearsome than it appears.

Most coverage of Republican voters paints them as a unified, unmovable bloc in support of the president. “Huge GOP majority backs Trump’s Putin performance,” reports Axios, summarizing results from a new poll. In it, 79 percent of Republicans endorse the president’s handling of the Helsinki press conference, similar to the 68 percent who supported Trump in a CBS News survey and the 66 percent who backed him in a poll from ABC News and the Washington Post. General approval polls—which give Trump upward of 90 percent support from Republicans—reinforce the perception that, among GOP voters, the president is untouchable.

His scandals and outrages—controversies and improprieties—have had an effect.

But that perception misses important context. Presidents always have partisans, and it’s rare that they break ranks. On the eve of his resignation in 1974, half of Republicans still supported Richard Nixon, and 59 percent said he shouldn’t be forced from office. Likewise, around 80 percent of Republicans backed Ronald Reagan at the height of the Iran-Contra scandal. For Trump, the key question is less “how many Republicans still support his administration?” and more “how many voters are still Republicans?”

Voters have to identify themselves with a political party, and that identification isn’t stable; it ebbs and flows with events and circumstances. Trump might win high marks from most Republicans, but the pool of Republican voters might be smaller than in the past. Far from standing tall over the entire GOP, Trump’s base may have eroded significantly from where it was at the beginning of his administration.

According to the Pew Research Center, Republican Party identification fell 3 points, to 26 percent, from 2016 to the end of 2017. The number of self-identified independents increased at the same time, from 34 percent to 37 percent, while the number of Democrats remained steady. Gallup shows a similar change: From November 2016 to November 2017, there was a 5-point drop in the number of people who called themselves Republicans, from 42 percent to 37 percent. Democratic self-identification remained unchanged at 44 percent.

The sheer size of the United States makes it easy to find vocal support for anyone and anything, and Donald Trump has his vocal supporters. But their staunch commitment overshadows the reality: a shrinking base for a president who won by the skin of his teeth, reliant on a small group of voters in just a handful of states. His scandals and outrages—controversies and improprieties—have had an effect. Even rank-and-file GOP reactions to Helsinki are revealing; according to CBS, 21 percent of Republican voters disapproved of the president, a striking number given typical partisan loyalty.

A smaller base is still a base, and depending on their locations and constituencies, Trump-skeptic Republican lawmakers may see a backlash if they challenge the president. John McCain’s anti-Trump defiance has earned him real anger from Arizona voters, even as he prepares to leave office.

But not all Republicans share the same vulnerability. Recently elected (or re-elected) lawmakers have room to not only speak against the president, but to actually maneuver against him. We saw this with Sens. Tim Scott of South Carolina and Marco Rubio of Florida, who blocked a circuit court nominee on account of his racist writings. They took advantage of the narrow Republican majority to hold the White House to account, joining with Democrats to keep Ryan Bounds off the bench. Critically, there hasn’t been blowback, and if it comes, both have four years to prepare for re-election.

Republican Trump critics in Congress insist there’s nothing they can do, lest they enrage the base and jeopardize their seats. But the steady erosion of Republican self-identification throws this into question. Perhaps there’s more space for resistance and critique than appears at first glance, and perhaps GOP lawmakers are playing themselves by hesitating—essentially allowing Trump to drag them down alongside him with the public at large.

Of course, there’s one other possibility: that these “critics” aren’t as serious as they appear. They might disagree with the president—even think he’s dangerous—but not so much that they would jeopardize tax cuts or additional conservative judges. If that’s the case, then the dreaded “base” may just be an excuse to avoid hard questions about inaction and to blame their complicity on something beyond their control.