Has Donald Trump fundamentally reshaped the Republican Party in his image? In fact, what evidence we have suggests that Republicans are only temporarily aligned with Trump on a range of issues, and the party will likely return to its earlier stances once his substantial media presence has faded.

My views differ somewhat from those of Julia Azari, who wrote that Trump’s “brand of politics has edged closer to defining what conservative Republicanism means today.” To be sure, there’s some evidence that conservatives, who were wary of Trump little more than a year ago, have become staunchly supportive of him and his message. Note the enthusiastic support for him at the Conservative Political Action Conference and their invitations to French fascist party members and other anti-immigrant speakers.

However, based on the evidence in my book, I believe that conservatism will lose its Trumpian twist once Trump is no longer a media presence. Realignments of party and ideology come from organized groups of activists rather than media-centered candidacies and presidencies. Using fame and social media, Trump managed to bypass many of the usual influential Republican party actors — neoconservatives, free marketers, and culture warriors. Normally, candidates depend on highly organized political networks to help them win elections and carry their message in the future.

Trump, by contrast, won by reaching out to unorganized voters through traditional and social media. This is bound to be short-lived because party elites tend to wrest control of a wayward party’s direction. Even now, George Edwards and Matt Glassman argue that Trump cannot influence fellow partisans on his truly heterodox positions. Republican rank and file might not have changed much since 2012, as Larry Bartels argues, but unorganized voters do not control a party’s direction.

A Republican Party out of power would likely find its way back into 2012 conservatism just as 1920 Republicans found their way back to 1908 conservatism.

We may see a test to these claims in the form of a primary challenge to Trump in 2020; there are at least hints of such a challenge emerging from a fellow Republican. If such a threat actually derailed Trump’s reelection bid, what might be the long-term effect on the party?

To answer this question, Azari’s piece looked at earlier challenges to Republican incumbents, such as Teddy Roosevelt’s 1912 third-party campaign against his former ally, incumbent President William Howard Taft. This challenge, and the Republicans’ loss to Democrat Woodrow Wilson that year, angered more conservative Taft-aligned Republicans, who became more dominant in the party in the following years.

However, Roosevelt’s challenge was not necessarily the reason that his progressive Republicans fell into decline. A number of historical factors peculiar to that era — especially including World War I, the federal government’s reaction to it, and public fatigue from it — led to conservative dominance of the GOP in subsequent years.

More importantly, Trump lacks the organized following of Taft, while today’s more conventional Republicans are considerably more organized than Roosevelt’s followers were. A Republican Party out of power would likely find its way back into 2012 conservatism just as 1920 Republicans found their way back to 1908 conservatism. In both cases, parties gravitate toward their organized activists.

A challenge to Trump, in the form of either a primary or a third-party challenge, is unlikely to succeed but not necessarily futile. Neither H. Ross Perot nor fellow Reform Party members fared well at winning elections, but both Bill Clinton and Newt Gingrich incorporated some of their positions. If Trump scandals reach the benchmark of Watergate in the eyes of the public, a Republican who dissociates from Trump will be well positioned to articulate an alternate vision.

If the vision is conservatism without Trump, the party bulwarks are there to support it. In the long term, any moderate challenge would also need a much more robust network of organized activists to compete with conservatives for the direction of their party.

Christopher Baylor is the author of First to the Party: the Group Origins of Political Transformation and an American Political Science Association congressional fellow.