Justice Anthony Kennedy is retiring, which means the makeup of the Supreme Court is about to shift dramatically. For the past 12 years, since Justice Sandra Day O’Connor retired from the bench, Kennedy has been the court’s median justice, with four justices to the left of him and four to the right. During that time, he has vexed both liberals and conservatives with sweeping, high-profile rulings on issues such as gay rights and campaign finance. And now President Trump will replace him with a judge who will almost certainly be more right-wing than Kennedy, decisively tipping the balance toward the court’s conservative faction. This will likely bring an end to the delicate equilibrium we’ve become accustomed to over the past decade.

Trump has announced that he will choose Kennedy’s successor from the shortlist of 25 contenders that he made public during the campaign and updated last year. Many of the people on this list are federal appeals court judges, so we have some sense of how they might vote if they made it onto the Supreme Court. If almost any of them are appointed, Chief Justice John Roberts will be the court’s new median justice. When the court hears a case that touches on the most contentious issues of American life, a conservative majority will be all but assured.

It’s easy to see why Kennedy’s departure could lead to such a seismic shift for the court. Although Kennedy is certainly not a moderate on many issues, his willingness to vote with both wings of the court made him the court’s median justice during every term since 2005. In that time, his score drifted toward the liberal bloc — think of his tiebreaking votes on important issues such as abortion and gay rights — while the other justices, with the notable exception of Roberts, remained more entrenched in their separate positions. Since Kennedy became the reliably median justice, the ideological range between the middle five justices has been wider than it was at any other time in modern Supreme Court history.

A Supreme Court with someone like Roberts in the ideological middle would not be unprecedented. O’Connor, for example, was in the middle before Kennedy, and she often scored as more conservative than Roberts does now. The same goes for Lewis Powell, a Nixon appointee, and Byron White, a Kennedy appointee, both of whom were the median justice of their day and clocked scores to the right of Roberts’ score today. And back in the late 1940s and early ’50s, many members of the court sat tightly bunched around an ideology score that’s slightly more conservative than that of Roberts in recent years.

But some of this empirical analysis may mask the true conservative nature of the future court. Looking at the raw numbers, Roberts’s ideology has been steadily drifting left since he joined the court in 2005. But this could be due in part to the fact that the court has in recent years taken up more of the sharp questions that conservatives want it to take up. Voting with the court’s liberal bloc on a right-wing issue would shift a justice’s score toward the more liberal side, even if the position they took would still generally be considered conservative. So perhaps the very benchmark the data revolves around is shifting.

It’s also possible that Roberts appears more liberal because, as the chief justice, he has an incentive to join the majority in cases where his vote alone cannot change the decision; this tactic allows him to influence decisions he doesn’t fully agree with, rather than simply dissenting. (The chief justice assigns which justice will write the opinion for whichever group he votes with, so he can indirectly shape a ruling by assigning the opinion to himself or a justice sympathetic to his views.)

There is one would-be justice among the federal appeals judges on Trump’s shortlist who would scoop the median-voter title from Roberts, at least as measured by Judicial Common Space scores — and that’s Thomas Hardiman, a judge who serves on the 3rd U.S. Circuit Court of Appeals.

To figure out how Trump’s nominee might transform the court’s ideological makeup, we looked at potential candidates’ Judicial Common Space scores, which measure judicial ideology relative to the full court, the president and Congress. The scores range from -1 (very liberal) to +1 (very conservative). Because pinning down the ideology of a lower court judge can be tricky, their scores are based on the political positions of the people who control their nominations to the court: the president and their home-state senators. An appointee’s senators are included because of a custom called “senatorial courtesy,” which is an implicit agreement that senators will not support nominees who are opposed by the senators from the nominee’s own state.

There are reasons to wonder if Hardiman is truly more moderate than Roberts: Hardiman earned his score thanks in part to the ideology of one of the senators representing his home state at the time he was nominated, the centrist Arlen Specter. SCOTUSblog labeled Hardiman “a solid, although hardly knee-jerk, conservative.”

Things will look more conservative still if the spot goes to a judge like Brett Kavanaugh, a former Kennedy clerk who now serves on the U.S. Court of Appeals in Washington, D.C., and is emerging as an early frontrunner. Kavanaugh is among the most conservative of the judges for whom we have ideology data, with a score that puts him to the left only of the archconservative Clarence Thomas.

One thing is certain: With a second Supreme Court vacancy in hand and two liberal justices in or approaching their 80s still on the bench, Trump appears poised to exert as much influence on the court as Richard Nixon, who appointed four justices in the space of three years — justices who transformed the court from the liberal, civil-rights-minded body of the 1960s to the conservative, business-friendly court we see today. So if Roberts becomes the median justice next term and Trump gets another chance to expand the court’s right wing, Roberts might not stay in the middle for long.