Congress isn’t the only branch of government that could go through a shake-up in 2018. For the second year in a row, rumors are circulating that Supreme Court Justice Anthony Kennedy, who is 81 years old and wrapping up his 30th year on the high court, is poised to retire when the term recesses in June.

Politically, the stakes couldn’t be higher. For several years, Kennedy has been one of the court’s most powerful justices thanks to his role as a pivotal swing vote, delivering victories to both the left and right in contentious cases. If Kennedy retires this year, President Trump and the Republican-controlled Senate will almost certainly replace him with a right-wing justice who is likely to secure a conservative majority on the court for years to come. But if Kennedy stays on the bench and Democrats assume control of the Senate in January, the promise of a solidly conservative successor for Kennedy seems more uncertain.

It’s challenging to predict when any justice will retire because they’re balancing a number of hard-to-measure factors, including the political environment, whether they feel they’ve built a legacy, and questions about their own health. Kennedy, though, is in a particularly difficult position because he holds an unusual amount of power on the current court, and he has good reason to be concerned that a Trump-appointed successor might undermine key parts of his judicial legacy.

Kennedy, a centrist conservative, was appointed by President Ronald Reagan. He’s the second-oldest justice on the bench after liberal Ruth Bader Ginsburg, so it seems safe to assume that he has at least contemplated the idea of retirement while the GOP controls the White House. But while several experts on judicial behavior believe that justices weigh the political climate when considering whether to step down, these considerations can be more complicated than they appear — especially for a justice like Kennedy who appears to be substantially more moderate than the current president.

Indeed, the political climate can’t be reduced to the political party of the sitting president: It includes the ideological makeup of the rest of the court, how liberal or conservative the president is relative to the retiring justice, and the composition of the Senate. Kennedy’s influence on the court, combined with his ideological distance from the current president and the Senate, help explain why even if Kennedy cares deeply about the political ramifications of his departure, he still might choose not to retire this year — and why it’s so difficult to predict when Supreme Court justices will leave.

Are there signs that Kennedy is planning to retire?

In 2014, Ginsburg said that liberals were “misguided” if they hoped she would retire soon so that then-President Obama could replace her with someone of a similar ideological bent. But Kennedy has not been so forthcoming, which means it’s especially difficult to deduce whether GOP politicians talking about his retirement are engaging in wishful thinking or whether they have some inside knowledge. (They anticipate that replacing Kennedy with a more conservative justice this summer would remind voters how valuable single-party control of the White House and Senate can be, which could help motivate the Republican base to turn out in the midterm elections.)

The most obvious sign that a justice is planning to retire is that they don’t hire a full slate of law clerks for the upcoming term — but if they do hire a full complement of clerks, that doesn’t necessarily signal anything about their intentions. Kennedy lined up his contingent of clerks late last year, but it’s common for other justices to take on clerks if the judge who hired them departs unexpectedly.

There’s also some reason to believe that Kennedy would want to refrain from retiring this year since justices typically avoid leaving during an election in an effort to prevent the politicization of the vacant Supreme Court seat. This convention may apply more to presidential election cycles than congressional midterms, though.

Others might watch this year’s decisions closely to see if Kennedy appears to be burnishing his legacy on crucial issues. But it’s a highly speculative endeavor, especially since Kennedy — as the court’s swing vote — has authored a number of high-profile rulings over the past few years and did not follow them up by retiring.

Richard Vining, a professor of political science at the University of Georgia, noted that Kennedy isn’t telegraphing any personal dissatisfaction with his position, which means that if he retires, the likely motivator would be his desire to depart under a Republican president and Senate. And here, past Supreme Court justices’ behavior (going back to the 1950s) can help shed some light on how this elite group has been influenced by the demands of politics.

When is it politically opportune to go?

There are several possible reasons why Supreme Court justices might assess the political climate when they’re deciding to retire. The first is simply that justices have partisan leanings like everyone else, and they would prefer to see their seat filled by someone of a similar ideological persuasion.

There is some evidence that justices are more likely to retire under presidents of similar partisan stripes, but that doesn’t necessarily mean retiring under a president of the same party as the one by whom they were appointed. For example, two justices saw an opportune political moment to retire when Democratic President Barack Obama took office: David Souter, who retired in 2009, and John Paul Stevens, who retired in 2010. Both were appointed by Republicans, but they shifted leftward during their years on the bench while the GOP shifted to the right, which meant they were solidly part of the court’s liberal wing by the time they departed under Obama and a Democrat-controlled Senate.

However, Kennedy may also care about something that is even more difficult to control: ensuring a like-minded successor. And this is an arena in which Supreme Court justices don’t have an especially good track record. To assess the ideological similarity between a departing justice and his or her replacement, we looked at Judicial Common Space scores, which measure judicial ideology relative to the full court, the president and Congress.

Developed by a group of political scientists including lead researcher Lee Epstein, at Washington University in St. Louis, and Chad Westerland, at the University of Arizona, the scores provide a snapshot of where each justice fell on the court’s ideological spectrum in his or her last term, and show how the incoming justice shook up the court’s ideological rankings. Some were replaced by relatively like-minded successors. But others were replaced by highly dissimilar justices — including some successors who even went on to undermine the departing justice’s legacy.

One challenge for Kennedy is that waiting for the perfect moment to depart can be dangerous. Past justices have made the wrong gamble. Thurgood Marshall, for example, was clear about his desire to retire while a liberal was president but, as his health failed, he was ultimately forced to give up his seat under George H.W. Bush. (He was then replaced by Clarence Thomas, who proved to be Marshall’s polar opposite.)

Ginsburg similarly refused to retire under Obama because she didn’t believe he’d be able to appoint a sufficiently liberal successor — and she is now grimly hanging on to her seat in her mid-80s to avoid departing under a Republican president she openly dislikes.

And even when a justice retires under seemingly ideal political circumstances, they still may not get an ideologically similar replacement. Sandra Day O’Connor — a Republican whose husband reportedly expressed horror at the prospect of an Al Gore presidency because she wanted to retire under a Republican — was ultimately able to step down under a fellow Republican, President George W. Bush. But she has subsequently criticized her replacement, Samuel Alito, for being too conservative.

“The justices are working with a lot of uncertainty here,” said Artemus Ward, a political science professor at Northern Illinois University and an expert on Supreme Court retirements. “They don’t know who will win the next election, they don’t know who the president will appoint to replace them, and they don’t know how long they’ll stay healthy. So they make the best decision they can using limited information, and sometimes they make mistakes.”

O’Connor’s regrets about her departure may be instructive for Kennedy, who replaced her as the court’s swing vote. Kennedy is currently the only true moderate on a court that has seen widening ideological polarization over the past few years.

Kennedy’s departure would likely result in an even more polarized court, and a Trump-appointed successor could vote to undermine some of his most influential rulings. “A generic Trump appointee will not share Kennedy’s views on gay marriage, for example, which is an issue he clearly wants to leave a legacy on,” Vining said. “And the longer he hangs on, the longer he gets to be the most powerful judge in America.”

Will it come down to party loyalty?

The question, then, may simply be whether Kennedy is enough of a party loyalist to give up his exceptionally influential role and allow his seat to be filled by a justice who will almost certainly be more conservative than him.

This is clearly the path that GOP politicians are hoping for. And he wouldn’t be the first justice to bow to the forces of partisan politics. Byron White was appointed by John F. Kennedy, a Democrat, and chose to retire under another Democrat, Bill Clinton, despite having drifted away from the party on key issues like abortion.

But at the time of his departure, White was not one of the court’s most decisive votes, so the fact that he was ultimately replaced by the even more liberal Ginsburg didn’t have a dramatic impact on the court’s ideological makeup.

Kennedy, on the other hand, has to grapple with the knowledge that his departure will almost certainly change the court for years or even decades to come. That means that he may be weighing the political climate differently than would justices who are more aligned with today’s Democratic or Republican party.

“It’s easier to think about leaving when you’re a left-leaning justice retiring under Obama,” said Christine Kexel Chabot, a scholar in residence at Loyola University-Chicago School of Law who has researched Supreme Court retirements. “But regardless of whether he retires under a Democrat or a Republican, Kennedy must understand that he’s really unlikely to get a like-minded replacement. So if he’s truly a partisan, he’ll retire now. But if he’s not, it’s hard to guess what would motivate him to leave.”