President Trump is moving quickly to replace Justice Anthony Kennedy, whose retirement announcement seems likely to create a solid conservative majority on the Supreme Court for the first time in decades. Although he was appointed by a Republican president, Kennedy earned a reputation as a moderate thanks to his willingness to join the court’s liberal wing in important rulings on gay rights, abortion, affirmative action and some criminal-justice issues. This track record led many to see him as a crucial bulwark against the court’s growing ideological polarization. In an open letter published in April that begged Kennedy to postpone his retirement, The New York Times characterized him as an “equal-opportunity disappointer” for both liberals and conservatives.

Except that’s not really the case. His sweeping rhetoric on gay rights, combined with a handful of key votes with the liberals in controversial cases, overshadows his track record of conservative rulings on a wide range of other questions.

An analysis of Kennedy’s voting record during his three decades on the court shows that he voted with the court’s right wing in the majority of cases — including controversial, closely decided cases — throughout his career. And although he was likelier to side with the liberals once he became the court’s swing vote, legal experts say that he occupied the ideological middle ground on relatively few issues. “When you have an odd number of justices, someone is always going to be in the middle, and Kennedy was certainly in the middle a lot,” said Steve Vladeck, a law professor at the University of Texas at Austin. But on many topics, Kennedy was as far to the right as his conservative colleagues. “The reality is that in a lot of cases, we may not see much of a difference between Kennedy and his successor.”

To evaluate Kennedy’s ideological leanings, we looked at data from Washington University School of Law’s Supreme Court Database, which identifies whether each justice’s vote in a particular case was liberal or conservative. Our analysis showed that Kennedy did move to the center on some issues over time, especially in close cases where only five justices were in the majority. But although he broke ranks with the court’s right wing more frequently than in the past, Kennedy still cast conservative votes in most cases, even after he became the swing justice.

Kennedy often voted conservative in close cases Percentage of votes coded as conservative in each justice’s Supreme Court career, through the 2016 term All cases Close cases justice Total Conservative rulings Percent conservative Total Conservative rulings Percent conservative Thomas 1,918 1,270 66.2 427 342 80.1% Scalia 2,461 1,566 63.6 583 472 81.0 Alito 768 485 63.2 182 153 84.1 O’Connor 2,484 1,483 59.7 567 419 73.9 Roberts 800 463 57.9 188 155 82.4 Kennedy 2,361 1,344 56.9 537 383 71.3 Breyer 1,624 675 41.6 379 81 21.4 Souter 1,497 594 39.7 337 60 17.8 Kagan 427 164 38.4 94 17 18.1 Stevens 3,515 1,335 38.0 798 178 22.3 Ginsburg 1,727 645 37.3 395 58 14.7 Sotomayor 523 193 36.9 109 11 10.1 “Close cases” are those decided by any five-justice majority, not exclusively 5-4 decisions. The 2016 term ended in September of 2017. Source: supreme court database

Since he joined the court in 1988, Kennedy voted in a conservative direction about 57 percent of the time — a record that’s nearly identical to that of Chief Justice John Roberts and only slightly less conservative than Justices Samuel Alito, Antonin Scalia and Clarence Thomas. And in close decisions, Kennedy sided with the conservatives 71 percent of the time. That’s not quite as consistently conservative as Roberts, Alito, Scalia and Thomas, all of whom cast conservative votes in at least 80 percent of cases with a five-justice majority, but it’s not exactly moderate either, since we might reasonably expect to see a moderate casting conservative votes closer to 50 percent of the time.

Looking at Kennedy’s career as a whole, though, masks a key shift that occurred when he became the court’s median justice. In his first two decades on the Supreme Court, Kennedy often fell to the right of Justice Sandra Day O’Connor. But when she retired in January 2006, he became the de facto tiebreaker on an increasingly conservative court, which may have changed his decision-making calculus. Between 1988 and the fall of 2006 (the end of O’Connor’s last term), Kennedy cast a conservative vote in 75 percent of closely divided cases. Between late 2006 and the end of September of 2017 — the latest Supreme Court term for which we have data — that number fell to 64 percent — and he was much more likely than in the past to vote in a liberal direction on cases involving criminal-justice issues, civil rights and due process.

Kennedy got more liberal on some issues over time Justice Kennedy’s Supreme Court votes in close cases, broken down by issue, through the 2016 term Pre-2006 2006 and later issue Total cases Conservative rulings Total cases Conservative rulings Change in conservative rulings Due process 22 77.3% 6 0.0% -77.3 Civil rights 65 87.7 29 55.2 -32.5 Criminal procedure 128 79.7 60 56.7 -23.0 Privacy 6 83.3 4 75.0 -8.3 Federalism 26 57.7 10 50.0 -7.7 Judicial power 25 68.0 21 71.4 +3.4 First Amendment 32 65.6 12 75.0 +9.4 Economic activity 36 69.4 25 92.0 +22.6 Total 361 75.1 176 63.6 -11.4 “Close cases” are those decided by any five-justice majority, not exclusively 5-4 decisions. Issues with very few cases are not shown, but reflected in the total numbers. The 2016 term ended in September of 2017. Source: supreme court database

This shift meant that although Kennedy remained quite conservative, his decisions became more difficult to predict and categorize. He was the fifth vote for the liberal bloc in cases that limited the use of the death penalty for minors and people with intellectual disabilities, while stopping short of indicating that he would vote to abolish capital punishment entirely. And he acted as a “tempering influence” in both directions on cases involving affirmative action and abortion, according to Yale Law School professor Robert Post. “These were issues where he saw both sides and voted on both sides,” Post said. For example, Kennedy wrote the majority opinion in a 2007 case that upheld a congressional ban on an abortion technique typically used after the 20th week of pregnancy, but voted with the liberals in a 2016 case striking down several state-level restrictions on abortion.

As dramatic and influential as these rulings could be, Kennedy’s moderate tendencies were limited. In his later years, Kennedy appeared to push the court to the left on a single, high-profile issue — gay rights — as he wrote a series of rulings that culminated with the legalization of gay marriage in 2015. “He was to the left of many of the justices on that issue,” Post said. “But that was an unusual circumstance for him.”

On other issues, Kennedy actually became more conservative as his career wore on. Between the term starting in 2006 and the one ending in 2017, he cast a conservative vote in 23 of the 25 close decisions on economic issues. During this time, he helped shape a broad view of freedom of speech that he used to justify unlimited corporate campaign spending and to strike down regulations on commercial speech. He also sided with the court’s conservative bloc in a wide range of cases that gave companies more power over workers, consumers and organized labor. It was Roberts — not Kennedy — who ultimately voted with the liberals to uphold the Affordable Care Act. And in his last term on the court, Kennedy didn’t side with the four liberals in any close decision. “With the exception of just a few issues, Kennedy voted with the conservative wing of the court in most highly ideological cases,” said Geoffrey Stone, a law professor at the University of Chicago. “I’d hardly call that moderate.”

Deciding when to retire from the Supreme Court is a tricky business, and Kennedy may certainly be disappointed with his successor’s views — and rulings — on gay rights. On most issues, though, a mainstream conservative successor will likely be doing more to reinforce Kennedy’s legacy than to tear it down. But that doesn’t mean conservatives won’t welcome the opportunity to replace him — or that the court won’t grow more conservative in his absence. “Kennedy was somewhat idiosyncratic and unpredictable,” Vladeck said. “So with a more reliably conservative replacement, it’s possible we’ll see more aggressive efforts by right-wing advocacy groups to upset precedents they were willing to live with under Kennedy, simply because they weren’t sure how he’d vote.”