With the death of Antonin Scalia on Saturday, the Supreme Court lost not only its longest-serving justice but also the most outspoken member of its conservative wing. A careful analysis of his tenure, however, shows that Scalia was almost never the most conservative justice on the court, and in fact moved leftward during the second half of his 30 years on the bench. With the exception of the 2014 term, he was usually in the majority in the court’s rulings, and the future direction of the court pivots on his replacement.

While a discussion of Justice Scalia’s most notable opinions is of substantial interest, we choose a different approach here. Instead of focusing on particular cases, we cast our attention to some of the general patterns that appeared in his voting behavior on the court over nearly three decades.

A first question is just how conservative Scalia was. To answer it, we turn to the ideological scores we developed (hence the name Martin-Quinn scores) to place the justices on an ideological continuum based on their votes in the cases in each term. These scores range from negative values (liberal) to positive values (conservative). The chart below plots the Martin-Quinn scores for the 1986 term (Scalia’s first) through the 2014 term (the data for the 2015 term won’t be available until the end of the term this June).

At the time of his appointment, Scalia was on the right; indeed, only Justice William Rehnquist was more conservative. Our measure indicates that from 1986 to 2000, Scalia drifted more to the right. During the same period, Rehnquist became more moderate, perhaps because of his role as chief justice. And, in 1991, Clarence Thomas joined the court as the most conservative justice, a position he has held for his entire career. After 2000, Scalia tracked slightly to the left, yet he remained the second-most conservative justice until the 2012 term, when Samuel Alito slipped into that position. Even though Scalia never ranked as the most conservative justice (except for a few months in the early 1990s), he was always a core member of the conservative block.

The second question is how often Scalia found himself in the majority on the court. The chart below plots the percentage of cases in each term in which Scalia was in the majority. Here we see that when looking at all cases before the court, Scalia was on the winning side at least 75 percent of the time, except in the 2014 term.

Arguably a more interesting statistic is the fraction of non-unanimous cases in which Scalia was in the majority. During each term the court decides many cases that are “easy,” in the sense that all the justices agree on the outcome. Since the fraction of “easy” cases varies over time, by looking at non-unanimous cases we can see how much Scalia agreed with the majority in “hard” cases. Looking at this statistic, we see that Scalia was more likely than not to be in the majority, again with the exception of the 2014 term, when he was in the majority just less than half the time. Even though Scalia was more conservative than most other members of the court, his views on the merits of cases were typically shared by at least four other justices.

To understand who wins and loses on the Supreme Court, political scientists look to the “median justice.” Simply put, the median justice is the justice most likely to supply the crucial fifth vote — the justice with four justices to his or her right and four to his or her left. In the closest cases, the median justice is the one who determines the winner or loser.

What makes Scalia’s death so significant is that until another justice is appointed, we won’t have a unique median justice. Until Saturday, Anthony Kennedy was the median justice. He has been the median since Sandra Day O’Connor left the court during the 2005 term, just over 10 years ago. Early in Scalia’s term, Lewis Powell was the median, then Byron White, then David Souter. The position went back and forth between Kennedy and O’Connor beginning in the early 1990s until O’Connor left the court and Kennedy cemented the role.

Today both Stephen Breyer and Kennedy are the median justices. Until a new justice is confirmed, it is very likely that many cases during the current term will end in 4-4 ties; when that happens, the decision of the lower court is affirmed without creating a precedent.

The next justice to join the court will define who the next median justice is, which is what makes the appointment so consequential. If President Obama can get the Senate to confirm a liberal nominee, Breyer is likely to become the median. If the Republicans in the Senate can effectively delay confirmation, and if a Republican wins the presidential election and appoints a more conservative justice, then Kennedy will become the median justice once again.