While Justice Anthony Kennedy squashed rumors about his retirement at the end of this current term, it’s unlikely he will continue serving on the Supreme Court for all of President Donald Trump’s time in office, according to an NPR report.

Kennedy has already hired clerks for this coming term, but has not yet filled the positions for the following term, which begins in October 2018. The 80-year-old justice apparently told applicants for the 2018 term that he’s considering retirement, according to NPR.

Kennedy’s retirement would be consequential as his vote is often what decides the outcome of many cases, especially this past month following the appointment of Trump’s Supreme Court nominee Neil Gorsuch, who so far has voted 100 percent of the time with his most conservative colleague on the court, Justice Clarence Thomas.

Gorsuch votes second most often with Justice Samuel Alito, according to the NPR report. On the other side of the ideological spectrum, Justices Ruth Bader Ginsburg and Sonia Sotomayor voted together most often this term, leaving Kennedy and Chief Justice John Roberts on the center-right and Justices Stephen Breyer and Elena Kagan, at the center-left.

Evidenced in the last few weeks of the current term, Kennedy and Roberts votes are what allowed the travel ban to go into effect with certain rules and allowed same-sex couples to have the same benefits linked to marriage as heterosexual couples, like having both parents names on a birth certificate.