Chief Justice John Roberts, long an affirmative action skeptic, questioned whether the “small” gains the university saw from a more diverse student body were worth “the extraordinary power to consider race.”

Justice Samuel Alito expressed a faith that guaranteeing admission to the top 10 percent in each high school would be race-neutral on its face, but would still guarantee a degree of diversity in college admissions. That contention is counter to the finding in the original 5th Circuit decision on the Fisher case, and, as Gregory Garre, a former U.S. Solicitor General and the attorney representing the University of Texas observed, it was a solution that only worked if the state perpetuated “a system in which not only are minorities going to separate schools, they’re going to inferior schools.”

Justice Ruth Bader Ginsburg seemed to agree. Under Alito’s assumption, diversity at the university level is, she said, “totally dependent upon having racially segregated neighborhoods, racially segregated schools, and it operates as a disincentive for a minority student to step out of that segregated community and attempt to get an integrated education.”

As with many cases before the Roberts court, attention has turned to Justice Anthony Kennedy. Kennedy has a long history of voting to upend affirmative action plans and reject race as a determining factor in a host of pro-diversity programs. As with the 2013 decision on Fisher, Kennedy again appeared to have doubts that there was enough evidence presented to pass final judgment on the Texas plan.

In the Supreme Court’s previous decision, it left intact the finding that race can be a factor when absolutely necessary to obtain diversity as the end goal. The court could reject the appeal or remand this case back to the 5th Circuit again and keep that precedent in place. But if the end goal itself is in question, as the comments of Alito and, more emphatically, Scalia imply, then the foundation for a long list of decisions on diversity initiatives could be back in play.