The Trump administration is sending colleges a stern warning: consider race as little as possible, or not at all, when you’re admitting students — a sharp reversal from the previous administration.

Under Obama, the Education Department stressed that diversity was valuable and that there were legal ways for colleges to consider race in admissions to obtain it. According to a new report from the New York Times, the Trump administration will overturn that guidance, returning to Bush-era instructions that played up the limits placed on colleges when considering race.

The move is, in one sense, just the latest swing in a decades-long back and forth: When Republicans take office, the Education Department emphasizes what colleges can’t do on race and admissions. When Democrats take control, it stresses what can be done.

This time, though, could be different. A 2016 Supreme Court decision that (barely) preserved the constitutionality of race-based college admissions, thanks to the retiring Justice Anthony Kennedy, also placed more limits on how colleges can use the practice. Meanwhile, the Trump administration’s pushback on affirmative action isn’t limited to guidance; the department is also investigating whether racial considerations discriminate against Asian students.

But the real threat to the consideration of race in college admissions has nothing to do with Education Secretary Betsy DeVos. The opportunity to appoint a conservative to Kennedy’s seat on the Supreme Court could give affirmative action’s opponents the opportunity to restrict the use of race in college admissions for a generation — dramatically altering the racial makeup of selective colleges, and therefore, American institutions as a whole.

There are already a lot of limits on considering race in admissions

Black and Latino students are still dramatically underrepresented at the highest echelons of American higher education, the hyperselective colleges where the debate about the use of race in admissions carries the most weight. (The majority of college students in America attend institutions that accept all or the majority of their applicants, and don’t carefully curate their student body to create a desirable level of racial, geographic, and other diversity.)

About 15 percent of traditionally college-aged Americans are black, according to a New York Times analysis in 2017, but black students make up just 6 percent of the freshman enrollment at “elite colleges” — a group of 100 colleges that includes prestigious public research universities, small liberal arts colleges, and the Ivy League.

Similarly, the analysis by the Times’s Jeremy Ashkenas, Haeyoun Park, and Adam Pearce found that Hispanic young adults make up 22 percent of the population, but just 13 percent of elite colleges’ freshmen classes.

The share of both black and Hispanic students at those colleges had declined since 1980. Over the same period, the federal government (and some states, including California) had systemically tightened the criteria for how colleges can use race in admissions.

Colleges can’t use a “quota system” — a requirement that, say, the student body reflects the racial makeup of the United States. Nor can colleges justify considering race in admission as a way to make up for the centuries of underrepresentation of nonwhite students in higher education, or the unequal opportunities still afforded to black and white students in public schools.

Colleges can only use race as one factor among many as part of a “holistic review” of applicants with a goal of building a diverse student body, which the Supreme Court views as a good thing for white students, too.

In 2016, the Supreme Court ruled by a 4-3 vote in Fisher v. Texas that affirmative action was constitutional, as long as the policies met a three-part test:

Universities have to give a “reasoned, principled explanation” for wanting a diverse student body.

Their affirmative action programs must be narrowly tailored, or specifically designed to accomplish a goal.

And the policies must withstand strict scrutiny, meaning colleges have to prove affirmative action was the only way to accomplish its diversity goals.

There’s enough leeway in that decision to allow the Education Department to swing back against allowing selective universities to consider race.

The Trump administration is emphasizing the limits of affirmative action

According to the New York Times, which broke the story, the Education Department will soon rescind the Obama administration’s guidance emphasizing the importance of diversity in both K-12 schools and higher education. The Times noted the department recently republished Bush-era guidance for elementary and secondary schools that encouraged schools to use race-neutral policies to make up their student body.

The message is clear: The Education Department wants colleges to consider race as little as possible, or, preferably, not at all, when admitting students.

The Education Department does have a draconian tool to punish colleges they view in violation of their interpretation of federal law by cutting them off from access to the federal grants and loans that most students use to pay their bills.

That cutoff is so dramatic that it’s nicknamed the “death penalty” and is almost never used. But a federal investigation can consume plenty of time and resources and generate lots of bad press — even if colleges are let off lightly. Under President George W. Bush in 2008, for example, the Education Department investigated whether Princeton University had discriminated against Asian applicants.

In 2015, after Bush had left office, investigators decided that the university had not, but the investigation eventually led to the release of unflattering documents where admissions officers discussed the race of prospective students in stark terms and an avalanche of negative publicity.

Under Trump, the Education Department is once again investigating a similar complaint; in the meantime, a closely watched federal lawsuit alleges that Harvard’s consideration of race in admissions discriminates against Asian students.

Reverting to the guidance emphasizing “race-neutral” alternatives — such as the University of Texas’s plan to admit the top students from graduating classes at high schools statewide — is the latest evidence that the federal government now views the use of race in admissions with a very skeptical eye.

The real action is on the Supreme Court

Advocates for race-conscious admissions policies decried the Trump administration’s latest move: “The Supreme Court has been unequivocal about the value of diversity and the core of achieving it, because it reflects the America,” Catherine Lhamon, who led the department’s office for civil rights under Obama, told the New York Times. “To retreat from those principles is damaging to the fabric of our country and to our students’ learning.”

But guidance, however influential it is in the short term, isn’t as consequential as Trump’s replacement of retiring Supreme Court Justice Anthony Kennedy.

For much of his career, Kennedy was skeptical of race-conscious admissions, siding against the use of race in college admissions and K-12 school assignment, as Vox’s Dylan Matthews recently wrote:

In 2003, he dissented in Grutter v. Bollinger, a case where O’Connor and the Court’s liberals ruled that the University of Michigan Law School’s racial affirmative action system was constitutional, and not an unconstitutional quota system. In 2007’s Parents Involved in Community Schools v. Seattle School District No. 1, he was the swing vote behind a ruling holding that Seattle, Washington, and Louisville, Kentucky’s voluntary desegregation programs were unconstitutional… He wrote the controlling opinion in 2014’s Schuette v. Coalition to Defend Affirmative Action, which upheld Michigan’s ban on affirmative action.

Kennedy’s deciding vote in Fisher v. Texas in 2016 proves that affirmative action is on the legal ropes. The next Supreme Court will likely be further to Kennedy’s right on this.

A big case on race-based admissions could already be already working its way up to the Supreme Court: the federal lawsuit arguing that Harvard’s race-conscious admissions discriminate against Asian students.

It would be unusual for the court to take up another case so quickly, given the recent precedent. But it wouldn’t be unheard of. In 2003, in Grutter, then-Justice Sandra Day O’Connor wrote in her majority opinion that race in college admissions should perhaps be reconsidered in 25 years — at which point O’Connor hoped that it would be unnecessary.

Instead, the Supreme Court took up the matter less than a decade later, when it first agreed to hear Abigail Fisher’s lawsuit against the University of Texas. That case was sent back to the lower courts, heard again in 2015, and decided in 2016, in the university’s favor, with a 4-3 majority. (Antonin Scalia’s seat was vacant, and Elena Kagan recused herself.)

If Trump successfully appoints Kennedy’s replacement, conservatives will have the votes to overturn the current precedent. That’s a much more permanent change, one that a Democratic president would find far more difficult to undo — and how elite colleges admit their students could change forever as a result.