The Justice Department’s civil rights division is recruiting attorneys for an initiative aimed at suing colleges that the Trump administration suspects of using affirmative action policies to discriminate against white applicants, the New York Times reported Tuesday evening.

An internal memo, obtained by the New York Times, asked for Justice Department lawyers interested in pursuing “investigations and possible litigation related to intentional race-based discrimination in college and university admissions.” While the phrasing of that announcement doesn’t specifically single out affirmative action or white applicants, the reported phrase “intentional race-based discrimination” set off warning bells for supporters of affirmative action.

“Colleges and universities have worked very hard to follow the law and interpret and apply the law in developing and maintaining their policies,” Kristen Clarke, head of the Lawyers’ Committee for Civil Rights Under Law, told VICE News. “There’s progress that we have to recognize that is fragile and threatens to be unraveled by the Justice Department’s latest actions.”

Because the Justice Department is backed by the might and resources of the federal government, Clarke added, even an announcement from the department that it will closely watch affirmative action could “cause a lot of chaos among colleges and universities.”

Right now, universities can consider applicants’ race as part of a broad, “holistic” evaluation, as the Supreme Court has recognized that having a diverse campus can better a college student’s education. Last year, the Supreme Court narrowly ruled that the University of Texas Austin could continue a race-conscious program, but made it clear that UT Austin’s program was unique and that the court could hear future cases on other affirmative action programs.

Roger Clegg, president of the anti-affirmative action Center for Equal Opportunity, told the New York Times that the civil rights division’s initiative was “long overdue.”

“The civil rights laws were deliberately written to protect everyone from discrimination,” he said, “and it is frequently the case that not only are whites discriminated against now, but frequently Asian-Americans are as well.”

The affirmative action initiative will reportedly be run out of the civil rights’ division’s “front office,” where political appointees typically work, which is also a cause for concern for Clarke.

“It appears that they are developing an effort that will be driven out of the front office of the civil rights division, without support from career attorneys inside the division’s education section, who have been doing this work for years,” she said, which she said signals that the initiative will be “hyper-politicized.” “This latest move makes it clear that the division is moving far afield from its core mission.”