Some Asian Americans have had a beef with race-conscious admissions for decades. Traditionally one of affirmative action’s biggest beneficiaries—so much so that the practice originally contributed to their reputation as the “model minority”—Asian Americans started to turn against it once they were no longer considered underrepresented. They’ve since been inundated with stories about prodigiously accomplished Asians being rejected from top schools. Today, they’ve in many ways become the face of the anti-affirmative action movement.

Last Thursday, the Supreme Court upheld the race-conscious admissions policy at the University of Texas at Austin, a decision many observers described as “a victory for affirmative action.” For many Asian Americans who would like the admissions process to be more colorblind, though, the fight is far from over. Fisher v. The University of Texas at Austin, they argue, only scraped the surface of a much deeper, much more insidious problem: the exclusion of deserving Asian Americans from higher education.

But it’s far from clear that race-conscious admissions policies actually put the so-called “model minority” at a disadvantage. Nor is it clear that Asians and affirmative action are the foes that the headlines and lawsuits and petitions make them out to be.

California has prohibited affirmative action at public institutions for two decades, and the ban certainly hasn’t hurt Asian Americans, who today account for a plurality—about a third—of the students at University of California schools despite making up just 15 percent of the state’s population. But when the state senate introduced a Democrat-backed amendment that would’ve asked voters whether to lift the ban, Asian Americans staged public demonstrations and wrote blistering editorials; they hosted a Republican-registration drive (“to scare the Democratic Party”) and gathered on TV talk shows to warn viewers of the proposal’s implications.

“This is the most racist bill ever,” said a participant on one such show. “We come from a faraway land, China, and [we came] here to pursue fairness, equal education opportunities. Education is an essence and a core value of our culture, and we pass it along to generations and generations … In the future, when [our kids] grow up, it doesn’t matter how much we devote to their education, it doesn’t matter how much effort they put into their own education—years of work will be gone, only because of their skin color.”

Much of the wrath has been targeted at Ivy League schools, which consider a range of academic and non-academic factors in the admissions process. Students for Fair Admissions (SFFA)—a group representing primarily Asian American students and parents—contends in a lawsuit that Harvard College uses implicit racial quotas even though they’re illegal. (It accuses the University of North Carolina at Chapel Hill of similar allegations in a separate lawsuit.) Despite being the country’s fastest-growing minority group, and despite applying to college in greater and greater numbers, the percentage of Asians admitted at elite schools has, according to SFFA, essentially flatlined over the last two decades. “That suggests that Harvard and the other Ivies have a hard-fast, intractable quota limiting the number of Asians that they will expect,” said Edward Blum, a scholar at the American Enterprise Institute and the president of SFFA.