No other group—not Hispanics, Muslims, or anyone else—have faced what black Americans have. Race in America is not about people of color. It’s about black people.

David Marcus recently criticized the New York Times for promoting a racial double standard when they decided to stand by Sarah Jeong despite her history of racist tweets. According to Marcus, the Times has “no problem denigrating white people in a way they would not any other group” because they have implicitly embraced “privilege theory and its hierarchies of oppression.” Marcus rightly worries that this double standard does more harm than good and “is a dangerous road to a dark place.”

While Marcus’s analysis is largely correct, there is an even bigger problem with Jeong, white privilege theory (WPT), and leftist racial discourse in general: It is profoundly, deeply unfair, and not just to white Americans. The Left’s inordinate focus on whites is especially unfair to African-Americans.

The crux of race in America isn’t that white people have “hidden advantages.” It’s that African-Americans have, and always have had, obvious disadvantages. Rather than white privilege, we should be focusing on black suffering. If there is a racial double standard, it shouldn’t be about whites at all. It should be about black people, and only black people.

African-Americans Have a Unique History in America

Especially as America has transformed into a multi-racial rather than bi-racial democracy, it doesn’t make sense to continually harp on white people. It makes even less sense to lump the African-American experience along with that of other minorities.

Take the term “people of color” (POC). It implies that we can view race relations as white people on one side and all racial minorities on the other. But that framing is nonsense. POC didn’t experience 250 years of slavery. Black people did. POC didn’t get lynched almost 4,000 times. Black people did. And POC did not put up with decades of Jim Crow and formal housing discrimination. Black people did.

WPT glosses over these horrors that were perpetrated only on African-Americans. But this glossing over, this equating of blacks with all POC, is inevitable if you fixate on white people.

Ultimately the problem with white privilege theory isn’t its focus on privilege or white people. It’s that it minimizes what African-Americans endured. Yes, it’s true that some people might be treated with “more respect and dignity based on skin color.” But as far as race is concerned, only African-Americans have been uniquely harmed by such disrespect.

The Obsession with Whites Sidelines Blacks

So rather than WPT, it would be better to adopt a black-centered framework when discussing race. In this framework, the Jeong controversy would not exist—not because white people aren’t sometimes privileged, or white supremacy didn’t exist, but because any racial discussion would be viewed entirely through the lens of blacks. In this world, it would be bizarre for anyone to be so obsessed with whites. The common liberal trope that “America is based on white supremacy” would disappear from our vocabulary, and instead become something like: “America was built on black slavery.”

While the difference between those phrases may seem small, the second does something that our current discourse tragically does not: place African-Americans at the center of our racial narrative and treat their journey with the solemn reverence it deserves. However many races there are in America today, black and non-black are still the only two racial categories that matter. Privilege theory and the Left’s approach to race gets it completely backwards.

To meaningfully engage with race, conservatives should thus do a couple things. In terms of our public rhetoric, we should acknowledge, as I’ve argued previously, that racial identity is a meaningful concept. But we should simultaneously insist that African-Americans are special, and strongly denounce anyone who suggests otherwise.

In policies, we should as much as possible treat all groups equally, something Marcus has also called for. But we should also be open to making exceptions on a case by case basis. Only two groups should qualify for exceptions: African-Americans and Native Americans. Although discussing Native Americans is beyond the scope of this essay, they too have suffered uniquely. All Americans should place the experiences of these two groups on a pedestal and never compare them to anything else.

All Immigrants Are Closer to Whites than to Blacks

Placing black Americans in a racial category by themselves and reducing the salience of “white” could also build national cohesion because it would be easier to highlight the commonalities among all immigrant groups. Muslim-Americans could appreciate that while they surely face bigotry and are “people of color,” they have much in common with the Irish and Italians. Minority immigrants must logically be compared to other immigrant groups, not African-Americans. In America’s racial hierarchy, all immigrants are much closer to whites than we are to blacks.

This approach may also help the Sarah Jeongs of the world see that whites do not automatically have privilege over non-whites: it is lunacy to assert that their race alone makes white Appalachians privileged over many second-generation Asian-Americans, whose parents were given visas because they were among the most talented doctors, engineers, and scientists in their countries of origin.

I have been called racial slurs many times since I moved to the United States. I was once even punched in the face because I am a “f-cking foreigner.” But as traumatizing as such incidents may be to non-black racial minorities, it is grotesque to compare them to what happened to black people. Black people had—and have—it much, much worse. A black-centric approach would continually remind us of this fact.

It would also help us to make sense of the overwhelming data that economic immobility afflicts African-Americans, and especially African-American men, over every other group. Given our history, why would we expect anything else? Even though others face hardship, black people stand alone in this regard.

In a 1967 interview, Martin Luther King Jr. was asked why black Americans couldn’t progress like other immigrants. King’s response, said at a time when blacks were effectively our only minority group, is still relevant today: “No other ethnic group has been a slave on American soil. That is one thing that other immigrant groups haven’t had to face.”

No other group—not Hispanics, Muslims, women, LGBT people, or anyone else—have faced what black Americans have. Race in America is not about whites or people of color. It’s about black people. White privilege theory trivializes this crucial fact.

How You Look Doesn’t Tell Us Your History

How did such a wrong-headed theory gain so much prominence? I suspect it’s because WPT is the only theory that could succeed in a multi-racial America. America never really wanted to view African-Americans as special. For a while we had no choice. But the second Hispanics and we Asians started coming over in significant numbers, WPT came along to save the day.

After all, here’s an academic theory that effectively says to black people: “Don’t view your experience as special. Sure you’re the only ones who were enslaved and lynched. And sure some minorities are significantly better off than the average American, much less black people. But you really should just lump yourself in with all people of color.”

We’ve accepted this fraudulent reasoning without realizing how ahistorical and morally bankrupt it is. Simply because they both generally happen to have darker skin than some other people, WPT has made it okay to analogize any brown-skinned immigrant, regardless of wealth or education, with an African-American whose ancestors experienced slavery and lynching.

How is okay to make that comparison? What kind of country will we be if we continue doing so? Sadly, the country we have always been.