Boots Riley’s new film, Sorry to Bother You, centers on a black man named Cassius “Cash” Green (Lakeith Stanfield) who’s struggling at his telemarketing job—until he starts talking with a “white voice,” and customers stop hanging up. This stark difference seems like it could be another one of the film’s fantastical elements, especially because Stanfield’s white voice is played to cartoonish effect by white comedian David Cross. Unfortunately, however, people really do respond differently to different accents and patterns of speech—often with negative effects on communities of color.

Discrimination against people who speak English with a nonstandard accent or nonstandard grammar is called “linguistic prejudice.” Despite sounding relatively benign, it has a severe impact on people throughout their lives, starting in kindergarten and reaching into searches for housing or employment and interactions with the justice system.

First, in case you doubted it, people really can identify someone’s race just by talking to them over the phone. A 1999 study by the linguists Thomas Purnell, William Idsardi, and John Baugh found that people were able to identify someone’s race with over 70 percent accuracy just by hearing them say “Hello.”

Arguments that stigmatize dialects as “illogical” abound, but are based on shaky logic themselves and aren’t applied equally to all languages.

People then use these racial identifications to discriminate. As part of the same study, Baugh, who is black and is familiar with standard English as well as African American Vernacular English, or AAVE, and Chicano English, called landlords in five different cities while alternating which dialect he used for each call. In predominantly black areas like east Palo Alto and Oakland, California, Baugh made slightly more viewing appointments when he used AAVE, compared to standard English or Chicano English. However, in whiter areas like San Francisco, Palo Alto, and Woodside, California, standard English outperformed AAVE and Chicano English by as much as 40 percent.

Dialect discrimination doesn’t just have an impact on people in the housing market. It also affects how likely you are to be believed when it’s most important: in the justice system. A 2016 paper by John R. Rickford and Sharese King argues that George Zimmerman’s acquittal in the Trayvon Martin murder case was based in part on linguistic discrimination. Jurors described the prosecution’s star witness, Rachel Jeantel, as “hard to understand” and “not credible” because she spoke AAVE. Her testimony was not mentioned once in the more than 16 hours of jury deliberations, which no doubt contributed to Zimmerman’s acquittal.

Linguistic prejudice also harms children in the educational system, where nonstandard English may be judged unintelligent or lazy. This has lasting impacts, as teachers put these children in less challenging classes and expect poorer performance from them, which is often a self-fulfilling prophecy.

Overall, public perception of nonstandard English dialects and their speakers is incredibly negative. The Yale Grammatical Diversity Project, where I’m a member of the research team, performs large-scale surveys exploring nonstandard grammatical constructions and where in America they are used. As part of these surveys, people are asked to rate how natural they find a variety of sentences, some of which include nonstandard grammar, and are invited to share any comments they might have. Our respondents have called sentences “cave-man speak,” said they made them “feel a bit gross,” and railed against “illiterate” people who are “too stupid to learn English.”

But perhaps these insults are justified. After all, aren’t some forms of English simply more correct than others?

According to linguists, the answer is a resounding and unanimous “No.” It turns out that our perceptions of nonstandard dialects are a reflection of our attitudes toward the kind of people who speak those dialects, not of any facts about the dialects themselves. For example, people who speak with a British accent are thought to be smart and wealthy. Meanwhile, AAVE speakers are considered to be careless or stupid.

Arguments that stigmatize dialects as “illogical” abound, but are based on shaky logic themselves and aren’t applied equally to all languages. Consider negative concord, or double negatives, which are prevalent in AAVE and many other non-standard dialects. People often say that two negatives in one sentence should cancel each other out, creating a positive, and that double negatives are therefore incorrect and illogical. However, Italian, Polish, and Hebrew allow two negative words to appear in a single negative sentence, just as AAVE does. If that’s illogical in AAVE, it should be illogical in Italian too—the rules of logic don’t change depending on what language you speak. And yet, this accusation of illogicality is only applied to AAVE, and never to other languages with negative concord.

A similar story applies to null copula, where the verb be is omitted from a sentence (as in “He tired”). Standard English allows null copula in certain sentences; for example, the sentence “He found it (to be) difficult” does not need to include “to be” in order to sound perfectly grammatical and natural to standard English speakers. Some nonstandard dialects just allow be to be deleted in a larger (though still rule-governed) set of sentences.

Linguistic prejudice can even affect people of color who speak perfectly standard English, as shown by “accent hallucination.” In a 1992 study by Donald L. Rubin, participants listened to a four-minute recording of a lecture delivered by a native English speaker from Ohio. While listening, half the participants were shown a photograph of a white woman. The other half were shown a photograph of a Chinese woman. The group who had seen the Chinese woman not only rated the lecture as less clear and understandable but, incredibly, had measurably worse comprehension of the lecture compared to the participants who had seen the white woman—despite the fact that they had heard the exact same recording.

None of this is to say that everyone who criticizes nonstandard English is a virulent racist. Most people’s prejudices about language are implicit and unconscious, and were reinforced early in childhood by an education system that teaches only standard English. This is done in order to prepare students for the workplace, where speaking standard English provides a clear advantage. But this also means that departing from standard English is viewed, or even explicitly taught, to be incorrect, uneducated, or illogical.

Linguistic prejudice is one of the last widely socially acceptable ways to discriminate against certain racial or socio-economic groups, by using criticism of nonstandard dialects as a proxy for criticism of their speakers. Confronting these prejudices won’t be easy, especially because the way we speak is so intrinsic to our identities and perceptions of ourselves and others. But being conscious of our attitudes about language is a major step forward.