People around the world are hearing more foreign accents than at any time in human history, as more people move around than ever before.

In 2013, an estimated 232 million people (PDF) lived outside the country they’re from, seeking refuge or employment. In the US, 13% of the country’s 316 million residents are immigrants. Around 60 million of the world’s migrants have been displaced by war, including over a million Syrians, Iraqis, and Libyans who have sought asylum in Europe since 2014. These numbers don’t even capture the migration that happens within countries. In 2015, over 260 million rural Chinese are living in cities like Shanghai, Beijing, and Hong Kong, and the new arrivals don’t talk like the natives.

What are the consequences of this linguistic jumble? Well, if you take what scientists are learning about how our brains process foreign accents, and stir negative stereotypes into the mix, then what starts as linguistic difference can often spiral into systemic social justice issues, as people with foreign accents encounter problems with access to jobs, health care, and education.

“That’s a basis on which it’s easy to make judgments about a person’s cultural affiliation or education,” says Ingrid Piller, a sociolinguist at Macquarie University in Brisbane, Australia, who studies language and migration and blogs about them. “It’s a springboard for a lot of heavy assumptions which may or may not be true.” Because of migration, says Piller, “language becomes a social justice issue in ways that are quite new.”

How our brains treat foreign accents

Scientists are finding that the reasons for that discrimination may actually start with how our brains process foreign accents in the first place. For one thing, if you’re not used to it, it’s simply harder to understand someone who is speaking with a foreign accent. The utterances are about 30% longer, they contain many pauses, the individual sounds may differ from ones we’re used to, and where stress is located in the word and in the sentence also differs. This perceptual difficulty can alter how non-native speakers are perceived.

In one study, Shiri Lev-Ari, a psycholinguist at the Max Planck Institute of Psycholinguistics in Nijmegen, asked non-native speakers of Polish, Turkish, Austrian-German, Korean, and Italian to record banal statements like “Ants don’t sleep” in English. Native English speakers recorded the same ones. When native English speakers rated the recordings for their veracity, they rated the speakers with the heaviest accents as least true, while native speakers were rated most true.

“We’re less likely to believe something if it’s said with a foreign accent,” says Lev-Ari. In her view, negative judgments are the result of the additional effort that our brains must make to process foreign speech. Our brains then shift the blame for this effort onto the veracity of the speaker.

In another experiment, Lev-Ari showed that native speakers remember less accurately what non-native speakers say. This is because “we expect non-native speakers to be less proficient speakers, so we rely on our expectations about what they’re going to say, rather than what they actually do say,” she explains.

From experiments like these, it can be tempting to conclude that the cognitive difficulties imposed by non-native speech inevitably lead to social discrimination. After all, its linguistic bases seem so unavoidably natural. Even babies prefer native speakers of their language than they do non-natives.

But as Lev-Ari points out, the more we’re exposed to foreign accents, the more our brains train themselves to parse the speech more efficiently. In as little as four minutes, a person can improve how much they understand of speech with a foreign accent. So something else is going on.

Life with a foreign accent

Even though our brains are plastic, people’s stereotypes of non-native speakers are much less so. Ultimately, it’s those stereotypes that shape what will happen when people hear foreign accents.

Take the case of a hypothetical Syrian refugee. Admittedly, he has many bigger problems to face immediately than how he sounds when he speaks German, French, or Turkish. But if he settles in those places, he’s going to have to contend with his foreign accent, which “only becomes an issue when they’ve spent some time in the country and acquired an advanced level of proficiency in grammar and vocabulary,” Piller says.

Say that he’s accepted for asylum in Denmark. He’ll be expected to start learning Danish, and a small financial allowance he receives from the government will be docked if he doesn’t attend free classes. (One irony of this requirement is that many Danes expect that the Syrians will go back to their country when the war is over.)

Suppose that this person sticks with Danish lessons. He may acquire a solid sense of the grammar and a good vocabulary, but because he learned the language as an adult, he’ll speak it with a foreign accent. That’s not a reflection of his interest in Danish or his abilities as a learner; it’s an unavoidable effect of having learned the language after a certain developmental window has closed.

The result will surprise native Danish speakers. “We never hear our language spoken with an accent,” says Nicolai Pharao, a sociophonetician at the University of Copenhagen. “We would much rather switch to English.” (This tendency makes the requirement to study Danish even more puzzling.)

Native speakers don’t necessarily attach negative stereotypes to all foreign-sounding Danish speech. A person with an American accent is thought to be rich, while someone with a French accent will be thought well-educated. But if you’re from the Middle East, your Danish accent will probably be lumped together with a dialect of Danish that grew up around the language experiences of Turkish and Middle Eastern immigrants in the 1980s and 1990s.

They created a version of Danish that contains words and intonations from Arabic, Persian, and Turkish. In Denmark, this dialect is called gadesprog, or “street language.” Many European countries have similar “street” dialects. In Germany, this dialect is called Kiezdeutsch; in Sweden, it’s called Rinkeby Swedish, after an immigrant neighborhood of Stockholm. (Here’s a YouTube video about “rinkebysvenska.”) According to the stereotypes, these dialects are typically spoken by under-educated young people probably involved with crime. The reality is that they’re widely spoken youth dialects, spoken by adolescents from immigrant and non-immigrant families alike. (This coffee commercial plays with the youth connection by showing two grandmas speaking Dutch straataal.)

So a new arrival’s foreign accent will always be heard through a complex social situation that differs from country to country, and which includes language attitudes and stereotypes. Sometimes exposure to those foreign accents reinforces the negative stereotypes, even as accented speech becomes easier for people to comprehend.

“Lots of people living in cities will encounter non-native Danish whenever they take a taxi,” Pharao notes, “but that hasn’t done anything to make them more tolerant.”

A new linguistic reality

Speakers with foreign accents can face discrimination even in a country with a diverse population and a long immigrant history. The US Equal Employment Opportunity Commission (EEOC) is tasked with enforcing federal laws prohibiting employment discrimination and has settled several foreign-accent discrimination lawsuits since 2010. In the most recent of these lawsuits, a Walmart employee from Gambia, Ebrima Jallow, complained that the store manager had mocked his accent. (The manager also made anti-African and Islamophobic comments.) When Jallow complained, the manager threatened to fire him and told other employees not to work with him. The EEOC and Walmart settled for $75,000.

The United States is unique because it’s one of the few industrialized countries with legal mechanisms and institutions that can be used to protect the linguistic status of migrants. Another country with a long history of linguistic migration, Australia, has nothing similar. It is actually becoming less accommodating to speakers of other languages, notes Piller, a native German speaker who has lived there since the late 1990s. The “populate or perish” mantra after World War II has been replaced by fears of being inundated by migrants. Those who are allowed in aren’t given enough resources for English learning.

Stereotypes of foreign accents as well as other language attitudes are becoming more open in Germany, she notes. Until the 1990s, the attitude used to be, if you’re new to the country, you have to suck it up and learn German. Today, some universities are preparing to allow refugees to take classes in English. There is a lot of openness about acknowledging that services are needed in many languages, as well as public discussions about how quickly people can be integrated linguistically. “Now, Germany is probably quite well-prepared to recognize the specific linguistic needs of new arrivals,” she says.

But much remains to be done. “We’re going to increasingly be exposed to accented speakers in the workplace, in doctors’ offices, in our neighborhoods, and in all the places we encounter folks we don’t know well,” says Alene Moyer, a linguist at the University of Maryland, whose new book, Foreign Accents, probes the social implications of sounding like a non-native speaker.

“The question is,” she adds, “whether or not we accept and acclimate to that reality.”

This post was updated to correct the location of the Max Planck Institute of Psycholinguistics. It is in Nijmegen, not Amsterdam.