As European countries grapple with the backlash to immigration, it’s become clear that there’s a growing cognitive dissonance between the global elite and ordinary voters.

Immigration has major benefits for both migrants and the host countries, but it’s important to remember that not everybody gains from the phenomenon.

Like free trade and finance, migration creates winners and losers. If Europe’s political elite doesn’t come up with ways to compensate low-skilled native workers who feel threatened or displaced by migrants, the anti-immigration wave will continue to surge.

My family’s history is a case in point. As the son of two unskilled Spanish migrants in cosmopolitan Basel, Switzerland, I enjoyed the pleasures of multiculturalism in school, got my Ph.D. in the U.K. and now teach in Madrid, where the students in my classroom come from all around the world.

But my father had a very different experience. He was an unskilled worker in Basel, with no command of the language. That made him easily displaced by newly arrived refugees from the war-torn Balkans who spoke German. In his world, there was no real cultural exchange: Workers tended to group by nationality or cultural affinity. Today, the images of hundreds of illegal migrants reaching the beaches of southern Spain are a source of concern for him.

One of the principle factors fueling the panic against immigration in the rest of Europe is the perception that the authorities have lost control.

Migration is a great tool to fight global poverty. Young migrants — skilled or unskilled — from poor countries earn higher wages than they would back home, allowing them to send remittances to their countries of origin. If they choose to go back, they take with them the skills, networks and know-how they acquired in the West.

The gains are substantial for host countries too. Unskilled immigrants do the jobs that native workers do not want, and those with skills tend to fill the gaps in technical, specialized and managerial positions. They increase overall productivity and enrich the intellectual and scientific output of the country. Societies that absorb people from different religious, ethnic, linguistic and cultural backgrounds are also technologically more advanced. It is no coincidence that Silicon Valley is a social melting pot.

The problem is that these benefits rarely reach the host country’s unskilled population — leaving them feeling angry and betrayed. For those who feel left behind, identity and cultural issues become increasingly important, creating social turmoil.

At anti-immigrant protests in the German city of Chemnitz last month, for example, one demonstrator said she was there because of her low pension and her son’s low salary. Asked why she was protesting against immigrants and not in favor of more income redistribution, she said her anger needed to be directed against someone.

One of the principle factors fueling the panic against immigration in the rest of Europe is the perception that the authorities have lost control. There are often no repatriation agreements that would allow European governments to send back economic migrants. Those denied asylum will sometimes stay illegally without the possibility of holding down a formal job. This forces them to beg or embrace criminal activities.

Many EU countries, especially France and Italy, have expanded their bilateral readmission agreements with third countries, especially in Africa, to counter the trend. But these arrangements do not cover the entire African continent, and some countries, including Germany, have few agreements with sub-Saharan Africa.

If the situation — or more accurately, the perception of the situation — worsens, more Europeans will come out against the application of the Geneva Convention that protects refugees. Italy, for one, is getting close to that point.

Words are no longer enough. Europe urgently needs to come up with solutions. Without strong distributive and compensatory policies, there is no way to address the real economic anxiety migration creates for the “losers” of this phenomenon.

The most persuasive idea, advocated by the World Bank, is to develop legal channels of migration based on the demand of the job market and better education and retraining systems to cover the displaced local workforce. These programs could be financed by the introduction of progressive fees on working visas. Crucially, these visas would be for time-limited periods, and only those migrants who integrate well would then acquire the nationality of the host country.

If we as a society want to continue to benefit from immigration, we will have to develop migration policies that are less liberal but take into account the concerns of those they displace.

But for such a scheme to work, borders would need to be better controlled, repatriation agreements further developed and better implemented, and inspections against hiring of irregular workers would have to be more intrusive and widespread.

It’s a tall order, but ultimately, one on which we have to deliver. If we as a society want to continue to benefit from immigration, we will have to develop migration policies that are less liberal but take into account the concerns of those they displace. If not, the far right will destroy the opportunities for progress altogether.

Miguel Otero-Iglesias is senior analyst at Elcano Royal Institute and professor at the IE School of Global and Public Affairs.

