Four years ago, Europeans were shocked by the photograph of the drowned three-year old Syrian refugee Alan Kurdi. Now, Americans are similarly horrified by pictures of El Salvadorans Óscar Alberto Martínez Ramírez and his daughter, Angie Valeria, dead on the banks of the Rio Grande. Meanwhile, in the UK we are struggling with what our immigration policy should look like after Brexit, with Boris Johnson trying to have his cake and eat it with the promise of an “Australian-style points system”.

Few subjects are more politically charged than immigration. However, like free trade, it unites most economists, regardless of their politics. Immigrants don’t take our jobs, nor do they have much impact on wages. Just look at the UK, where sustained high levels of immigration have coincided with unemployment falling to its lowest level in 40 years. More importantly, immigration makes economies more dynamic and is generally positive for productivity and prosperity. Even legitimate concerns about “brain drain” from developing countries turn out to be exaggerated, with such countries often gaining from remittances and new economic connections.

The number seeking to move countries – whether economic migration, refugee flows, or a mix – will continue to grow

The economic and political forces driving immigration are only likely to intensify, in both Europe and the rest of the developed world. They will be powered by “demand” – demographic pressures, with every single country in Europe having a fertility rate below replacement level – and “supply” – population growth in developing countries, especially in Africa, and perhaps climate change. So the number of people seeking to move countries, whether through economic migration, refugee flows or a mix, will continue to grow. At the same time, we will need migration; even Japan, long resistant, has recently begun to liberalise policy.

If the economic benefits are clear, what explains the recent political backlash? What is the connection between the election of Donald Trump, the Brexit vote and the rise of far-right populists in continental Europe? A decade on from the financial crisis, the political foundations of the postwar (and post-cold-war) liberal order appear to be crumbling.

But while anti-immigrant rhetoric and sentiment are common themes, the circumstances of individual countries are very different. In the US, Trump’s focus is on irregular migration from Mexico and Central America and its supposed impact on crime and security, although there is little or no evidence, in the US or elsewhere, to substantiate his claims. In the UK, the ostensible focus of the Brexit campaign was on EU free movement, predominantly by white eastern Europeans, although future migration from Turkey and points farther east was also a strong theme. In western European countries such as Sweden, Germany, France and Italy, rightwing populists were boosted by public reaction to refugee and migrant flows from Syria and Africa. And in Poland and Hungary, while immigrant flows are extremely small, parties in power have successfully appealed to nationalist sentiments by focusing on the threat of Muslim immigration overrunning “Christian” Europe.

Twitter is full of lunatics who talk about “race replacement” or “white genocide”. But more respectable versions of much the same argument can be found in the mainstream press. London’s population is no longer majority “white British”, but most of us were born in the UK and even more identify as British (white, black, Asian or mixed) and have British citizenship. Nevertheless, the eminent economist Paul Collier claims that the “indigenous British [have] become a minority in their own capital”. Spectator writer Douglas Murray, who argues for reducing or eliminating Muslim immigration, says London has become a “foreign country”. Eric Kaufmann, a political scientist and author of Whiteshift, suggests that we should favour immigrants from ethnic or cultural backgrounds who are easier to “assimilate” into the white majority. The favourite philosopher of some Conservatives, Roger Scruton, thinks it’s impossible for the (British-born) children of Muslim immigrants to be loyal British citizens. So the view that only white people can be “really” British, and that black or Asian Britons are still somehow alien and threatening, remains prevalent in some elite circles.

Some argue that if progressive politicians fail to accommodate these views, they will drive their traditional voters to the populist right. But there’s little evidence that this will help those most vulnerable to the lure of rightwing populism or improve public perceptions of immigration. Instead, countries such as Ireland, Canada, and Spain have combined relatively open policies with public consent by building a pro-migration coalition across much of the political spectrum.

Germany offers a particularly striking example. Amid the panic about the refugee influx in 2015, I wrote that it was an opportunity more than a threat. But many argued that it would be impossible, economically, socially or politically, to absorb so many people from supposedly “alien” cultures. But three years on, while far from perfect, the balance sheet appears mostly positive. Refugees are learning German and getting jobs. Although the far right continues to try to whip up anti-immigrant hysteria, crime is at its lowest level in almost 30 years.

Facebook Twitter Pinterest A migrant from Syria holds a picture of the German chancellor Angela Merkel as he and about 800 others arrive in Munich on 5 September 2015. Photograph: Sean Gallup/Getty Images

Brexit, paradoxically, offers a window of opportunity. The most illiberal and restrictionist prime minister in living memory is about to depart. Public concern about immigration has fallen sharply and attitudes towards its effects are more positive than for many years. Both Johnson and Jeremy Hunt, as well as the home secretary, Sajid Javid, have signalled that they want an immigration policy better attuned to the needs of the UK economy.

Politicians could make the case for liberal policy not just on economic grounds but much more broadly, defending the rights of immigrants, eg EU citizens resident in the UK, UK-born children of immigrants who are denied British citizenship, UK citizens who marry people from abroad, and so on. This would also include a more positive approach to the impacts of immigration on communities and services at a local level – by promoting integration and channelling funding to areas where there are pressures resulting from population growth. There is a chance for a “reset moment” not just in policy but in our wider public and political attitudes to immigration and immigrants: we should not let it slip away.

• Jonathan Portes is professor of economics and public policy at King’s College London and the author of What Do We Know and What Should We Do About Immigration?