THE EARLY STAGES of Europe’s refugee crisis produced heartwarming images. Volunteers flocked to Greek islands to help the refugees clambering ashore from their overloaded rubber dinghies. Locals lined the platforms of German railway stations, applauding the migrants as they stepped off the trains. “I’ve never been so proud of my country,” says Kadidja Bedoui of We Do What We Can, a voluntary outfit in Sweden, which at the peak was receiving over 10,000 migrants a week.

But as the people kept coming, more voters started to believe populists who claimed that governments had lost control. The Sweden Democrats, an anti-immigrant party that had gained notoriety with a television spot showing burqa-clad Muslims overtaking a shuffling pensioner in a race for public funds, topped polls. Eventually Sweden’s overwhelmed government slammed on the brakes, erecting border controls and tightening asylum rules. Other European countries saw the chaos in Sweden and Germany as an example of what not to do.

International law obliges governments to help refugees who reach their borders, but domestic politics constrains their room for manoeuvre. Europe presents a double challenge. First, it is a rich region with a commitment to human rights that happens to sit next to two poor, troubled and crowded ones: Africa and the Middle East. Second, it is a (largely) borderless club of geographically concentrated states with widely varying economies, benefit systems and labour markets. Asylum-seekers shop around, leaving some EU countries to bear a far heavier burden than others. That sets governments against one another.

The EU’s progress on migration and asylum rules has been agonisingly slow

Over the years the EU has taken halting steps to manage this problem. The 2004 Qualification Directive, built on the framework of the 1951 Refugee Convention, extended the scope of protection beyond the definition in the convention and provided for common grounds on which it could be granted. Other rules guaranteed minimum reception standards for asylum-seekers across the EU and set out which country was responsible for each asylum claim (usually the first one a migrant sets foot in). Member countries were unwilling to sacrifice too much sovereignty (for example, by allowing EU officials to adjudicate asylum claims), but the rudiments of a common asylum policy worked well enough when the number of arrivals was limited.

But all that changed when over 1m asylum-seekers reached Europe last year. Most of them had travelled across safe countries; indeed, a good number hailed from them, particularly those Africans who sailed to Italy from Libya. European governments had to decide whether to follow through on the promise of comprehensive protection implied in the EU’s asylum directives—and in the rhetoric of some of their leaders.

The answer turned out to be no. A Eurobarometer poll last July, even before the arrivals peaked, found that immigration had become Europeans’ biggest concern, far ahead of the economic issues that usually dominate such surveys. Populist parties like the Sweden Democrats linked governments’ handling of migration to their established claim that elite parties are incompetent or treacherous. Once the Willkommenskultur in Germany and elsewhere receded and the backlash began, panicked governments put up border controls and struck questionable deals with third countries to keep migrants out. A series of sexual assaults by migrants on German women in Cologne on New Year’s Eve further darkened the mood.

Other rich countries have not faced irregular arrivals on anything like Europe’s scale, leaving them largely free to design their own refugee policies. The United States, which signed the 1967 protocol to the Refugee Convention but not the original document, has traditionally taken in the bulk of refugees resettled by the UNHCR. But for decades its response was driven by foreign-policy considerations; between 1956 and 1968 all but 1,000 of the 233,000 refugees it admitted came from communist countries, according to Gil Loescher, a refugee analyst. America continued to resettle refugees after the end of the cold war, and remains the UNHCR’s biggest donor by far. But today security fears, sharpened after last year’s terrorist attacks in Paris and the mass killings in San Bernardino, California, make it harder for the Obama administration to take in more than its current plan for 10,000 Syrians (on top of 75,000 refugees from elsewhere). Lingering memories of the large numbers of illegal immigrants who arrived in America from Mexico and Central America in search of jobs in the 1990s may also have limited politicians’ options.

Pick and choose

Canada has taken a different approach. Its physical remoteness from any refugee streams has allowed it to pursue a generous but selective immigration policy, based mainly on its own economic needs. One-fifth of its population is now foreign-born, the highest rate in the G8, and nearly half the immigrants have a tertiary education. Last year Justin Trudeau, the newly elected prime minister, declared Canada the world’s first “post-national” state. Public confidence in the country’s migration policy allowed Mr Trudeau to campaign on a pledge to resettle 25,000 Syrian refugees, a rapidly executed policy that proved so popular that the quota was increased to 35,000. A grateful Filippo Grandi, head of the UNHCR, calls Mr Trudeau one of his two “saviours” (the other is Jim Yong Kim, the president of the World Bank).

But geographical seclusion can cut both ways. Japan showers money on the UNHCR but until 2010 accepted no refugees at all, in keeping with a closed-door migration policy that few voters seem minded to overturn. Australia does resettle thousands of refugees each year, but has taken a tough line on spontaneous arrivals since a surge in boat people from South-East Asia three years ago. The navy now intercepts all asylum-seekers at sea and either sends them back to their port of departure or directs them to detention centres in Papua New Guinea and Nauru. This expensive and legally dubious policy enjoys bipartisan political support, and Australia bristles at international criticism. Politicians have hinted at revising or even withdrawing from the 1951 convention, which Australia signed.

EU officials often fume about the opprobrium heaped upon Europe over migration while rich Gulf or Asian states look the other way. Every time spontaneous arrivals to Europe surge, so do calls for barriers to be erected, navies to be dispatched, laws to be scrapped—and allegations that Europe has succumbed to a new age of prejudice. That is unfair. In Britain, for example, where concerns about migration have been rising for years (as has immigration itself), standard measures of xenophobia have been declining: just 15% of people worry about a relative marrying someone of a different race, down from 50% in the mid-1980s, according to research by British Future, a think-tank.

Instead, the hostility springs from a fear that governments have lost the ability to manage who may or may not cross their borders, supposedly one of their primary responsibilities to citizens. “Nothing erodes public acceptance of migration like the perception that it is out of control,” says Paul Scheffer, a Dutch analyst. That is why Europe, having messed up its initial response to the migrant crisis, had no alternative but to strike its deal with Turkey in March.

Voters have also grown tired of confusing and contradictory messages. Rare is the politician who can speak honestly about immigration. When foreign workers first arrived in large numbers in western Europe decades ago, political leaders, especially in Germany, insisted that theirs were not immigration countries. But over the years voters watched a different story unfold as the sights and sounds of the streets changed and governments started to write integration policies. In countries like Spain and America, politicians’ reluctance to acknowledge that they needed foreign workers led to growing irregular immigration and, later, to embarrassing amnesties. America is still battling this legacy. Immigration from Mexico to the United States went into reverse at least two years ago, yet 38% of American voters agree with Donald Trump’s proposal to build a giant wall along the country’s southern border.

Politicians still struggle to talk about immigration, says Sunder Katwala of British Future. He thinks they should avoid dismissing public anxiety by spouting facts and figures, which preaches to the converted but confirms sceptics’ fears about detached elites. But they should also resist aping the rabble-rousing of populists who will never command majority support. The messages that resonate best with voters acknowledge the pressures of migration while calling for the benefits to be harnessed. Some politicians are getting the message. Since Sweden’s mainstream parties lifted the taboo that once surrounded debates on immigration, support for the Sweden Democrats has slid.

Most importantly, politicians should remember that unrealistic promises may come back to haunt them. David Cameron, Britain’s prime minister, will never live down his doomed vow in 2010 to reduce annual net immigration to his country to below 100,000 (“no ifs, no buts”). Politicians in Sweden and Germany may be repeating that mistake by pledging to send back tens of thousands of failed asylum-seekers, which on past form they will find hard to do. Designed largely to deter new migrants, such talk instead risks further eroding public trust.

But many liberals also need to come clean. The police and media cover-up after Cologne shattered many Germans’ confidence in their government’s policy. All sides need to accept that rich countries cannot remain immune from the global increase in mobility, and that certain sectors would collapse without migrant labour; but also that refugees are not invariably a great boon to economies, as advocates suggest. But to ensure the maximum benefit from the arrangements for everyone, the newcomers have to be properly integrated.