Whatever happened to the good Europeans, those nice folks in small northern countries who liked to think of themselves as the world champions of liberty and tolerance?

Of course, many liberal Europeans are still alive and well. But first in Denmark, then in the Netherlands, and now in Sweden, illiberal, populist parties stirring up fear of immigrants – specifically Muslim immigrants – have managed to gain enough power to set, or at least influence, their countries' political agendas.

These parties are not confined to Scandinavia and the Low Countries, but are part of a global wave of anger against political elites, who are blamed for all of the insecurities that come with global economics, the financial crisis, and living in more ethnically mixed societies. The psychology behind the Tea Party in the United States and the anti-immigrant parties in Europe is similar, even if their policies vary.

Modern European populists don't wear black shirts, or indulge in street violence. Their leaders are youngish men in sharp suits who don't use the language of race, but that of freedom and democracy.

The Dutch Freedom party (whose only member is its leader, Geert Wilders), the Danish People's party, led by Pia Kjaersgaard, and Jimmy Akesson's Sweden Democrats claim to be the defenders of western civilisation against its main enemy: Islam. They talk about western liberties, including freedom of speech, but Wilders wants to ban the Qur'an and the burqa, and a Danish member of parliament has called Islam "a plague upon Europe".

All these countries may soon be following the Danish model, in which the illiberal populist parties pledge their support without actually governing, thereby gaining power without responsibility. Denmark's Conservative government could not govern without the support of the People's party. Sweden's recently re-elected conservative Moderate party will have to rely on the Democrats to form a viable government. And Wilders has already received assurances from the conservative and Christian Democrat parties that, in exchange for his support, the burqa will be banned in the Netherlands and immigration curbed.

The influence of these slick new populists, waging their war on Islam, goes well beyond their countries' borders. Nativism is on the rise all over the western world, and Wilders, in particular, is a popular speaker at rightwing anti-Muslim gatherings in the US, Britain and Germany.

European populism focuses on Islam and immigration, but it may be mobilising a wider rage against elites expressed by people who feel unrepresented, or fear being left behind economically. They share a feeling of being dispossessed by foreigners, of losing their sense of national, social, or religious belonging. Northern Europe's political elites, largely Social or Christian Democrats, have often been dismissive of such fears, and their paternalism and condescension may be why the backlash in those liberal countries has been particularly fierce.

The question is what to do about it. One possible solution is to let populist parties join the government if they get a sufficient number of votes. The idea of a Tea Party candidate becoming US president is alarming, to be sure, but European populists could only be part of coalition governments.

True, Hitler's Nazis took over Germany almost as soon as they were voted into power, but the new European right are not Nazis. They have not used violence, or broken any laws. Not yet. As long as this is so, why not give them real political responsibility? They would then not only have to prove their competence, but also moderate their attitudes.

That is why the Danish model is probably the worst solution, for it requires no governing ability from the populists. As long as Wilders and his European counterparts stay out of government, they have no incentive to temper their illiberal rhetoric and stop stoking up hostility towards ethnic and religious minorities.

That is what happened in the one European country that tried to bring its populists into government – Austria under Wolfgang Schüssel a decade ago. There the populist Freedom party led by Jörg Haider splintered as some opted to moderate their views in order to succeed in government. But the EU's decision to impose a form of diplomatic limbo on Austria for Schüssel's decision to include the Freedom party in his governing coalition may discourage other conservatives from going down this route. As a result, mainstream conservatives are more likely to compromise on principles that we have long taken for granted, such as civil equality and religious freedom.

Indeed, fearful of the populists' power, in or out of government, the response from mainstream conservatives – and even some social democrats – to their illiberal views has already been inexcusably soft. There are in fact plenty of ways to fight back, but not with outdated ideologies. Those who see the danger of a culture war with Muslims, or the hostile branding of minorities, should be able to influence opinion with practical arguments. It will no longer do merely to warn against racism or promote multiculturalism.

Instead, people must be convinced that without controlled immigration – and not just asylum for refugees – Europeans will be worse off. With falling birthrates, immigrants are needed to maintain European prosperity. At the same time, Europe's economies should be less enmeshed in protective regulations, so that immigrants can find work more easily.

Finally, the argument must be made more forcefully that it will be much harder to protect our societies against the revolutionary terrorism of radical Islam without the active support of all law-abiding Muslims. Europe will not be safer under politicians who claim that we are at war with Islam. On the contrary, their influence will make life not only less civilised, but a great deal more dangerous.

• Copyright Project Syndicate, 2010