Europe faces a migrant crisis, but not the one we imagine. The dilemma it faces is this: on the one hand, any moral and workable immigration policy will not, at least for the moment, possess a democratic mandate; on the other, any policy that has popular support is likely to be immoral and unworkable.

This dilemma exists not because European populations are particularly drawn to immoral or unworkable policies but because of the way that the immigration issue has been framed by politicians of all hues over the past 30 years.

Politicians have recognised a need for immigration. They have also promoted it as a problem that must be dealt with. At the same time, politicians often express disdain for the masses, many of whom they regard as irrevocably racist and incapable of adopting a rational view on immigration. Gordon Brown’s description during the 2010 election campaign of pensioner Gillian Duffy as “a bigoted woman” because of her worries about east European migrants captured the contempt of elite politicians for the little people’s immigration concerns.

The fallout over the events in Cologne on New Year’s Eve reveals how toxic this approach can become. Hundreds of women seem to have been harassed, mainly by men of North African or Middle Eastern appearance. The authorities, fearful of the public reaction, decided to cover it up. When the details emerged, they created an even greater backlash against migrants.

The contradictory needs and desires have also resulted in an incoherent, unworkable set of policies that have been paradoxically exacerbated by the development of free-movement policies within the EU. The Schengen area, the group of EU nations that have abolished passport and other controls at their common borders, was established in 1985. Today, it comprises 22 of the 28 EU members, with another four committed to joining. Only two countries – Britain and Ireland – are stand-outs.

The dream of free movement within the EU has also spawned paranoia about the movement of people into the EU. The quid pro quo for Schengen has been the creation of a Fortress Europe, a citadel against immigration, watched over by a hi-tech surveillance system of satellites and drones and protected by fences and warships. When a journalist from Germany’s Der Spiegel magazine visited the control room of Frontex, the EU’s border agency, he observed that the language used was that of “defending Europe against an enemy”.

Many of the policies enacted over the past year give a sense of a continent at war. In June, an emergency EU meeting came up with a 10-point plan that included the use of military force “to capture and destroy” the boats used to smuggle migrants. Soon afterwards, Hungary and other east European countries began erecting razor-wire fences. Germany, Austria, France, Sweden and Denmark suspended Schengen rules and reintroduced border controls. In November, the EU struck a deal with Turkey, promising it up to $3.3bn in return for clamping down on its borders. This month, Denmark passed a law allowing it to seize valuables from asylum seekers to pay for their upkeep.

Despite the sense that the crisis is unprecedented, there is nothing new in it or the incoherence of the EU’s response. People have been trying to enter the EU, and dying in the attempt, for a quarter of a century and more.

Until 1991, Spain had an open border with North Africa. Migrant workers would come to Spain for seasonal work and then return home. In 1986, the newly democratic Spain joined the EU. As part of its obligations as a EU member, it had to close its North African borders. Four years after it did, it was admitted into the Schengen group.

The closing of the borders did not stop migrant workers trying to enter Spain. Instead, they took to small boats to cross the Mediterranean. On 19 May 1991, the first bodies of clandestine migrants were washed ashore. Since then, it is estimated that more than 20,000 people have perished in the Mediterranean while trying to enter Europe.

Spain possesses two imperial outposts in Morocco, Ceuta and Melilla. After joining Schengen, Spain built a €30m bulwark to seal off these enclaves off from the rest of Africa. The EU began paying the Moroccan authorities to round up and detain any potential migrants, often with great brutality.

The Spanish approach provided the template for subsequent EU migration policy: a three-pronged strategy of criminalising migrants, militarising border controls and externalising controls by paying non-EU states, from Libya to Turkey, huge amounts of money to act as Europe’s immigration police. None of this has prevented migrants from trying to enter Europe. They have merely been forced to find different, and often more hazardous, routes. This is one of the reasons why so many of them now travel through Greece and the Balkans.

Large though they are, it is worth putting into context the numbers of migrants coming to Europe. A million refugees and migrants arrived last year. That represents not much more than 0.1 per cent of the EU population. There are already 1.3 million Syrian refugees in Lebanon – 20% of the population. That is the equivalent of Europe playing host to 150 million refugees. Turkey, the country on to which the EU would like to offload its migrants and refugees, already hosts two million refugees.

Compared to elsewhere in the world, refugees are hardly “flooding” into Europe. Some of the poorest countries already bear the greatest burden and that is the most reprehensible aspect of the EU’s policy: at its heart seems to be the idea that dealing with migrants and refugees should be an issue only for poor countries. The second factor in the current migrant crisis is the political context. The postwar political system in Europe, built around the divide between social democratic and conservative parties, is being dismantled. The political sphere has narrowed and politics has been reduced to a question of technocratic management rather than social transformation. One way in which people have felt this change is in a crisis of political representation, a growing sense of being denied a voice and of political institutions being remote and corrupt.

Immigration has played almost no part in fostering the changes that have left so many feeling disaffected. It is not responsible for the weakening of the labour movement, or the transformation of social democratic parties, or the imposition of austerity policies. Immigration has, however, come to be a means through which many perceive these changes. The EU, meanwhile, has become symbolic of the distance between ordinary people and the political class. The result has been growing hostility towards migrants and panic among policy-makers.

Facebook Twitter Pinterest ‘As Germany’s chancellor, Angela Merkel, has discovered, pushing through liberal immigration policy without winning public support can have disastrous consequences.’ Photograph: Bernd Settnik/dpa/Corbis

So what is to be done? Is it possible to square a moral and workable migration policy with the democratic desires of the European public? Many seem happy to jettison the need for a democratic mandate, others to forgo the establishment of a moral and workable policy. The predominant view is that Europe needs stricter controls, bigger fences, more military patrols. Such policies may attract popular support but although those who promote such policies portray themselves as “realists”, it is not just an immoral approach but an unworkable one. The story of the past 25 years is that however strong one makes Fortress Europe, fences and warships will not deter migrants.

Nor will tighter controls change public opinion. Making Europe more of a fortress will do nothing to assuage the sense of marginalisation and voicelessness that many feel. The “idealists”, on the other hand, seek to promote a more moral immigration policy, but also seem willing to ignore the democratic will to do so. This approach is no more workable or moral than the realist stance. No policy to which the public is hostile is likely to work in practice.

As Germany’s chancellor, Angela Merkel, has discovered, pushing through liberal immigration policy without winning public support can have disastrous consequences. Last August, Germany unilaterally suspended the “Dublin accord”, the EU rule that migrants have to apply for asylum in the first EU member state they enter. Merkel made little effort, however, to convince the German people of the worth of her new policy. There was an intense backlash and within a fortnight Germany had reversed its stance and reintroduced border controls. The consequence was greater hostility both towards migrants and Merkel herself.

In immigration policy, there are no quick fixes that allow us to tie together the moral, the workable and the democratic. The migrant crisis is a longstanding one and whatever policies are conjured up will not solve it this year or next. Indeed, the key problem lies not at the level of policy at all, but at the level of attitude and perception. That is why we need to think more in the long term.

Liberal immigration policies can be enforced only by winning public support, not in spite of public opposition. Winning such support is not a chimera, there is no iron law that the public must be irrevocably hostile to immigration. Large sections of the public have become hostile because they have come to associate immigration with unacceptable change. That is why, paradoxically, the immigration debate cannot be won simply by debating immigration, nor the migration crisis solved merely by enacting migration policies. Anxieties about immigration are an expression of a wider sense of political voicelessness and disengagement. Until that underlying political problem is tackled, the arrival of migrants on Europe’s shores will continue to be seen as a crisis.

Kenan Malik’s most recent book is The Quest for a Moral Compass

• Comments will be opened later today