On a stormy night in September 2002 a wooden fishing boat carrying 150 Liberian asylum-seekers broke up on a reef near the long, sandy beach at Realmonte, on Sicily’s southern coast. Tourists were dancing at a café nearby, but such was the noise of the freak hailstorm on the plastic roof that it was some time before they heard the cries for help coming from the water. Of the 35 Liberians who drowned, one was a 15-year-old girl. Most of the dead had no names; their graves, high in the walls of the cemetery at Canicatti, are marked only by a single letter of the alphabet, in bold black type, to distinguish one from another.

The reaction of the tourists and the local people, once they had recovered from the horror, was one of surprise. Who were these strange Africans, washing up on their shore? But the survivors were welcomed, fed and looked after; one of the women, who was pregnant, was given nappies and baby clothes. There was little press coverage of the event.

That was 14 years ago. Today, when the weather in the Mediterranean is fine, boats bring over a thousand people each day to the Greek island of Lesbos alone. Others ­arrive in Europe through Malta, Lampedusa and southern Italy and by the land route through the Balkans. About 42,500 people are said to be leaving their homes every day to seek protection. These people come from Afghanistan and Eritrea, from Libya, Mali, Nigeria, Somalia, Sudan and Tunisia, and from Syria, which on its own contributes 52 per cent of all the new arrivals. Well over four million Syrians are now refugees in 107 countries. There are young men and women, whole families, children on their own, and many of their sea journeys have been preceded by terrifying land crossings, negotiating deserts, bandits and traffickers. As the numbers keep growing, so the figures rather than the people become the story: so many on a single day in April, so many through Serbia, so many others into Italy. It is in order to turn these numbers back into people, each with an identity, past, character, fears and hopes, that three journalists have written new books about what Angela Merkel has described as the defining hum­anitarian issue of our age.

Wolfgang Bauer is a reporter for Die Zeit. In April 2014, taking with him a photographer and posing as an English teacher from the Caucasus, Bauer joined a Syrian friend planning to cross the sea from Egypt to Italy. He grew a beard and bought a false ID, but even so it was a perilous undertaking, because people-smugglers have little time for reporters who might expose their lucrative rackets. Most of the sea journeys are nightmares, involving leaking and capsizing boats and gangs of violent smugglers, often drugged, but Bauer experienced one of the worst. Even before his group left Egypt, they were kidnapped by a rival gang on to whose territory his smugglers were said to have strayed. What followed were days in squalid, unfurnished rooms while the gangs brokered a deal. Bauer excellently re-creates the predatory, tense world of these shadowy men, whom he likens to travel agents, constantly on the phone, bribing, threatening, changing plans. The man who negotiated his trip confided that he had sent 250 boats across the Mediterranean in 2013, each carrying about 200 people.

Once the deal was made, the group was moved to a beach – another dangerous moment, for here, as dusk falls, bandits arrive and smugglers try to extort more money. Here, too, families get separated and children disappear. Bauer never made it across the Mediterranean: dumped by his smugglers on an island and arrested by coastguards, he was eventually rescued by being able to show a European passport. His fellow travellers were not so lucky.

Charlotte McDonald-Gibson is a former deputy foreign editor of the ­Independent. Dividing her inquiry between the five years since the start of the Arab spring and chronicling the most significant moments in that period, she follows in the footsteps of a cast of travellers. One of these is Majib, an 18-year-old working in Libya when Gaddafi began to round up migrant workers. The son of a prosperous doctor and philanthropist, Majib had seen his father killed by a mob during fighting between Christians and Muslims. Subsequently the young man was kidnapped, smuggled and enslaved. Then there are Sina and Dami, an Eritrean husband and wife, both engineers, whose lives have been made impossible by President Isaias Afewerki’s repressive policies, which have driven over 320,000 of his countrymen abroad. McDonald-Gibson keenly evokes the hell of their voyages: water lapping over the sides of boats, nothing to eat or drink, failing engines, bodies thrown overboard. To read these vivid stories is to understand not just the enormity of what is taking place, but the courage and desperation of those who embark on them.

In March 2015, the Guardian appointed Patrick Kingsley as its first migration correspondent; he set out to visit 17 countries and write about people as they fled across deserts and seas. Of the three books under review, The New Odyssey is the most analytical, consistently trying to make sense of information and pin down the facts. Kingsley has gone further than the others in trying to explore the economics of the smugglers and their accomplices. He writes at fascinating length about the “second sea”, the Sahara, which most people from the Horn of Africa have to cross and where many die even before they reach the Mediterranean. In Agadez, he discovers about 50 compounds where smugglers gather their customers before despatching them in overcrowded Land Cruisers across the sands to waiting boats, with the connivance of the Nigérien military and police. Interviewing smugglers, he spells out the profits: with each of a group of 100 paying $1,000 or more, and the only costs involved the buying of old boats and bribery of coastguards, the profits are immense. Middle-class professionals from Syria face extortionate demands. It is a world of blackmail and thuggery against vulnerable, frantic people.

Once Libya had slid into civil war, its borders made porous by lawlessness, the Syrians found their route to Europe. What is striking is how appalling their lives had become before they were driven from their homes; how much they lost; how they were exploited, menaced, terrorised along the way; and how dismally and ungenerously they were treated on arrival. Some encountered kindness but this kind of treatment was the exception. How far they fled and the means of travel depended on how much money they could raise. All but a few arrived in Europe destitute, having lost houses, cars, jobs. As a mirror to modern life, all three books make for bleak reading.

It was only in October 2013, when 368 people drowned within sight of the Italian coast, that notice began to be taken of the mounting numbers of deaths at sea. The then president of the European Commission, José Manuel Barroso, vowed that such a tragedy would not be allowed to happen again. In its wake came talks, guidelines and promises. “As things stand,” Malta’s prime minister said, “we are building a cemetery within the Mediterranean Sea.” Pope Francis inveighed against the “globalisation of indifference”. That the deaths have not only continued but the volumes grown – 700 reported in 2013, 3,500 in 2014 and 3,771 in 2015, with the true figures certainly considerably higher – says much about the intractability of the problem, something all three writers try to address. As Bauer optimistically puts it, “We need to stop the wars in the Middle East from robbing Europe of its concept of humanity.”

All offer the same eminently sensible ideas: a need to improve lives in Syria’s neighbouring countries; the importance of identifying the dead; greater investment in Africa; more aid for Lebanon and Jordan, both home to vast refugee camps; more support for Italy and Greece, which bear the brunt of the arrivals. Yet these suggestions have been made many times, and there is little will to help realise them.

Rightly, Kingsley offers scathing criticism of the myth that European leaders like to milk – that the smugglers are the problem, and that once you do away with them, the frenzy of migration will cease. As the interceptions at sea, crackdowns on traffickers and strengthened monitoring of borders close one route, so another route opens. When the crossings to Lampedusa were reduced by more interceptions at sea, so those to Lesbos grew. When three fences with motion sensors tipped with razor wire – the trenches in between them filled with more razor wire – were put up at Ceutá, the Spanish territory on North Africa’s coast, a new route was found. In camps across Europe, in disused factories, tented cities and crumbling buildings, under dripping tarpaulins or clearly visible out in the open, the population of displaced and unwanted is growing steadily. If they were a nation, they would be the 24th-largest country in the world. In refugee circles, the vocabulary is all about growth: more child refugees, more migrants in detention, more people in more camps, more asylum applications.

As nationalist parties make electoral gains by delivering xenophobic speeches, and as political leaders squabble and temporise, with Merkel one of the few to consider the moral implications of the present crisis, so European countries prefer to erect more barriers, pay for more security measures and bicker over commitments, rather than attempt to reach humane and practical agreements. In the summer of 2015, the UN High Commission for Refugees was $2bn short of what it needed to keep its camps functioning in Egypt, Iraq, Jordan, Lebanon and Turkey. As Kingsley notes, more people landed in Greece in a single month in 2015 than the whole of the European Union has undertaken to share between its members over the next two years. The wealthy states, as Jeremy Harding wrote in the London Review of Books in 2000, “have learned to think of generosity as a vice”. At the peak of the landings on Lampedusa, Silvio Berlusconi spoke of the “grave danger” that refugees posed to Europe’s stability. The right-wing Lega Nord put it more succinctly: the party’s leader told migrants to “piss off”.

The future, in this context, does not look promising. Global warming threatens to send people displaced by flooding – the so-called environmental refugees – to join the flight to safety. Half the population of Bangladesh lives less than five metres above sea level. Given continuing conflict across the Middle East, the rise of murderous fundamentalism, the enduring powers of military dictatorships, and the extreme poverty and lawlessness in which so many parts of the world live, it is perfectly possible that up to three million more refugees could reach the shores and borders of Europe within the next three years. One of the things that makes the subject so confusing is the way it shifts: Egypt, once considered a safe haven in the Middle East, ceased to be one when Abdel Fattah el-Sisi and the military took power and turned against the Syrians who had found shelter there.

Whether those who flee are “good” refugees (in the sense of falling under the 1951 Refugee Convention, facing a justifiable “fear of being persecuted for reasons of race, religion, nationality” if they return home) or “bad” (so labelled because they are seeking work and a better life) has become largely meaningless in the world today. No one, ever, anywhere, wants to be a refugee, but for many there is no alternative. A Syrian man told Kingsley, in words that are repeated, in different forms, by many of the people interviewed for these three books, that whatever steps Europe takes to keep migrants out, even if they include bombing their boats, they will make no difference, because if he stayed home he was “dead already . . . a destroyed human being”.

There are precedents for the absorption of migrants, whose presence in Europe can in any case be hugely beneficial to ageing populations. At the end of the Second World War, not long before the Refugee Convention was drafted, about 12 to 14 million people made stateless by the fighting and the shifting borders were resettled throughout Europe. So were 1.3 million people after the war in Vietnam. In comparison to the numbers of refugees settling in countries bordering on those in conflict – there are 1.2 million Syrians living in Lebanon alone, and 85 per cent of the world’s refugees remain in their own regions – those who survive the journey to Europe are relatively few. It is the global South, not the prosperous North, that lies in the eye of the storm.

There is a crisis in migration but, as Kingsley insists, it is largely of our making, caused less by the flow of arrivals than the chaos of how we have received them. As right-wing parties make gains, governments respond with varying degrees of panic; scenes of rioting at borders, at train stations and at ports lead to more barbed wire, more attacks on refugees and more fodder for populist politicians. Yet sealing off Fortress Europe is not a viable proposition; fences and walls are nothing more than symbols, illusions for domestic audiences, promoting the fallacy that what is happening is a temporary phenomenon. And the more difficult it is made for refugees to reach Europe, the more refugees will die. Barriers, leaking boats, deserts and people traffickers are doing nothing to halt the flow. So, what will? There are 60 million people now on the move, half of them children. The choice that faces the West today seems to lie between an orderly system of mass migration – and chaos.

Caroline Moorehead’s book “Human Cargo: a Journey Among Refugees” has recently been reissued by Vintage

Crossing the Sea: With Syrians on the Exodus to Europe by Wolfgang Bauer is published by And Other Stories (144pp, £15)

Cast Away: Stories of Survival from Europe's Refugee Crisis is published by Charlotte McDonald-Gibson (272pp, £14.99)

The New Odyssey: the Story of Europe's Refugee Crisis by Patrick Kinglsey is published by Guardian Faber (336pp, £14.99)