I live on the edge of England. You can’t go any further south without falling into the sea, which is a long way down from the top of the high cliffs near my home. The walk over the backs of Beachy Head and the Seven Sisters is often named as one of the most beautiful in the country, with rolling downland that ends suddenly in bright, white chalk falls and a seascape horizon so wide you can see the Earth curve.

For the last couple of years I have been privileged to be writer in residence at the Belle Tout lighthouse, which sits alone on one of these hills, almost on the edge of a 120m drop. This Georgian tower no longer shines a light, but it has been restored and transformed into a lovely place for bed and breakfast.

I live just eight minutes’ drive away in Eastbourne, but as often as possible, whatever the season, I have been going to the lantern room at the top of the lighthouse to write. From up there you can watch the sun rise in the morning and set in the evening. Lobster boats collect baskets close to shore, trawlers pass along a little further out and beyond them a ferry sets off from Newhaven for France twice a day. Way out in the distance, the giant freighters barely seem to move.

This is a borderland, psychologically as well as geographically, where nature and the sublime seem to merge on bright spring days. But the weather changes fast as a storm sweeps in, a gale picks up and waves foam above the hidden dangers of the rocks and currents that have sunk so many ships over the centuries.

I was up there in the tower watching the waves in early 2018 when I became aware of the new reality beginning to unfold out in the Channel, with increasing numbers of refugee men, women and even children risking their lives to cross in tiny boats. I heard on the news that nine people had been rescued from a sinking dinghy far out to sea and brought to safety near Eastbourne. As I thought about them, I happened to be watching a rigid inflatable RNLI dinghy fighting against the waves to get close to the rocks down below.

Most of the visitors here come because of its beauty, but they can lose their footing. Some people are drawn to the area because they are desperate and seek an ending. There are volunteers who patrol in all weathers, night and day, saving hundreds of lives, but every year around 30 people go over. We are used to seeing the helicopters hover and the lifeboats go out to search for the bodies. But as I watched this one, the thought was shocking: if the rescuers were struggling in a state-of-the-art dinghy with the latest equipment and training, what hell would it be for cold, wet, frightened strangers to the sea in a far smaller, far flimsier boat?

In the year that followed, more boats began to come. The Home Office said 297 people made it across the Channel that way in 2018. Last year 1,892 made the trip, according to the BBC, and earlier this month 90 were rescued in a single day. They used to leave from Calais, but now they use more remote beaches and make longer, even more dangerous crossings. Most landings are still in Kent but some get as far as Sussex.

I understand why they think it possible, because I have stood on the beach at Calais on a sunny day and seen Dover’s white cliffs looking startlingly close. But I happened to be there with a French fisherman, while making a Radio 4 documentary, and was told a gale was starting out in the middle of La Manche, invisible to us. The sea looked calm enough to cross but when I asked if he would go that day he laughed and said: “Non! That would be suicide.”

There’s something hard-wired into the psyche locally about being constantly on guard against ‘the invader’

At the end of 2018, the then home secretary Sajid Javid sent more Border Force boats to the Channel and declared a “major incident”. This was controversial, not only because the numbers involved are tiny compared with the millions seeking sanctuary in countries such as Turkey, but also because it came amid a fierce debate about Brexit and played into the fears of those who felt these shores were under threat from foreigners. Some even said the boats should be punctured and sunk to send a strong message of deterrent.

That appalled Matt Coker, a fisherman from Dover who had been among the first to encounter a migrant boat out at sea and rescue those on board. “The only people who don’t feel sorry for them are people on the land, who don’t realise what they’re actually going through when they’re out there,” he told me. “They come up with silly things like: ‘You should burst the dinghy!’ They should come out there with us. Whatever your politics, if someone’s in trouble in front of you, most human beings would do whatever they could to help.”

His words highlight the paradox along these shores. Here, some places are closer to France than they are to Westminster, we can hear French voices on our radios and sometimes our phone signals even flip over to theirs. Yet town after town along the coast voted to leave the European Union. Much was said about shutting the door on foreigners, including our closest neighbours. In Folkestone and Dover the vote for leave was 62%, slightly higher than the Kent total of 59%. Sussex was closer, nearly 50-50, because of remain places such as Brighton and Lewes, but in my town of Eastbourne the vote was 57% leave.

Last summer, a self-styled vigilante group promised to patrol the beaches between Dungeness and Deal to intercept incoming migrants and hold them there until the authorities arrived, not seeming to realise that many want to contact the police as soon as possible, to start claiming asylum. Meanwhile, the Sun ran a photograph that, it claimed, captured The Moment Migrants Storm Kent Beaches, as if they were armed and not just scrambling ashore, grateful to still be alive.

Border Force rescuing migrants in Dover. Photograph: Rex/Shutterstock

Wartime defences are still visible along this stretch of coast. They go back much further than the second world war, to Martello towers from the Napoleonic era and even Pevensey Castle, where William the Conqueror set up his first post in the ruins of a Roman camp. Bridget Chapman, who works with refugees in Folkestone, says: “There’s something hard-wired into the psyche locally about being constantly on guard against ‘the invader’.”

She’s right, and yet there are many who don’t feel this way, judging by the extraordinary generosity people have shown in raising money, finding food and donating clothes. She has invited me to take the train to Folkestone, where her group Kent Refugee Action Network has a portable building under the railway viaduct. A small group of late-teenagers is learning English with volunteers and being taught how to negotiate life in this new land. Adults and families are dispersed all over the country while they wait for their asylum claims to be heard, but unaccompanied youngsters who are under 18 on the day they arrive are usually kept in Kent, with foster families if there are places available, or in rented flats, in groups of five or six. Here then is a rare chance to meet some of the people who have crossed over. Who are they? And why have they taken such enormous risks to get here?

Akoy is dressed in skinny jeans and a roll-neck sweater, with a shaved neck but a big fat quiff. He’s only 16, but his eyes say otherwise. Last summer, he made the long, perilous journey from his home town in Iran to the shores of northern France. There, he paid a smuggler £3,000 for a place on a small boat leaving for England, with money wired by his big brother, but he was deceived. “The man told me lots of lies. He said: ‘I send you by big boat, like the ferries they have in Dover. You have food, drink, everything you need on board.’”

Instead, Akoy found himself standing on a wet, windy beach in the dark with a group of others, looking in horror at a tiny craft. “We were all really scared when we saw this boat. Three metres.” He identifies it on Google as the kind of inflatable meant for six or eight people. “There were 22 of us. The boat was in a big cardboard box. We had to unpack it and pump it up by hand. It took three hours. The man brought a small motor for the back and gave it diesel.” A boy was chosen as the driver because he had worked as a fisherman back home in Afghanistan. “The man pointed to a red light on the other side and said, ‘Don’t worry. Just aim for that.’”

Akoy panicked. “I said, ‘Why did you lie to me? I’m not going.’ The man said, ‘You must go. I won’t give your money back.’” It was a terrible moment. That was all the money his family had. So he got on the boat, reluctantly, but it turned over in the surf and threw them out, three times. They were soaked. It was 2am. Out in the Channel, they realised they were in real trouble. “We were all saying the prayer we say as Muslims when we are going to die. The Afghan people texted their families to say goodbye. I would have texted my brother, but I didn’t have my phone.”

There’s a narrative that these people are all Isis fighters. Most of them have come because they don’t want to fight

The boy driving had thrown some bags into the water during the journey, presumably to keep the boat afloat. There was huge relief when a large Border Force boat spotted them at dawn and sent a rescue launch. The people on board were all young men like Akoy, although the others were from Afghanistan. As they were given lifejackets and taken off the dinghy, Akoy braced himself for trouble. “In France, the police hit you. They come to the camp, put spray in your eyes and beat you up. The English police were not like that. They were so good. I was so cold. They helped me with a blanket, clothes and food. I didn’t think it would be like this.”

The young men were taken to a reception centre in Dover harbour, where specialist immigration officers interviewed Akoy and asked why he was there. His answer is the same now as it was then: “I come from a dangerous place. I am looking for a quiet place.”

Akoy comes from Sardasht, a majority Kurdish town in Iran near the border with Iraq, where Saddam Hussein once carried out chemical attacks. Lately, there have been violent clashes between the Kurds and Iranian security forces. Home Office figures show that two-thirds of Iranians who apply here are given asylum.

The story Akoy tells is like an old folk tale. His mother died when he was five. His stepmother rejected him. His father got sick and he had to go to work at 13. He dreamed of escape and saved to do so, supported by his brother. Akoy made it to Turkey then tried to cross to Italy by sea, but the boat was stopped and sent back twice. Unwilling to give up, he paid to be locked with three other boys in the back of a sealed container lorry going to France, only to realise at the last moment how dangerous it was. “I was scared to get in. I thought we were going to suffocate and die. The man hit me three times to make me get on.”

Akoy’s eyes become red and glisten as he remembers the fear. He was 15 then. It was only last summer. “We were inside the lorry for four-and-a-half days. The driver, who was Turkish, would open the door and give bread and some drink then close it quickly.” (Kent Refugee Action Network says Akoy’s story rings true. We have changed his name.)

‘I came here to study, to get my dreams’: Zahra, 18, from Iran, landed here, at Sunny Sands, Folkestone. Photograph: Jonathan Browning/The Guardian

“We had no toilet for four days,” Akoy continues. “Sometimes there was no oxygen. It was horrible. We were banging on the walls and the door, calling out to the driver, ‘Please help us.’ Three or four times, we called for him. He was not coming.”

They were finally let out in Lille, France. Confused and distressed, Akoy had no idea what to do. “I saw Kurdish people at the train station and one boy said he was going to Dunkirk, where there was a place for Kurds. I needed to see people who spoke my language.” Smugglers in Dunkirk openly offered places on boats. His brother sent the money through Western Union.

On arrival in the UK, Akoy was lucky enough to be found a place with a foster family until he turns 18. “I like my foster mother very much. She is like my mum. I cook for her. I am learning English food. It’s hard. I miss my family.”

Akoy is still waiting for his asylum claim to be processed by the Home Office, having been told there is a backlog. If he is granted refugee status, he will have five years’ leave to remain in Britain, with the ability to work and apply for travel documents. He is determined to make a success of life here, if allowed. He is busy studying. “I need to finish school and college then university. I would be a good chef. I can’t go back, it is too dangerous.” He sighs. “I am glad to talk about this. I want to get all the bad memories out of my head.”

Listening to him, it strikes me that if people like Akoy were perceived as being “us”, we would tell their lives as adventure stories. Mostly, of course, they are not.

“There’s a narrative that says these people are all Isis fighters,” Bridget Chapman says. “Look around the classroom. Most of them have come precisely because they don’t want to fight. We have people from Afghanistan who didn’t want to be recruited by the Taliban and people from Eritrea escaping military service for life, which is slavery. They are peace-loving people with so much to offer us.”

Bridget Chapman, of the Kent Refugee Action Network. Photograph: Jonathan Browning/The Guardian

These are not sentiments that were heard very often in Eastbourne before and after the Brexit referendum. I was invited on to the panel at a Brexit debate in May 2016, in front of 1,600 people at the town’s Congress theatre. I was there as a neutral, because like so many people I had not yet got my head around the issues. I may have been undecided about how to vote, but I was alarmed and bemused by the full-throated shouts, groans and moans of those in the audience who said they felt under attack by foreigners.

There were boos from the audience when I pointed out that the oldest known remains of an inhabitant of these parts belong to a woman discovered at Beachy Head who appears to be from sub-Saharan Africa. The other speakers on the panel were from out of town and didn’t know this, but seeing a reconstruction of Beachy Head Lady’s face in a local museum has challenged many people’s perceptions of who we are. In a parallel way, Chapman says, opinions change when people meet individuals like Akoy. “It’s easy to dislike this amorphous, faceless group of people: ‘Them.’ It’s very different when you meet actual people, face-to-face.”

Coker, the Dover fisherman, would agree, having been surprised to find a dinghy full of migrants in trouble in September 2018. Now he sees the boats nearly every day, but then such encounters were a rarity. He and his father Mick were taking a dozen sporting anglers out to sea. “I looked through my binoculars and I could see something. It was 10 in the morning on a flat, calm, sunny day. We were four miles off Dover. As we got closer, I could see they were waving a white T-shirt tied on an oar. It was obvious they were in distress. I thought holidaymakers had drifted out too far.” As they got closer, he could see they were desperate. “I got alongside, and one of them leapt for the boat. He was hanging on my railing as we were still moving. There was panic in the dinghy. They’d obviously been out there a while. They didn’t speak English so I pointed towards France and a woman nodded. I went straight into the wheelhouse and called the coastguard. They sent the Border Force out to meet me.”

The people on the dinghy were in a bad way. “The lady was being quite ill. There was a bloke bringing up blood, that might have been through dehydration. My anglers actually gave them some of their clothes, because the dinghy was half full of water.”

I wanted Coker to show me what it was like out there, so we set out at dusk with the wind picking up and the temperature dropping. Soon I was freezing cold and wet, despite being dressed appropriately. Six miles from shore, on the edge of the shipping lanes, he turned off the engines and let us drift. I knew I was safe on this modern fishing boat, but the inky blackness of the sea suddenly felt overwhelming and I had just a hint of how terrifying it might be to be stuck out here at night in an overloaded dinghy. Being that low in the water without proper lights or signals would put you in serious danger of getting run down by one of the many vessels using the lanes. The freighters looked to be moving very fast now and they were vast, like skyscrapers lying on their sides, heading for us.

There has been at least one death, although it is impossible to know if others have slipped beneath the waters

“Hang on, we’ve got company,” said Coker as we drifted, having spotted the symbol for a Border Force cutter on his radio. The grey that camouflages these large patrol boats during the day must act like stealth paint at night, because all I could see was a black shape and a couple of lights. “They must think we’re smugglers,” he said, firing up the engines again. As we picked up speed, the Border Force must have been satisfied, because they left us alone. Five red lights in a vertical line dominated the darkness and I took them to be the lights Akoy’s driver had been told to aim for. They were attached to the very high UHF mast at Church Hougham, near Dover. “You can see that all the way over on the other side, but it is so dangerous what they’re doing,” said Coker. “I’m really surprised more people haven’t died.”

There has been at least one death, although it is impossible to know if others have slipped beneath the waters. Mitra Mehrad was a 31-year-old PhD student from Iran who jumped over the side of a sinking dinghy in August last year to try to catch a rescue rope and save a baby. Nine days later, her body was washed up in a wind farm off the Dutch port city of IJmuiden, 100 miles away.

A man from Afghanistan and another from the Netherlands were convicted of manslaughter in Boulogne last December. The court heard Mehrad had paid them £2,900 for a place in a Zodiac rib designed to hold six people; there were 17 on board when it was picked up off the coast of Ramsgate. The men had bought boats from Leboncoin, a French equivalent of eBay, and made up to £50,000 for each crossing from the Calais coast. One wept in court and said: “I’m sorry. I didn’t know anyone was going to die. I just thought she would be able to get to England.”

Thankfully for Zahra, an 18-year-old from Iran, the sea was flat when she crossed on Christmas Day 2018. Zahra, who also attends the sessions under the viaduct in Folkestone, set off for England after a year in a refugee camp with her mother. “The camp is very dangerous for young ladies. My mum decided to send me to another country.”

They believed the UK was the only country that would not send her back to the camp straight away. Zahra tried to jump lorries at Calais several times but was stopped. “There were lots of agents among the refugees, telling us we could go by boat.”

Her boat was unpacked and inflated on the beach like Akoy’s, and this time she made it across. They landed on Sunny Sands in Folkestone, a wide, flat beach with a curving promenade that is built on distinctive brick arches. It’s busy in summer but was deserted in the early hours of the morning. They were spotted, though, by someone living in a house on the hill above.

“We were very wet and very cold. The police were very kind. They took us to Dover.” After being given hot drinks, food and dry clothes, Zahra was eventually placed with an emergency foster family. She now lives independently with others. “I came here to study, to get my dreams. When I was living in Iran, I wasn’t allowed to go to school, to study or to have my dream job. We do not come here to sleep. I want to be a pilot.”

That’s ambitious but may not be impossible. Zahra has had three flying lessons already, sponsored by a generous company director who works in Greece and has been moved by the plight of refugees there. “Most of the boys say women can’t be pilots. I say, ‘I can. You watch.’”

There is something Chapman wants to show me, so we walk to the Folkestone Museum. A large, colourful painting called The Landing Of The Belgian Refugees illustrates people arriving in little boats in 1914, having fled the invading German army. It was painted that same year, by one of those who came.

“There are obvious parallels with what is happening today,” she says. “On the busiest day in 1914, 16,000 people arrived in Folkestone. Last year 1,800 people crossed the Channel and we called it a crisis. The Belgians were bedraggled and cold, hypothermic, arriving in boats as they are now. People gathered with food and clothes. They had empathy.”

They still do, but this was a big display of welcome led by the mayor of Folkestone in all his pomp. We don’t do it like that any more, down on the edge of England. Brexit has cracked us open, allowed our strongest feelings to come out and left us polarised. There are still people who offer help and welcome, but they are not running the country. “I wonder what’s changed,” says Chapman quietly. “When did we become so hard?”

• Cole Moreton’s novel The Light Keeper is published by Marylebone House; he is the co-host of the podcast Edge Of England.

If you would like your comment on this piece to be considered for Weekend magazine’s letters page, please email weekend@theguardian.com, including your name and address (not for publication).