Its monks left more than a century ago but the whitewashed stone walls of the monastery in Capel-y-ffin stand proud. Outside, a large, well-preserved statue of the Virgin Mary welcomes visitors to the venerable Victorian building, which has now been converted into self-catering apartments in great demand when the crowds flock to nearby Hay-on-Wye for its celebrated literary festival.

Since the monks’ departure, little has changed in this picturesque Welsh hamlet of a few houses, a chapel and a scattering of farms. Nestling in the foothills of the Black Mountains, it is a place of moss and bracken, stone walls and brooks, and has the lingering solemnity of an untended churchyard.

Part of the Brecon Beacons national park, Capel-y-ffin has long repelled development. Its one obvious nod to modernity is the local phone box. When installed half a century ago or so, it was painted a muted green, in keeping with park guidelines; it is now a vivid red.

But this sepulchral place, without a mobile phone connection and fast internet access, looks set to be dragged into the 21st century, to the dismay of many local people and a host of literary figures who have sought refuge in Capel-y-ffin.

The Home Office and telecoms network EE want to build a 17.5-metre mobile phone mast with three antennas, two microwave dishes and a pole-mounted satellite dish just above the hamlet, next to a track that provides a gateway to the Black Mountains, with their extraordinary views across Wales and England.

According to the planning application, the mast will have a “medium” impact on its environment, but the Brecon Beacons Park Society disagrees. In its letter to Lisa Williams, the authority’s planning officer, the society says: “Many people would find the contrast of the lines of the brutalist tower, with its attachments, cabinet, concrete base and mesh fenced compound, an extremely stark contrast with the natural flow of the landscape.”

The novelist Ian McEwan is among the 60 people who have objected to the proposal, which goes before the Brecon Beacons national park planning authority next month. “I have hiked in this region for almost half a century,” he wrote. “Its unchanged nature is just one of its many attractions.”

“There’s something special about this place,” says Erica Mary Griffiths, who was born in the monastery in the 1930s and runs a pony trekking centre in the area. “People who are a bit lost, not quite sure where to go, they tend to find their way up this valley, to this place of tranquillity.”

Griffiths remembers how the telephone wires had to go underground rather than on poles when the hamlet got landlines, such were the stringent protections laid down by the park authority. She remembers, too, how the monastery took in refugees during the second world war. After the horrors of what they had fled, its solitude must have seemed a balm.

It wasn’t just refugees who found solace in its isolation. Painters, poets and sculptors were inspired by time spent in the valley. And a veritable Who’s Who of 20th-century fiction has been touched by the place after spending time at Carney, a small stone cottage in danger of being overrun by rosehips and brambles, owned by the publisher Tom Maschler.

Doris Lessing, Kurt Vonnegut, John Fowles, McEwan, Arnold Wesker and Edna O’Brien all stayed at the cottage, which is only 150 metres from the site of the proposed mast. It was in Carney that Bruce Chatwin wrote his novel On the Black Hill.

The beat poet Allen Ginsberg was inspired to write Wales Visitation after staying at Carney. Photographs of Ginsberg, taken by Maschler, show him rolling around on the hills “smelling the brown vagina-moist ground”, as he enthused in his poem, lauding their remoteness from the connected world: “160 miles from London’s symmetrical thorned tower / & network of TV pictures”.

Facebook Twitter Pinterest In a photograph taken by Tom Maschler, Allen Ginsberg rolls around the hills. Photograph: Macmillan Publishers

Alice Maschler, Tom’s daughter, has been instrumental in mobilising opposition. “When I heard about this proposal, I cried, and felt like I had lost someone I relied on, no matter what,” she said. “Carney and its surrounds is a sanctuary, where people come and touch a magical presence.”

While many believe the mast will be an ugly intrusion, others fear the impact of its technology. In her letter of objection, Griffiths writes: “There are so many unanswered questions about these new mobile wireless phones, iPads and computers being introduced everywhere today.”

Her views accord with those of a growing army of people who, like the refugees who once came to Capel-y-ffin, are fleeing something they fear. In online forums, “electromagnetic refugees” discuss the best places in the world to live away from microwave radiation, which they believe is making people ill. The movement is particularly vocal in Australia and pockets of the US.

Others, however, see the mast– which initially will deliver a new emergency services network – very differently. One farmer who declined to give his name said: “I had a heart attack two years ago and luckily was near the landline. But if I’d been a couple of miles away, I’d have needed a mobile phone then. The people complaining only come here twice a year.”

Amid the standoff, some locals seek a compromise – moving the mast to a less prominent position or substituting it for a different type that would be lower and less obtrusive. The planning authority’s own guidelines state: “Where developments would be visually prominent, evidence must be provided to show that alternative locations have been investigated and are impractical.” Some locals question whether such alternatives have been discussed.

As they made their way up the valley last Thursday, walkers Rod and Angie Edwards stopped to drink in the view. Rod used to design windfarms for a living. “I’d go ballistic if someone wanted to put a windfarm here but a 17.5-metre mast isn’t so high,” he said. “We live on the edge of Snowdonia and our children couldn’t get a mobile signal when they were teenagers. We’ve got superfast broadband now and it makes all the difference.”

But Griffiths, born and bred in the valley, believes the mast is too high a price to pay. “People find a lot of peace and quiet here,” she said. “There are not things going on like there are in other places.”