If you stop and think about it, radio is a bit magic. You buy a box with a metal stick coming out of it and plug it into the mains and it connects you to a world of stories and music and laughter that arrives invisibly through the air. Twiddle the dial on the front and you can find almost any­thing and everything – the noise of other countries, other times, other perspectives. During a debate entitled “God v Science”, Richard Dawkins said, “To a medieval peasant, a radio would have seemed like a miracle.” In which case, I’m a peasant and proud of it.

In the past ten years, the way we listen to the radio has changed. Your MP3 player can probably tune in, as can your laptop and your smartphone. Podcasts – both downloadable versions of broadcast radio programmes and shows original to the format – have risen vastly in popularity. In July this year, yet another technological barrier was breached when the BBC updated its iPlayer radio app to include the facility to download any programme for 30 days after transmission. Not only do you have a device in your pocket that can pick up those invisible waves of sound, but now you don’t even need a signal to hear them.

Somehow, radio has bucked the overwhelming trend of the digital age. With books, television and journalism, the internet has turned us into serial dilettantes, flicking between different sources, abandoning the long-established channels (and funding models) in favour of the wider selection and greater convenience to be had online. Following this logic, radio, too, should be in terminal decline, losing out to the innovation and agility of podcasts and streaming-only services. I shouldn’t be considering investing in a fourth radio, so I can have one in the bathroom as well as in all the other rooms in my flat. And yet I am.

Nothing exemplifies this against-the-odds story better than radio comedy. In Britain, this is almost exclusively the property of the BBC, with the cost-benefit ratios making it uneconomical for commercial stations to replace DJs and music with spoken-word shows that may or may not make people laugh. Historically, BBC Radio 4 has always been, as it proudly describes itself, “the home of radio comedy” – and so it still is, for better or for worse.

Like many aspects of the BBC, its radio comedy has been slow to evolve. The prime slot in the weekly schedule, Friday at 6.30pm, is occupied in rotation by The News Quiz, which was first broadcast in 1977, and The Now Show, a relative stripling in BBC terms, having been on air for a mere 17 years. I’m Sorry I Haven’t a Clue has been going since 1972 and Just a Minute since 1967. Nostalgia and habit surely keep some people tuning in but they don’t explain the global popularity that the station’s output enjoys.

Given how firmly set in their ways many of these shows are, the departure of Sandi Toksvig as the host of The News Quiz this summer was a pivotal moment. In the chair since 2006, she was initially so unpopular with some listeners – aghast at the idea of a woman in charge – that the then producer, Ed Morrish, had to go on Radio 4’s Feedback programme to rebut calls for her

predecessor Simon Hoggart to be reinstated. At the recording of her final show, over 200 episodes and a 116 million downloads later, Toksvig was watched from the front row by the News Quiz creator John Lloyd as she wiped away tears while reading the credits for the last time. She led the charge when it came to making the programme’s panellists more diverse, while also crafting a warmth and tone for it that clearly resonated with people around the world.

This is not to suggest that radio comedy runs solely on long-standing panel shows. One of the biggest hits for radio (and the BBC as a whole) in the past few years was Cabin Pressure, a sitcom by John Finnemore set on a small airline. Some of its success was down to the cast: Benedict Cumberbatch, Roger Allam, Stephanie Cole and Anthony Head returned for four series and several specials. It was also tightly written, conjuring laughs from wordplay and imagined physical comedy. This made it timeless: jokes about otters, “Surprising Rice” and obnoxious passengers will never get old.

There were more than 20,000 requests for the free tickets to the recording of the final episode of Cabin Pressure in 2014. I spoke to Finnemore just afterwards and he admitted that he had been concerned at first that a crowd of Benedict Cumberbatch fans wouldn’t be much of a studio audience for a radio comedy about aeroplanes and word games. “I was worried that they were just there for Benedict and they didn’t know or care about the show. But I was completely underestimating them . . . They’re clever, they pay attention, they get the jokes and they’re just as interested in Stephanie’s lines as Ben’s.”

They may have come for the chance to see the star of Sherlock up close but once this new audience had discovered radio comedy, there was no stopping it. A significant portion of them have become fans of Finnemore’s sketch show Souvenir Programme and other Radio 4 shows. There is a thriving community online writing Cabin Pressure fan fiction and drawing scenes inspired by some of the sketches. Morrish, who produced Souvenir Programme, has joked to me that he must be the only BBC radio producer who has appeared in personal fan art on Tumblr. He is probably right.

Radio comedy often serves as an incubator for what become cultural juggernauts in other forms. Miranda, That Mitchell and Webb Look, The Hitchhiker’s Guide to the Galaxy, Little Britain, Count Arthur Strong, The League of Gentlemen and many more began life as radio comedies. Without Chris Morris and Armando Iannucci’s On the Hour, there would have been no The Day Today on television and no Alan Partridge in any form. Looking down that list, you could get the sense that radio has been treated as television’s testing ground, where new ideas (and people) can be tried out without the vast expenditure of shooting whole series. Caroline Raphael, whose three decades at the BBC included time as Radio 4’s comedy commissioning editor, says that she always resisted this idea that radio exists to serve television. “If they want to take it on and make something after­wards, that’s fine. But I was never going to do something on the radio for that reason,” she explains. “It always had to work for the radio, in that format.”

What does work on the radio? The answer is subjective. The latest BBC Trust review of speech radio, issued in August, reports that while the vast majority of listeners think the comedy is good, “A minority of listeners strongly dislike the comedy output, either because the satirical and politically focused comedy feels divisive to them or because it feels similar or stale.”

The alchemy of what “works” is hard to come by – as is the kind of dedication that drove Iannucci and Morris to spend hours borrowing microphones and studios from other divisions of the BBC so that On the Hour would sound like a news programme, not a satire of a news programme. And, according to Gillian Reynolds, a radio critic for the Daily Telegraph and commentator on the industry since 1967, budget pressures at the BBC will only make it harder to find.

“They concentrate now on panel shows, because you can sell the format on and it’s cheaper,” she says. “Paying a couple of gag writers is not as much of an investment as paying someone to develop a scripted show with characters, with actors, possibly music.” Yet there are a few exceptions still coming through. That Miles Jupp’s In and Out of the Kitchen is still on air and that Susan Calman’s newly premiered sitcom, Sisters, came into being show what can be done when the funding is there.

The challenge is to keep the money flowing for new, strange, innovative ideas when commissioning budgets are expected to remain flat or even be reduced in the drive to find savings. It is generally agreed that radio comedy (and radio in general) doesn’t blow its own trumpet enough – it’s the big news scoops and TV exports that the director general cites as the pinnacle of Brand BBC’s achievement. “Nobody ever pays any attention to radio,” says Reynolds. “It’s actually the foundation of so much, and not dead: absolutely not dead. It fits the new age of communication like a hand into a glove.”

Perhaps this is the crux of the problem: radio has adapted too well to the internet age. It has made the transition that every other kind of media seems to be struggling and groaning over. In the BBC’s new TV advert for the iPlayer app, various radio personalities, such as the Today programme’s Mishal Husain and Radio 1’s Nick Grimshaw, are integrated seamlessly into their listeners’ lives. The news is told over a cuppa in the caff; the latest tracks are shared on the sofa at the barber’s. Sadly, there is no comedy element – I would have loved to have seen the entire cast of I’m Sorry I Haven’t a Clue crowded into a bus shelter.

The same gloriously lo-fi technology that would have had Richard Dawkins’s peasant gawking in astonishment has expanded out of our homes but stayed in our ears. We might be a long way from the time when an episode of In Town Tonight had 20 million people crouched around their radios on a Saturday night – but we’re still listening and we’re still laughing.

Now listen to Caroline discussing radio comedy on the New Statesman’s pop culture podcast, SRSLY: