The route from Footballers’ Wives to Radio 4 is not an obvious one – but now Ambridge fans are distraught to see O’Connor go to EastEnders. As the everyday tale of farming folk reels from domestic violence, the man behind it says it’s time to take a gentler approach

The Archers editor has worn a tweed suit in every media appearance I have seen. I assumed this was deliberate, calculated to reassure Radio 4 listeners who took fright when the former Footballers’ Wives producer was put in charge of their beloved show three years ago. Now in its 66th year, The Archers is the longest running soap opera in the world – but many among its five million fans who favour the term “serial drama” took a dim view of Sean O’Connor’s appointment. Accusations of sensationalism and “sexing up” beleaguered his first year; one critic memorably accused him of turning The Archers into “EastEnders in a field”.

Instead, he has transformed the show into a national conversation. Longstanding fans previously embarrassed to own up to their addiction to “the everyday tale of farming folk” have been engaged in fevered debates with the newly converted, all mesmerised by a storyline based around the new criminal offence of coercive control. They have listened in horror as a hitherto tepid character, Helen Archer, fell for the deceptive charms of a suave hunting’n’shooting type, Rob Titchener, who proceeded to undermine, isolate, bully and control her. The story, plotted painstakingly over 900 episodes, reached its climax last month when Helen finally snapped and stabbed Rob with a kitchen knife, delivering that rare soap-opera alchemy when fiction transitions into real life. The stabbing made the front page of almost every national newspaper, while more than £100,000 has been raised for the domestic violence charity Refuge, and, in the pages of the Times, barristers have been pitching the defence case they would make for Helen. The recent announcement of O’Connor’s departure from The Archers this summer for EastEnders is now being lamented by many of the very listeners who had previously bombarded online forums demanding he be sacked.

'The show is successful because it uses the prism of farming to talk about Britain in all of its difficulty and glory.'

O’Connor could be forgiven for saying, “Ha! I told you so,” but if that’s what he’s thinking, he isn’t letting on. His manner is an appealing hybrid of scholarly and luvvie, and he talks about The Archers in the literary terms of the English undergraduate he once was. On his original application form for the editor’s job, he tells me, he simply “ignored the questions” and wrote what he felt the show was really about.

“I said I thought it was about who we were, who we are, and where we’re going. The show is so successful because it uses the prism of farming to talk about Britain in all of its difficulty and glory. In Genesis, Adam is told he’s going to have to till the soil for the rest of his life, so it’s a very ancient culture. But farming is also uniquely modern, because it’s about European directives, or it’s about drones. Ultimately, farmers live with that strange knowledge that fate invades their lives in the way that it does in Thomas Hardy. There’s something more important to you, and that’s the farm, and no other soap opera has the same structure. It’s unique. And so every day, incrementally, you are adding another layer to a piece of ongoing social history, so it’s a work of art in the making. I was trying to get across that it was more than just a programme. As real fans know, it’s not just a drama.”

Had I not been listening to The Archers for 40 years, I might suggest this sounded a little pretentious. But if I did, O’Connor says he wouldn’t care. Without self-consciousness, he references A Doll’s House by Ibsen, Patrick Hamilton’s Gas Light, The Secret Agent by Joseph Conrad, and, above all, Hardy’s Tess of the D’Urbervilles as inspirations for the Rob and Helen storyline. One of the show’s writers researched the new offence of coercive control, and together the team crafted a relationship that appeared to those around Helen to be a loving one, but which was, in private, a textbook case of the kind of abuse made illegal last year. Home Office guidelines offer a long list of examples of coercive control – humiliation, rape, denigration, threats of violence – all of which would be instantly recognisable to Archers’ listeners.

Realism, however, has its limits. For months, listeners had been waiting in dread for Rob’s violence to escalate, so O’Connor’s decision to have Helen wield the knife came as a surprise. Some have complained that the denouement was absurdly unrealistic; abuse victims very seldom commit violence against the perpetrator. “Yes,” O’Connor concedes. “But if Helen was killed by Rob then there would be no story, and all we would be doing then is following a realistic plot. Ultimately, the point of the story is to really investigate her as a character, as a central Archers family member.”

Another criticism came from a perhaps unexpected quarter. Erin Pizzey, the founder of Britain’s first women’s shelter, which evolved into the charity Refuge, wrote in the Daily Mail: “Helen fans have bombarded Twitter, saying Rob deserved his fate. But does anyone actually deserve to be stabbed, however badly they’ve behaved?” I ask O’Connor if he thinks Rob deserved it, and he looks uncharacteristically uneasy.

Facebook Twitter Pinterest Helen Titchener (Louiza Patikas) and Rob Titchener (Timothy Watson). Photograph: Pete Dadds/BBC/PA

“Well, it’s a very complicated question, because the issue that we’re dealing with at the moment in the show is, who is the victim? The five million people who listen to the show know that Helen is the victim.” Does he agree? He becomes cagily non-committal. “Erm, well ... She shouldn’t have stabbed him, but it’s interesting, the audience are very partial in their responses to the show. They jump up and down cheering when Rob’s being stabbed, but, you know, she did nearly murder him and that’s not right. We don’t solve the issue of domestic violence by killing the perpetrator. But I would say, there was a little boy involved [Helen’s five-year-old son witnessed the attack], and she believed that he was maybe in danger, and that’s also self-defence if it’s your child. So it is ... we believe at the moment it’s going to be an act of self-defence.” Is he saying that the boy was in danger? By now O’Connor looks contorted with spoiler anxiety. “Er ... I believe that she believed that at the time – ‘he said’,” he adds, smiling, “‘thinking about what the lawyers keep telling me’. Which is fascinating. Because ultimately, you’re in a situation where you’ve got two adults who are saying this happened or that happened, and the only witness is a five-year-old.”

What complicates Helen’s defence, O’Connor explains, is the parallel real-world uncertainty about the coercive-control legislation. He has employed a barrister who is currently drafting an entire prosecution and defence case, to assist the scriptwriters. O’Connor still hasn’t made up his mind what the jury’s verdict will be. “We talked to lots of lawyers who said: ‘Well, this might happen, that might happen, but we can’t be clear with you because the law is changing.’ Most of the lawyers said it would be so much easier if you just killed Rob. But I think that’s quite a soapy way of the story ending, and what we have now is ... well, we think in dynastic terms in The Archers. And fundamentally, we’ve got two children [Rob had been given parental responsibility for Helen’s son during their marriage, and she is pregnant with his child] and as long as those kids are alive, Rob and Helen will be tied together. And I think it’s a much more realistic dramatisation of what happens in real lives; these couples have to find a way of living around each other.” And presumably no one wants to kill off radio gold? O’Connor grins conspiratorially. “Of course you don’t. I mean, Rob is a great character.”

He’s also the only character O’Connor does not like. Among his favourites are Brian and Jennifer Aldridge, and he reveals that he’s already cueing up a “titanic struggle” when Brian’s illegitimate 13-year-old son, Ruairi, comes of age. “Brian always wanted a son, he doesn’t want his wife’s lovechild, who happens to be gay. Brian’s Lear-like positioning of Ruairi is going to be one of the great shows of the future.” He has more immediate plans for another of his favourites, Shula Hebden. My eyebrows shoot up at his affection for such a pious bore, but he chuckles knowingly. “Shula is the audience. She’s exactly the right age. And one of my jobs, if I can try to do it, is to recalibrate Shula.” He’ll have his work cut out livening her up, I suggest. “Well, the doctor is there for a reason,” he offers archly. But, of course, he won’t reveal more. “Real fans” won’t want to know what’s coming, so we will have to wait to find out what he has planned for Shula and the recently returned Dr Locke.

A team of 11 writers, which O’Connor calls “tiny”, work on the show. Each takes turns to write a week’s episodes, which sounds rather like a game of consequences. “It is unlike any other show I’ve ever worked on. It’s very collegiate. There are no rows.” Really? “No, honestly. It’s extraordinarily respectful. In other meetings, in other shows, people are throwing things across the room, shouting and walking out. It doesn’t happen on The Archers.” Even over the Rob and Helen story, “the biggest and most intense discussion we had was about whether Helen should lose her maiden name or not”. (O’Connor takes the view that any man who insists on his bride taking his name is one she “probably should not marry”.)

O’Connor, 48, is married to a man he met 16 years ago, who works in the City; the couple live in south London. O’Connor has been listening to the Archers since his teens, but admits he has never lived in the countryside – although he quite fancies the idea of moving to a village in East Sussex. After growing up on the Wirral, he studied classics and English in London and worked in the theatre before joining The Archers as maternity cover in the 1990s, going on to produce EastEnders, Hollyoaks and Footballers’ Wives.

“People think, ‘Oh, he’s just a slum producer that does slummy shows like Footballers’ Wives.’ But actually, it’s just about storytelling. It is exactly the same. When I was on Footballers’ Wives, I would talk to the writers about it being like the later Roman empire. So the audiences are quite different, but fundamentally, my approach to all the things I’ve done has been the same, and it is to make a show that I would watch or listen to.”

That said, he is “heartbroken” to be leaving The Archers for EastEnders. “I can’t bear it.” But after the current high drama, he acknowledges: “You couldn’t do a story of this magnitude again for a few years, and the show needs to return to a much gentler way of telling the stories for a while.” But when he left in the 90s he always knew he would be back, and is sure he will return again. “I don’t think I’ll ever leave. I don’t think I can leave.” (As for the new job, he’s not allowed to talk about it.)

I wondered whether he might at least be glad to see the back of Archers’ fans. The original job advert, O’Connor recalls, read something along the lines of “‘How would you like to work for five million bosses?’ And that’s what it’s like.” Last year, for example, there was minor uproar over fans’ suspicion that the writers had invented an extra bedroom in one of the farmhouses. O’Connor has the air of a man still recovering from it. “But if we went online and said, ‘No, no, no, we’re right, you’re wrong,’ all they would do is complain about everything. So we had to sit there and say nothing, which is very frustrating, but there’s nothing you can do about it, because otherwise you’d need another five staff in the office.”

But he insists he loves the fans. One told him her husband had been in a vegetative state for 13 years, but the consultant noticed that his pupils dilated when The Archers theme tune came on the radio, so she has been playing the show to him twice a day for more than a decade. “When people say that to you, that makes up for anybody that says, ‘Oh, there are only three bedrooms at Brookfield.’ It doesn’t matter, because you know it touches people so profoundly.”

He didn’t wear the tweed suit to ingratiate himself with his five million bosses, he adds as he leaves. “It’s just the only suit I’ve got that still fits.”

• This article was amended on 10 May 2016 to clarify that, in the Archers, Rob Titchener was given parental responsibility for his stepson, rather than legally adopting him.