The BBC has been moping about the wrong kind of news. Those of us who suggested that Radio 4’s Today programme would lose its way if it persisted with its new obsessions around the arts, country life and daily puzzles appeared to have been proven right when the latest audience figures were published. Yet the corporation has sternly told us that is not the reason for Today’s loss of 839,000 listeners in a year. No, the problem is the state of our politics. “There has been one dominating story – Brexit – that does not advance in a linear fashion,” said the editor, Sarah Sands. The BBC press office cited the general election campaign as a factor that had driven up the ratings in 2017, but was unable to explain why debate then was apparently more compelling than it is now – when Britain is facing its biggest decision in generations.

It is true that covering Brexit is not easy. Today has been subject to ferocious attacks for its alleged partisanship, and broadcasters have rotten vegetables chucked at them by Remainers and Leavers. This is an inevitable consequence of allowing free expression of opinion, but the trickier challenge for a public broadcaster is also defending the values that matter: truth, accuracy – and being mindful of our national interest. The BBC has always accepted that there are civic duties that flow from its privileged position as a universally funded broadcaster.

The corporation knows it needs to sharpen its act. The latest reshuffle at BBC News will see Kamal Ahmed become editorial director, charged with “shaping future editorial strategy, focusing on storytelling and explanatory journalism” and taking an overview of Brexit. He knows about being in the firing line as the political editor of the Observer when the paper supported the Iraq War. The appointment reflects how difficult it has been for the existing management to focus on programme output through months of distraction from the foolhardy pursuit of Cliff Richard and the rows about pay. Yet the wobbliness in the BBC’s Brexit coverage goes back further than that.

My former colleague Mark Damazer was right to argue in these pages that the referendum showed the need to have more Leave advocates on the airwaves. In the 1990s and 2000s, the case for withdrawing from the EU – as opposed to being a sceptic on an ever-closer union – was seldom articulated on the BBC. Yet having suddenly lurched into a 50:50 balance, the BBC struggled to cope. Many of its approaches were deeply unhelpful. There was the “robotic balance” of the campaign itself, in which a comment by Barack Obama immediately had to be contradicted by Nigel Farage – leading to a yah-boo kind of politics in which nobody was allowed to develop an argument. This was accompanied by a reluctance to differentiate between 300 experts on one side of the debate and a couple of mavericks on the other.

It’s now commonplace to say that the Leave campaign never defined what relationship it wanted Britain to have with the rest of the world – but too often it was never asked. Compare and contrast days of headline coverage in 2018 for Boris Johnson’s views on burqas with no equivalent scrutiny in 2016 of his plans for the single market or free trade. The BBC’s flagship referendum debate majored on the economy (which Remain claimed as their topic) and immigration (which Leave wanted to talk about) with barely a mention of tariffs or the Irish border. Two years on, Westminster process stories still dominate, and “Brexit” becomes boring because it’s presented as being about party leadership ambitions rather than, say, jobs in Bradford.

Remedying this doesn’t require any compromise on impartiality: it’s simply about making an editorial choice of which issues matter. Most of the country hasn’t a clue about the differences between the Chequers plan and the EU’s negotiating position – and yet programmes such as Question Time disdain experts and therefore fail to challenge politicians who say whatever’s in their head, irrespective of reality. This applies to Remainers as well as Leavers: a “People’s Vote” and “an exit from Brexit” are often chanted, but there has been scant examination of the detail. How would this fit with a meaningful vote in parliament? Is there even time?

The broadcasters have always claimed that they are doing the briefings and the fact-checks somewhere in the output: online or in a genteel programme on Radio 4. This is often true, but there is a gulf between the headlines on the daily mass-audience bulletins and the correctives on the website. It’s encouraging that more effort has been made to bring the two together, with the BBC’s Reality Check more prominent in television news and on the Today programme, where it discomfited the Tory MP Bernard Jenkin. But this isn’t a “nice to have”: it’s something that should be in every moment of every day, so facts win out over prejudice.

It’s more than 30 years since John Birt began a revolution in the BBC’s journalism that prioritised expertise and analysis, and which discarded the froth of news to concentrate on its substance. A celebrated edition of the main television news devoted a majority of its airtime to explaining perestroika and glasnost in the Soviet Union. His lieutenants included two future directors-general in Mark Thompson and Tony Hall. Perhaps Ahmed should invite Hall out for a coffee to find out how they did it, and how today’s BBC News might recover its lustre.

If glasnost was worth exploring for half a network news bulletin, shouldn’t there be a similar commitment to examining issues such as what leaving on WTO terms would mean for the people of this country? And when there is a reckoning about what happened to our politics, the broadcasters cannot and should not be exempt.

Roger Mosey is master of Selwyn College, Cambridge, and a former head of BBC Television News