Do the public have a right to understand why some government decisions go so badly wrong? That question is at the heart of a power struggle which is being fought in Westminster and Whitehall.

The leading figure on one side is the fiercely energetic chair of the MPs on the Public Accounts Committee, Margaret Hodge. She has upset the Civil Service with her searing inquiries into waste and mismanagement in government because she has adopted the novel view that senior civil servants should be held responsible for their actions and their advice.

On the other side, leading the fight to maintain the status quo and determined to stop this trend for public accountability before it goes any further is the ex-head of the civil service and one-time moderniser, Lord (Gus) O’Donnell. The conflict between them, which broke into the open with the leaking of a silkily unpleasant and highly critical letter from O’Donnell to Hodge last month, will ratchet up this Thursday when Hodge gives a speech to the think tank Policy Exchange, setting out just why she and Parliament shouldn’t allow these vested interests to keep their actions hidden from the rest of us.

Hodge is one of a new generation of select committee chairmen who are fed up with finding their investigations into government behaviour blocked by the 19th-century convention that civil servants aren’t accountable to Parliament. The only exceptions to this are Permanent Secretaries, who are required to answer questions about how their departments spend their budgets. For everyone else, and in every other area, the theory is that officials answer only to ministers, and it is ministers who must take responsibility for what their department does.

This convention may have made sense a century ago, when a department such as the Home Office consisted of 28 civil servants, and a Secretary of State really would have overseen all its key decisions. Today, with the same department employing 70,000 people, both directly and in government agencies, and with all departments having grown in the same way, the idea that ministers can control or answer for everything that their officials do, or have done in the past, is just a silly fantasy. But if ministers often can’t know what’s going on, and officials aren’t required to explain or justify themselves, that leaves a huge hole where public accountability ought to be. It leaves the Civil Service as the last major institution in Britain which doesn’t have to answer to any outsiders for the way it behaves. That is exactly the problem that Hodge and other active MPs, such as those on the Treasury and the Public Administration Select committees, are attempting to remedy.

The sheer inadequacy of the current system of inquiry is shockingly evident in a recent PAC report on an abandoned project, FiREControl. The last government wasted almost half a billion pounds over six years on this attempt to rethink the fire service’s response to emergencies. The Coalition cancelled it. It was an appalling squandering of public money but convention meant that the PAC couldn’t question any of the people involved in the decisions over time, from the three Communities Secretaries who let it run on, to the two Permanent Secretaries, the five “Senior Responsible Officers” and four project directors. Instead, the only civil servant they were expected to interrogate was the current Permanent Secretary, in his role as accounting officer for his department. Since he wasn’t there at the time the project was happening, the usefulness of his contribution was limited. Exasperated, the Committee noted that “no individuals have been held accountable for the failure and waste associated with the project”.

This goes to the heart of the accountability issue. Officials are insulated from interrogation from MPs but they are rarely penalised by their departments either. They are supposed to answer to ministers, but ministers can’t manage them, because that would threaten the Civil Service’s impartiality. Politicians cannot appoint, promote or dismiss staff, no matter how talented or useless they may be. And the Civil Service itself, left to operate in private, doesn’t do much to manage performance.

An IPPR report six years ago, Whitehall’s Black Box, had devastating quotes from insiders about the prevailing culture. Interviewee after interviewee agreed that there was no price for failure, that there was a complete lack of internal accountability, and that poor performers were not removed but moved around. Insiders say there’s been no effective change since.

It is for all these reasons that Hodge has started pushing at the boundaries of what committees do. In a recent inquiry into a total fiasco over subsidies to farmers at the Rural Payments Agency, she insisted on an order requiring the then Permanent Secretary, not the current one, to appear and answer questions. That sent a wave of fear and indignation through Whitehall; what precedent was being set? Then came her famous attempt last year to uncover the truth about the way in which an HMRC official had secretly allowed Vodafone to escape paying up to £10 million in tax. The Treasury Select Committee had already been frustrated in its attempts to get answers. Hodge, faced with an evasive HMRC lawyer, was so irritated by his unwillingness to be straight that she fetched a Bible and demanded that he swear on it. The Committee’s subsequent report was blistering in its account of how HMRC had indeed reached secret deals against taxpayers’ interests, and had hidden crucial facts from all of us.

There is no question that Hodge is performing a public service. O’Donnell wants to stop her. His letter to her accuses her of “theatrical exercises in public humiliation”, disagrees with the principle of questioning senior civil servants directly, and warns her that “there is now a serious issue about the way that you are perceived in the Senior Civil Service”. This is basically an attempt to cow Hodge and the PAC into pulling back.

It cannot be allowed to work. Senior ministers, ex-ministers, political opponents, the Institute for Government, the IPPR — everyone I spoke to agreed Hodge is right. Senior people who are responsible for decision-making must be held accountable. Anything else is absurd, politically and administratively. O’Donnell’s rearguard action is just the last defence of an institution which is clinging hopelessly to its privacy and its privilege.