On Monday, only hours before the drama of parliament’s prorogation, John Bercow announced he would retire as Speaker before the next election. Downing Street had declared war on him in the Sunday papers, briefing that it would break convention and seek his removal from his seat at the next election. Instead, Bercow has sought to undercut the government by giving the current parliament, habituated to autonomy and rebellion against the whips, a vote on his successor.

Bercow’s ‘constitutional vandalism’ is unlikely to trouble any government with a majority

An erstwhile member of the Tory hard right – in the 1980s he chaired the Federation of Conservative Students, which was notorious for its “Hang Nelson Mandela” literature – his political journey has seen him end up an improbable poster boy for dewy-eyed liberals. Even more surprisingly, his glowering manner, gravel-voiced barking and theatrical chiding of MPs have made him an internet meme. He has presided over significant, and in some parts deeply resented, changes to parliament’s functioning; the contest to replace him will be the most highly politicised and bitterly fought in living memory, with MPs aware that in a deadlocked Commons the Speaker’s powers are profound.

When elected, a new Speaker lays claim to “ancient and undoubted rights and privileges” of parliament, including “freedom of speech in debate, freedom from arrest”. Defence of those rights has been the Speaker’s watchword since the tenure of William Lenthall in the 17th century. In one of the most dramatic episodes of England’s descent into civil war, Lenthall had refused, to the king’s face, to identify five MPs Charles I sought to arrest. The moment has entered British civic myth as the first clear assertion of the Speaker’s loyalty to parliament rather than to an overmighty executive power; it is painted in fresco in Westminster.

Lenthall’s lessons for modern Speakers aren’t confined to the doughty defence of parliamentary rights. His defiance was a finely tuned balancing act: courage hedged with deference to an irascible monarch. Fraught management of contending political forces made him a target of a newly free press, which gossiped of his avarice and pilloried him as a secret royalist. Bercow might recognise that.

The Speakership is an odd institution. Its long evolution since Lenthall’s day has imbued it with considerable power: to shape the debate in parliament, select amendments and interpret parliament’s rules. That power comes at the cost of scrupulous political neutrality, a principle that emerged in the 18th century and which has been observed as ironclad for at least the past 100 years. In the US House of Representatives the role is highly partisan; in France and in Germany the speaker maintains an active involvement in party politics. The UK’s Speaker resigns from their party and usually stands for re-election unopposed by major parties; Baroness Boothroyd, Bercow’s only living predecessor, observed that a new Speaker is frequently harder on their former ideological confreres, in an attempt to achieve impartiality. Bercow’s enemies in parliament – the current government and its predecessor, as well as the no-deal hardcore of the European Research Group (ERG) – charge him with shattering this impartiality during the long Brexit impasse.

Operation Yellowhammer and parliament prorogued - Politics Weekly podcast Read more

Bercow had to change things. His election, in 2009, came after the resignation of Michael Martin, in the midst of the MPs’ expenses scandal. Bercow billed himself as a moderniser and “backbencher’s friend” – his drift towards social liberalism had left him out of favour with Tory leaders, and was disfavoured by the whips. A memo circulated to all candidates by the then clerk of legislation, Robert Rogers (now Lord Lisvane), laid out a 75-point menu for reform of the Commons; it was beyond the Speaker’s scope alone to implement much of it, but it certainly inspired Bercow’s expansive granting of urgent questions – 152 in the last year, as opposed to just two in the year before his accession – allowing MPs to haul ministers to account in the Commons. Any Speaker elected in 2009 would have necessarily been a reformer: the impetus to reform came from the political establishment itself, aware that Westminster had reached a new nadir in public esteem.

Until this parliament, it was Bercow’s personality rather than his relatively modest procedural innovations that was the focus of ire. His aggressive, occasionally vaudevillian style in the chair, and his willingness to chastise ministers, led to an abortive plot to oust him in 2015. More seriously, allegations of bullying have overshadowed his office in recent years, and perhaps as a consequence he has been ineffective in acting to stem the bullying and harassment endemic in Westminster. Accusations of partiality, some rather confected, long smouldered on the Tory benches – but it is his handling of Brexit procedure that whipped them into conflagration.

Bercow’s accusers claim he has ripped up parliament’s rulebook to suit his own political preferences (in 2017 he unwisely disclosed his own vote for remain). Not only has he allowed MPs to seize control of proceedings from the government, but colluded with them by widening the scope of emergency debates to allow parliamentary insurrections. The latter is inescapably a constitutional innovation. Bercow doesn’t shy away from the charge of novelty: for him, the letter killeth but the spirit giveth life – he insists on parliament’s right to express its will, and on defending that right in the face of an executive determined to marginalise it. Nor is he finished yet: in last night’s Bingham lecture, he vowed further “procedural creativity” should Boris Johnson prove determined to ignore the law.

The orgy of suspicion and paranoia that followed the Brexit referendum has touched many putatively impartial institutions: both press and judiciary are regularly accused of barely veiled remainer bias, but nowhere has been more assailed than parliament itself. It is a symptom of a deeper malaise, in which the reserves of civic trust necessary for political representation have evaporated.

The Speaker’s rulings are a consequence, not a cause, of the political impasse – the unavoidable effect of a conflict between legislature and executive eagerly pursued by successive governments. Small majorities or minority governments – as much the result of a democratic process as the referendum – find their rebels empowered; if any such government treats parliament as a truculent inconvenience, it will find MPs ready to humble it. Bercow’s instinct to empower parliament is real and consistent – though he delights in making enemies – but his “constitutional vandalism” is unlikely to trouble any government with a majority. Whatever the tabloids claim, this government’s inability to win a vote is its fault, and its fault alone.

An alliance between progressive parties would be fragile, but it could win an election | Zoe Williams Read more

The competition to replace Bercow is already under way. Some candidates, such as Labour’s Chris Bryant, promise a return to procedural orthodoxy: umpire-like impartiality, slavish adherence to the rulebook. Yet it is hard to see how any candidate could avoid controversy in a restive, finely balanced parliament with an executive determined to test the limits of its power and a legislature impatient with abuse.

Brexit has seen the withering of many parliamentary delusions: the strictures of convention, the stability of the parties, the reliability of the “good chap” theory of office. Bercow’s pugnacious defence of the legislature is a boon, but he must realise that reform conducted solely through the Speaker’s chair also tests democratic legitimacy; such a realisation must be behind his embrace, in last night’s lecture, of a more codified constitutional settlement. As the legal case over prorogation waits to be heard by the supreme court next week, and the looming Brexit deadline leaves the government scrambling for political safety, Bercow’s final act remains yet to be played. It is certain to be a stormy one.

• James Butler is co-founder of Novara Media, and hosts its weekly radio show on Resonance FM. He writes regularly at the London Review of Books