On the face of it, a party leadership election that stimulates over half a million people to take part should be a cause for celebration; a remarkable political mobilisation in an age when parties are in decline. Yet the Labour leadership election is being derided as a fiasco and a political disaster. The new rules which have allowed non-members to take part have come under particular fire. Rumours of legal challenges abound. How did it come to this?

As director of the Labour Party’s Constitution Unit I saw at first hand the process by which the new leadership election rules were developed, and worked alongside Ray Collins and Jenny Smith on the final package of reform that was passed by an overwhelming majority at the special conference in March 2014. The change was sold by Labour spinners as a modernising moment on a par with Tony Blair’s revision of Clause IV. The reform was undeniably significant. But it was not the culmination of a carefully crafted modernising plan. Rather it was the product of political panic, damage limitation and - ultimately - a deal with the unions.

Refounding Labour

That is not to say the changes lacked any sense of political vision. A recurring theme of Ed Miliband’s leadership was the desire to resurrect Labour as a ‘movement’ with a mass membership actively involved in community politics. That had been evident through the “Refounding Labour” reforms of 2011. Those changes added a new commitment in the party constitution which obliged Labour not just to fight elections but to “make communities stronger through collective action”. More substantially, Refounding Labour established for the first time the right of non-member supporters to vote in leadership elections.

That, too, was spun as a significant change. But the headlines masked a crucial caveat. The new section could only be triggered once a minimum of 50,000 supporters had joined. Everyone rested easy after that. It would never happen. The unions weren’t going to organise supporters and, in the absence of any pressure from the leadership, neither was the party. The new supporters section would most likely have withered on the vine – had it not been for a row over selecting a parliamentary candidate in Falkirk.

The St Bride’s Speech

The Falkirk episode was a self-inflicted mess that got out of hand – personal psychodrama as much as political dispute. It culminated in the resignation of Tom Watson from the shadow cabinet and a panicked decision to call in the Scottish police.

Four days after the police referral, on 13 July 2013, Ed Miliband delivered a speech at the St Bride’s Foundation on transforming Labour’s relationship with the trade unions. The speech contained no direct reference to the trade unions’ role at Conference, their representation on national committees, or their share of the leadership electoral college. Instead it covered a strange hotchpotch of measures including a commitment to introduce open primaries for mayoral selections; new limits on spending in internal elections; and reform of constituency development plans.

At its heart, however, was a proposal to turn union funding of Labour on its head. Instead of trade union executives paying affiliation fees on the basis of the number of levy payers in their ranks – a predictable and relatively strings-free flow of cash for Labour – in future millions of individual trade unionists would each have to give their active consent to the payment of fees. The goal was a new and more direct relationship between the party and individual trade unionists. Those who consented to the payment of affiliation fees would effectively become members.

For years Labour had resisted Tory calls for union money to be subject to an “opt-in” process (which they are now implementing, with bells on, in the Trade Union Bill). There was a case for reforming Labour’s finances, but in the context of a broader change to the entire political funding framework. The sudden pronouncement to upturn affiliation fees was seen by many in the party as an act of unilateral political disarmament. A Lib Dem representative who had sat opposite me during months of cross-party talks on political funding emailed to ask why we had done it.

The principled case was that in the twenty-first century it no longer made sense for trade union members to pay fees to Labour unless they had personally chosen to do so. Yet the moral high ground had a cost. Internal estimates suggested the party could lose up to 90 per cent of the affiliation fees through the opt-in mechanism.

There was some suggestion that the party could make up for these losses by continuing to accept big discretionary donations from trade unions. But such a shift from affiliations to donations would put Labour in a terrible position. A stable source of income was being swapped for an unstable one that handed trade union executives greater leverage. Yet it ultimately contained the seeds of destruction for the union link, as it would fatally undermine the traditional model of collective affiliation.

According to one recent account, fear that the reforms would break the link prompted the hard left in the unions to oppose the reforms. In actual fact they welcomed Ed’s speech for that very reason. Len McCluskey was the most enthusiastic endorser of his plans. That reflected shifting politics at the top of Unite, where there was a growing belief that the union should begin to detach from Labour and reduce its funding for the party.

The Unite leadership looked upon affiliation fees as wasted money which was not delivering sufficient influence. Instead, they sought two strategic goals: first, to secure the selection of Labour parliamentary candidates who were close to the union; second, to develop a more transactional financial relationship with the party by switching from affiliation fees to discretionary donations.

In contrast, other unions led by general secretaries more strongly attached to the Labour Party were horrified at the St Bride’s speech. Paul Kenny, of the GMB, was apoplectic. While his union, like Unite, initially reacted by slashing the amount of affiliation fees they paid to Labour, the motive was entirely different. They were trying to pressure Miliband into a U-turn by showing him the consequences of his actions.

The Collins Review

In turning the St Bride’s speech into practical reality the Collins team determined that the proposals needed to be recast in two key respects.

First, to maximise affiliation fees individual choice would have to be accommodated within a structure that still enabled unions to affiliate as collective bodies. In other words, we would retain the traditional federal structure whereby organisations and individuals could both be joined to the party. That would work by determining union representation according to the numbers of their levy paying members who gave active consent to the payment of fees.

Secondly, to protect parliamentary selections from manipulation, only full party members would have constitutional rights at a constituency level. That was bitterly opposed by Unite. Trade unionists who agreed to the payment of affiliation fees and, separately, signed up as Labour supporters, would not be eligible to participate in local or parliamentary selections but would have certain rights at the national level; most notably in respect of leadership elections.

After all, members of affiliated trade unions already had that right. In 2010, 2.5 million members of affiliated unions received a ballot for the Labour leadership. They were required to tick a box declaring their support for Labour values, but that was the extent of the verification. The ballots themselves were distributed not by the Labour Party – which was never given the names and addresses of these people – but by the unions themselves. That enabled them to plaster envelopes in propaganda for Ed Miliband. In the event it was the extent of Ed’s lead among affiliated members that enabled him to secure the leadership despite losing in the other two sections.

The leadership election system contained many imperfections and was ripe for reform. The St Bride’s speech, perhaps unwittingly, ensured it would happen.

The Leadership Election Rules

The move to an “opt-in” system of affiliation fees was bound to dramatically reduce the number of trade unionists eligible to take part in a leadership election. That undermined the claim of affiliates to hold a third of the Electoral College. It would have to be reformed.

The initial plan, agreed with the leader’s office, was to press for a new college based on one-third MPs and MEPs, and two thirds members and affiliates. Two problems immediately arose. One was the complicating factor of (non-trade union) registered supporters, whose potential presence in the leadership ballot had been established under Refounding Labour.

The other more serious issue was trade union opposition to the dissolution of the affiliates’ section. That was ultimately only overcome by agreeing to dissolve the entire college and removing the MPs as well.

The result was a new system in which members and supporters alike voted in a single ballot. That gave the party oversight of the electorate, which meant multiple voting could be ended and abuses round the dissemination of ballots stopped.

The new process was trumpeted as the final realisation of the OMOV reforms begun by John Smith. But it was by definition not an OMOV ballot. Supporters were now involved.

Some warned that the change was dangerous and gave the unions what they always wanted – the destruction of the MPs section of the college. They argued that Ed should have insisted on a pure OMOV system as the price for removing MPs from the college. But great fanfare had already been made of opening the process up to non-members in 2011. Rolling back from that position would have been politically difficult.

In any event, the trade unions would have rejected demands for OMOV. Not for the first time in Labour’s history, what arguably should have happened didn’t correlate with what could have happened.

The 2015 Leadership Election

In practice the new system has evolved in ways that no one imagined. For a start, no one foresaw Jeremy Corbyn making the ballot, let alone emerging as the favourite to become leader.

Ironically the system was intended to stop a candidate who could not command the Parliamentary Labour Party (PLP) reaching the ballot. In response to the dissolution of Electoral College, including the MPs’ section, the nomination threshold to enter the contest was raised to 15 per cent of Labour MPs. Consideration was initially given to a higher threshold of 20 per cent or 25 per cent. But that was rejected on the grounds that it would narrow the field too much; members from both the Blairite and Campaign Group wings of the party favoured a lower number. Nonetheless, 15 per cent was judged to be a safe barrier to any outsider – especially someone from the hard left. That judgement proved to be mistaken.

Corbyn was placed on the ballot. His presence has stimulated a surge in supporter registrations that has prompted claims of organised entryism. But the extent to which hostile political groupings have managed to infiltrate the contest on a scale that could distort the outcome looks open to question. If Trotskyist and Green groups really have that kind of influence they have been keeping it well hidden. Furthermore, the Labour Party – for all the criticism directed its way – has worked tirelessly to vet the new joiners; way beyond anything it has run in the past. Individuals who could be identified as having campaigned for other parties have been weeded out.

Yet despite this activity constituency reports suggest that up to 15 per cent of new joiners do not have any record of voting Labour. That has to be a cause for concern. The supporter concept was built on the premise that there are large numbers of people may not be Labour members but vote Labour and could be encouraged to become members via the supporter stepping stone.

If, in practice, significant numbers have entered the party under the guise of “supporters” but with malign intentions, then that raises a question about the validity of using such an open process to select the party leader. It seems unlikely that the current election will be halted but looking ahead the party will want to consider the use of freeze dates, especially in terms of registered supporters.

That also relates to the issue of affiliated supporters. The capacity of trade unions to organise recruitment was always understood and seen as a double-edged sword. The party needed active union support if affiliation levels were to be maintained at any kind of level. But the recruitment of supporters around a leadership election held obvious dangers, because it risked unions recruiting on behalf of a particular candidate. That went against the spirit of the reforms, which were aimed at moving away from the sort of organised influence that unions had exercised in 2010. Had a freeze date been set for before the point that nominations opened, it may have helped to reduce that activity.

But, important though they are, issues around freeze dates are in essence technical points. The bigger question for the future is where to draw the line between having a committed, secure membership and an open democratic process that encourages participation. That is not a new question. When the Labour conference first considered the proposal to admit individuals as members before the First World War, the idea was rejected by trade unions and socialist societies for fear that it may open the door to “men who had not the interests of the party at heart”. Eventually the benefits associated with admitting individual members were judged to outweigh the risks. The same judgment may yet be made in respect of the contemporary system for electing the leader, though for many the advantages are not currently obvious.