How on earth could one of the world’s most advanced cities manage to disrupt its essential transport infrastructure, the lives of so many, and its economic activity so foolishly, and with such little interest from those in power?

That was my conclusion in 2003. Today, exactly those words apply to another rail foul up with Southern Rail and to similar circumstances for many other British commuters and cities.

The scale of human misery, stress, discomfort and the impact on working lives cannot be measured in words. And that’s from someone who’s spent years writing reports for governments and finding answers. You would think I could come up with a way to qualify this. Let’s just call it plain idiocy, because that is what it is.

Despite the pressure from millions of angry commuters, transport select committees, marches on the Department of Transport, petitions and tweeters desperate for a line of communication with the operator… Nothing. There is no effective channel for change.

This tweet – by Channel 4 news presenter Cathy Newman – gets to the nub of the question:

What I can't understand is, despite passengers' frustration, nothing is changing. Are operators or govt to blame? https://t.co/4dAq5pcYm3 — Cathy Newman (@cathynewman) November 22, 2016

Hmm. Operators? Regulators? Politicians? Civil Servants? Who the hell is to blame? I’m an engineer at heart, turned consultant, turned policy adviser. So one day when the engine fell off a tube train and left the Central Line out of service and ½ million passengers a day to find ‘alternative routes’, I began investigating. I was keen to unpick the thread which worked out where responsibility lay for me being without my service for 3 months.

Do bear with me. This was over a decade ago. But this experience saw a similar scale of human and economic disruption to a British city alongside scant interest from government to the effects of today’s rail chaos.

The story – in some ways a piece of detection – shows why governance lies at the heart of idiocy off the rails.

Plus, the incident looms large in my life – it was the launchpad for me writing a book, based on 100s of interviews with ministers, peers, judges, civil servants, councillors, doctors, teachers, gardeners, builders and many more for a system of government that delivers. But first, let me tell you about the day my tolerance of a crap commute snapped.

It was snowing in London. It had been a long day. Usually I would have walked into Bank station and been home thirty-five minutes later.

My feet dragged. Some weeks before, a train had derailed in a tunnel after its engine fell off (no one knew why), injuring thirty-two passengers, closing the Central Line. My route that night was overground to Chingford where I came out of the station at 11 pm. With no buses operating or taxis for hire, I walked for an hour through the snow in Epping Forest. Home once more, cold, wet, tired, pissed off, and so looking forward to battling transport once again early the following morning. Joy.

Three months after closing, the line fully reopened.

Being a problem-solver by occupation and inclination, my frustrated mind turned to this: Surely keeping the engine attached to the train is a key task for rail operators. Who was responsible, first for trains with detachable engines, and second for sorting them out when they went wrong? And where was the sense of urgency? To whom did this matter?

The answers were fuzzy. Back then responsibility for London Underground lay with the Department for Transport, which is run by civil servants who value the ‘clever generalist’ and the ability to ‘master a brief’. Because they are clever in an academic sense, these generalists believe they are capable of almost any of the hundreds of diverse tasks found in every government. But no one there had any operational experience of commuter railways, or of running a large organisation like London Underground, or of attaching engines to trains.

Vitally, neither had the ministers, who were unable to interrogate the rail management and challenge its decisions.

So here we go. Issue number 1: There were few/no specialists in government who understood the, ahem, nuts and bolts of running a rail service.

Next, in terms of public pressure the chain of accountability was as strong as a cardboard bicycle lock. It wound its way from us, the users and voters, through to the national government and the election of a political party every four or five years, through its many priorities, on into a minister of the day with his or her agenda, onward to the civil servants and thence to London Underground. In other words, the next time the government might feel some heat over this issue was two years hence at the general election, by which time it would have been put to bed and politically forgotten. In effect, at that time, there was almost zero accountability of London Underground to the public.

Onwards. Issue number 2: There were no formal means to address policy failure. This, coupled with the regular changing of the ministerial guard, meant there was no one with whom the buck stopped. Starting to see any similarities here?

Many modern countries have real local government, where non-functioning train lines become top priority very quickly. London had finally got this with the election of its first executive mayor in 2000, and the establishment of its operational arm Transport for London. Good local democracy increases transparency and galvanises that accountability chain.

But a spanner had been inserted in the works. The government of Tony Blair – another of the ‘absolute monarch’ tendency – prevented the transfer of control of London Underground to the GLA until 2003, after itself signing controversial, flawed and ultimately failed private finance initiatives for track maintenance. Why they stopped the transfer was as much to do with political and personal jealousy as contracts – the new mayor had stood for office as an independent against the wishes of his party’s hierarchy. Even worse, he was popular and competent.

So, bam. Issue number 3: A step to introduce democratic accountability for London and its public infrastructure had been derailed.

Once Ken Livingstone got hold of the Underground it started to be managed more for its passengers than for the rail union’s officials, management, and drivers. As an executive mayor, with both the power and the responsibility to appoint his civil servants, he replaced 27 of the top management at TfL and brought in hardened and experienced metro managers from New York. The service improved as the executive mayor delivered what he had been elected to do. If the government system around it is further developed (developments identified in my book) then it will get better again. Otherwise it won’t.

Totting this all up, the underlying reasons that led to the engine dropping off were (intake of breath!): an unaccountable London Underground; civil servants without the experience or idea of how to run a large metro service, nor interest, nor organisational motivation; no feedback or monitoring of results by Parliament; prime ministers with too much power, and with psychological flaws; and no executive mayor in control.

Two political parties, three prime ministers, five governments and a herd of transport ministers had between them created, or allowed the conditions for, something off the dumb scale. And none of them meant to.

There was no single cause, no headline howler. It was the system – the system of government to, in this case, run the trains – that was at fault.

And, the solution was not to be found anyway near privatisation but in governance.

So there’s my story of 3 months spent tramping through Epping Forest at uncommon hours of the day. But I hope you get my drift. While superficially the cause was a broken engine, the unseen hand of governance was really the villain and is still the villain today.

What do I mean with by this? Well, it’s the bigger picture. You can’t replace a screw or tighten a bolt when the system is an old banger. And our system of government is an old banger. Contrary to what many in power would have us believe – that the system we have is essentially the only one possible – there are all manner of ways to run governments and public services. The UK happens to have one way. Don’t say it can’t be changed – it can; but not by using the thought processes usually employed by those within these government systems. A different perspective and discipline is needed across the board.

Back to today’s railway system, how would a sensible government sort out this idiocy? It does not work, for three main reasons:

Its Design: Our government still seems to believe that because it’s private it will work. We don’t experience astonishing quality in cars because they are produced by the private sector, but because they are the product of the system of market capitalism: highly competitive and reasonably accurately regulated. Contrast this with ‘privatised’ railways – highly uncompetitive and mis-regulated. Competing a monopoly of a rail franchise once every 10 years is not competition. Imagine a single company being awarded a franchise to build all cars in one region for 10 years.

If it is assumed that a problem can be solved by the application of say, economic theory or a legal construct alone, then the result will miss key parts of the whole system. Privatising the railways was dreamt up by an economist – a former colleague of mine, and it suffered from single discipline syndrome. It requires experienced contract managers to run the operating companies tightly, and not the civil servants of the Department for Transport (or DaFT as Private Eye terms it) hiding behind contracts sold to them by expensive city lawyers as running themselves. No contract does. Successful contract management is found in the sort of process plant contractors I used to work for, where the company has developed and refined this specialism over decades, along with its staff. No ‘generalist’ would be let anywhere near managing the construction of an oil rig. The whole system conspires in hiding as much as possible, thus to prevent any accountability to customers and taxpayers. Passengers are plied with platitudinous pretexts from the immortal ‘signal failure’ to the absurdly disingenuous ‘half term’. We all know when half terms are coming. If only, as in Martin Rowson’s cartoon, we could insist on a real answer for late running trains.

Despite this, it looks as though the government is to reach into its ideology handbag yet again and pluck out the privatisation potion as the solution by transferring Network Rail’s track responsibilities to the train operators, like Virgin and Arriva. This comes smothered in irony as British Rail in the 90s had restructured into sectors of combined track and train operations in separate regions/routes. Privatisation split them. The failed private Railtrack became the public Network Rail. Now Network Rail is being split and a large chunk going to the Train Operating Companies – combining track and trains, with a structure very similar to British Rail in the 90s – but now private. Dizzy? Zigzag?

May I humbly suggest an alternative way forward?

We should start by engaging all those specialists and everyday users and staff in bringing to the table relevant facts and information, independently verified, with spun statistics banned. With these ‘stakeholders’ Parliament would examine successful railway systems in other countries, carefully understanding the whole system and its governance. It would then develop options for a new one, again on which it would engage and deliberate. A preference would form. Laws would be passed and then the difficult stage commence: doing it. With all round will this could be done in 1-2 years. Then decent railways would return.

This is not a matter for the zigzag of two party adversarial politics left to the psychologically flawed to express their ‘beliefs’ in the form of some time-expired ideology, with the next swerve in policy a new minister or a new government away, working in private.

Too simple? Party politics would show no self-control? Getting our railways right is an example of where a ‘Politics Aside’ approach would not just work, but would arrive on time. Let’s kick out the ideology handbag, along with the ‘half terms’, and create a world class railway that actually works.

Ed Straw

2000 words