What a very British disgrace it all is. After endless delays, strikes and ticket-price hikes, as the latest crisis on the railways grinds on, the outgoing chief executive of Network Rail is made a Commander of the Most Excellent Order of the British Empire, whatever that is. The secretary of state for transport faces calls for his resignation, but grimly stays put. And every day, the ongoing saga of Northern trains highlights not just the daily pain inflicted on thousands of travellers in the north-west and beyond, but the awful state of this country’s public transport.

To recap: new timetables were meant to be introduced as part of a big drive to improve services. But, as with Govia Thameslink in the south-east, Northern – a franchise operated by Arriva, the multinational transport giant that is a subsidiary of Germany’s state-owned Deutsche Bahn – had not trained enough drivers. At the same time, Network Rail compounded the mess by allowing electrification work to overrun. An overlooked factor in the chaos is the legacy of something that happened six years ago, when Network Rail centralised its timetabling operations in Milton Keynes, and created a system that had far too little connection with realities on the ground. Such is yet another example of one of the great ironies of recent history: that Thatcherite believers in the liberating wonders of markets have proved to be very good at creating byzantine, top-down, endlessly failing systems rather suggestive of the worst aspects of the old Soviet Union.

The town that refused to let austerity kill its buses | Aditya Chakrabortty Read more

Millions of us live with the consequences of this organised madness, not as a matter of sudden crises but an unending daily reality of delayed journeys, skyrocketing ticket prices, and inexplicable timetables. And all the time the pain is intensified by the sense that, far away, everything is being overseen by panicked people who know far too little – something vividly demonstrated every time Chris “failing” Grayling opens his mouth.

I live in Frome, in Somerset, a growing town of 25,000 with a railway station that sits on a branch line that ends in Weymouth. It is basically a single track covered by a huge wooden shed, and is regularly unstaffed. Our trains are run by the Great Western Railway, formerly known as First Great Western, but renamed in an apparent attempt to make us avert our eyes from daily delays and think of Isambard Kingdom Brunel. If you want to travel the 100 miles to London at peak time, a return ticket bought on the day will cost you £172.

Bristol, by contrast, is 25 miles away; Bath, 12. Lots of people here work in those cities, and travel to them for no end of leisure reasons. But our local transport services make no sense.

During the working week there are three feasible morning rush-hour train services to Bristol, two of which leave before 7am, with an hour’s gap before a service that leaves just after eight o’clock. Each one takes around an hour – almost as long as it takes to drive. For the two hours up to 10.15am, there are no trains at all. To make things even more painful, as subsidies have been slashed, local bus provision has been hacked down. So the car remains king, and the town is subject to regular gridlock.

Plenty of people reading this will have their own version of these problems, and a set of obvious questions about what has gone wrong and how to put it right. I spent a lot of last week discussing potential solutions with people from pressure groups such as Railfuture (which wants “bigger, better railway that people are proud and pleased to use”) and the Campaign for Better Transport. I heard about the endless drawbacks of train franchising: the way that its bidding system emphasises squeezing costs and raising prices, and the fact that deals that tend to last for around eight years combine with the increasingly short cycles of politics to ensure that long-term thinking is always in short supply. People reminded me that the labyrinthine structures of the UK rail system mean operating costs here are reckoned to be around 40% higher than they are in Europe; they also talked about the obstacles to reopening abandoned lines, for which there would be massive demand.

Facebook Twitter Pinterest Rail passengers at Manchester Piccadilly station on the first working day following changes to train timetables, dubbed ‘meltdown Monday’ by unions. Photograph: Christopher Thomond/The Guardian

The recurring theme was the fact that the railways tend to exist in a bubble, cut off not just from other aspects of transport policy but the voices of communities up and down the country. This is one of the reasons why those tweeted calls to renationalise the entirety of the train system that explode whenever something goes wrong have a tendency to sound glib. They offer some answer to the insane complexity and short-termism of our railways, for sure, but fail to get near two other crucial issues: the neglect of local needs, and the fact that trains are only one part of a much bigger problem.

If you live in London, you will know about the benefits of an integrated transport system run by people with knowledge of the places they serve: thanks to Transport for London, it exists. North of the border, with strong caveats, similar things can be said about Transport for Scotland, and in particular its oversight of the ScotRail franchise; in Wales, new devolved transport powers have triggered a plan for hugely improved train services and a new metro system for the South Wales valleys.

In far too much of the UK, by contrast, trains remain a byword for delays and frustration, and the various strands of public transport are wildly unaligned, while most of the country suffers from a continuing failure to tie public transport policy into the everyday world it ought to serve. An example: can you imagine a housing development being built around a new train station, rather than more roads?

Rail Q&A: Northern and Thameslink train chaos explained Read more

To understand how far we have to go, it’s instructive to return to the part of the country at the heart of the Northern trains debacle. In Greater Manchester 80% of public transport journeys actually take place on buses, and though local politicians and public servants are moving towards calling time on thedisastrous 30-year experiment in marketised services and integrating buses with the Metrolink tram system and the area’s railways, the process is so complex that it looks set to take several more years. Meanwhile, even as the area’s population increases, bus use continues to go down.

The Department for Transport has serially refused to cede crucial powers over transport to decision-makers in the north, even when the case for doing so has been completely straightforward. An example: the Greater Manchester Combined Authority wants to take over 96 train stations and their surrounding land currently owned by Network Rail, so that it can spruce them up, get going on plans to put new houses and businesses close to train services, and give people a taste of a more efficient, eco-conscious future. But when presented with the plans last year, the department was reportedly noncommittal – and, according to a Mancunian insider I recently spoke to, responded with little more than an offer of “exploring partnership options”.

“Far too little progress has been made,” he told me. In far-off London, I could make out a familiar sound: the man from the ministry, frantically trying to pull the rusting levers he so jealously guards, while trains carried on inexplicably pulling into sidings, buses remained locked in their depots, and, sighing, people got back in their cars.

• John Harris is a regular Guardian contributor on subjects including politics, popular culture and music