The era of “private good, public bad” is drawing to a close. Unshakeable faith in Margaret Thatcher’s privatisation creed is being killed off not just by counter-ideology, but by the sheer irrationality, expense and failure of so many private contracts. Carillion’s spectacular collapse makes big headlines, but out of the spotlight local councils, under extreme stress from cuts, are cancelling contracted-out services. Why? Because it saves them money and improves services. I have been talking to councils around the country where in-sourcing is how they best cope with savage budget reductions.

Start in Thurrock: there is nothing leftist about this council, controlled by minority Conservatives propped up by Ukip. In 2015 they bought out their Serco contract, bringing office services back in-house. Lyn Carpenter, the council’s chief executive, is blunt: “The contract was a disaster. Nothing was flexible, everything set in stone. If we wanted anything done differently, they said ‘That’ll cost you.’” In that inflexible 15-year contract, Thurrock says, everything was still done on paper, as Serco had no incentive to modernise. Escaping it was “quite a fight”. Thurrock paid £9.9m to buy out a £19m-a-year contract, but the cost was recouped through extra efficiency and saving “the £3.6m a year profit they were taking – money not spent locally”.

I talked to some of the 300 staff brought back in-house, who described how frustrating it was to sit next to council colleagues yet never be able to cooperate or answer their phones for fear of affecting Serco key performance indicators. “We could never suggest better ways, as everything was done to contract.” Street cleaners brought back in-house “as part of our team” now monitor potholes and fly-tipping.

Everywhere, council budget cuts of between 40% and 50% mean losing staff and services, yet they can’t touch anything fixed in a contract. Time and again, councils report the results of outsourced work to be out of date and inflexible. Nick Forbes, the Labour leader of Newcastle council, tells how in 2010 Gordon Brown forced Tyneside to put its metro system out to tender. DB Regio, a German government-owned company, won the contract but it was “a disaster”, he says. “When they couldn’t make a profit they cut services and complaints soared. On Great North Run day, when 55,000 runners rely on the service to get them back again at the end, they failed to put on any extra trains.”

The local transport executive found the company was budgeting for the fines brought about by failure. They took the metro back in-house two years early at no extra cost. Forbes boasts of “a significant improvement in reliability”, an excellent relationship with unions and replacing old trains instead of leasing them. When North Tyneside was Tory-run, it was a Conservative flagship for massive outsourcing. But now the Labour-run council is taking services back, as “the contractors added cost, not value”. One bad contract has years more to run: “But we can’t afford £40m to buy it out, as there are punitive penalty clauses.” Again, they talk of private contract inflexibility and refusal to invest. Councils say neither writing nor trying to escape contracts is ever a level playing field, as they find themselves up against super-sharp company lawyers.

In Redbridge, the council’s cabinet voted last week to bring back in-house their waste disposal services: the current Amey contract allows no waste food collection, and they complain about one badly drawn-up 25-year contract with “zero flexibility”. In Blackpool their Onyx contract runs out next year and they are taking waste disposal back: “No flexibility, not to collect Christmas trees or cardboard. They cost £3.8m and we can deliver better for £3.5m, and pay staff the real living wage, with money staying in our community.” West Sussex, Birmingham, Slough and scores more spanning political parties are now in-sourcing. Why? They complain of cost, and “red tape” – the traditional rightwing objection to the state.

Talk to Reading and they boast of how, along with Nottingham, only they managed to hold onto their bus service in the monstrously misguided Thatcher deregulation. Both cities see rising bus use, falling almost everywhere else. Their profits are up, paid back into improvements. Other cities eye their success with envy.On Monday the Smith Institute launches a timely report, Out of contract: Time to move on from the “love-in” with outsourcing and PFI. It calls for an immediate pause in all public service contracting, plus a regulator to oversee a “Domesday Book” audit of all existing deals. It estimates £100bn is contracted out, but no one knows which contracts are good or bad value. Public managers are contracting in the dark, often with companies that are tax avoiders, and refuse to contribute to the state from which they cream profits.

Contracting out sees the public ethos swallowed up in boardrooms

Last week the National Audit Office judged PFIs a dereliction of public value for money. Add to that the many famous failures of the near-cartel of big outsourcing providers: the probation service; G4S and Serco, fined over dishonest electronic tagging; the G4S Olympic security calamity; hospital cleaning, neither clean nor responsive; Capita failing GP back-office services; the failed courts translation service; shudders of alarm at Virgin Care taking over entire children’s and adult services. Stagecoach and Virgin will exit early from the East Coast mainline and will not pay the government as much as originally forecast, though when state-owned it made a good profit. Learndirect has failed apprentices. Look at how privatising care has led to the collapse of care homes. The UK does by far the most outsourcing in the EU, often to EU state-owned companies – yet it has worse services.

Where is the border between sensible and senseless private involvement in public services? That’s always a wobbly line – no one expects the NHS to manufacture beds or drugs – but the old “private best” dogma has run out of road. Carpenter of Thurrock talks eloquently not just of money saved, but of Thurrock council as the largest local employer plugged into its community through its workforce: “They know the area, they are our eyes and ears and we need them to feel valued by us. I want them all – street sweepers, everyone – to be the friendly face of the council wherever they go.”

Contracting out sees that public ethos swallowed up in boardrooms controlled by shareholders with no interest in the public good. No wonder renationalisation of utilities and rail has 80% support in polls. The questions will be how it’s done, at what cost and with what guarantees of future state investment. But those who imagine that Venezuela beckons if the state becomes preferred provider should listen to those councils struggling with the financial and social costs of runaway privatisation.

• Polly Toynbee is a Guardian columnist

• This article was amended on 23 January 2018 to clarify details relating to the Virgin/Stagecoach early exit from the East Coast mainline contract.

