Once upon a time, not that long ago, housing was largely what councils were for. In the years after the second world war, after all, public housing was intended to be an option for all who wanted it, as universal a welfare service as state education. And so, in the 1960s and 70s, visionary architects and idealistic planners alike flocked to local government: if you wanted to make your mark on the fabric of a city, this was the place to do it.

After the Thatcher government introduced right to buy, the political class began instead to treat council housing as a last-chance saloon – a place for that tiny sliver of the population too poor or too lacking in aspiration to get on and buy. What’s worse, councils were barred from using the proceeds of right to buy to build replacement housing. So they stopped building. In the mid-70s, councils were building as many as 150,000 new homes a year. In 2004-05, they built just 130.

Now, with home ownership in freefall and an increasingly large and influential share of the population stuck in the insecure and under-regulated private rental sector, council housing is starting to look pretty good once again. Even on the right, there are many who now accept that a key part of boosting housing supply will be for councils to get back in the building game. The problem is, those visionary architects and idealistic planners are long gone. So is the money to pay for their work.

It’s nonsense to suggest that the only reason anyone could possibly oppose the Haringey redevelopment is a Momentum membership card

Private developers, though, are short of neither expertise nor capital. What they are short of is the one thing councils generally have: land. So local authorities are increasingly teaming up with the private sector to regenerate parts of their patch. On the east side of Manchester, you’ll find Manchester Life, a partnership between the city council and Abu Dhabi United Group, busily redeveloping Ancoats. On the east side of London, meanwhile, Sadiq Khan’s office is teaming up with housing association and developer L&Q on Barking Riverside, a £500m plan to build an entire new neighbourhood in the wastes beyond Docklands.

That one, though, is relatively uncontroversial: few Londoners will miss the grim industrial estates it’s replacing. Other schemes aren’t so straightforward. Last week, councillors in Tower Hamlets, east London, voted to freeze a planned redevelopment of Chrisp Street Market. They had been petitioned by market traders who, not irrationally, feared being priced out and replaced with a Waitrose.

The big one, though, is the Haringey development vehicle: the north London borough’s planned £2bn joint-venture with Australian developer Lendlease, and the source of perhaps the biggest non-Brexit row of the year so far. The 20-year scheme is intended to create 10,000 new homes, plus a new library, school, health centre and retail facilities.

Facebook Twitter Pinterest Campaigners for improved social housing provision in Haringey, north London. Photograph: Alamy

Even if you haven’t followed the story, you can probably guess where the controversy comes in. Its opponents – the local Liberal Democrats, a large chunk of the local Labour party whose leadership first proposed it, plus two local Labour MPs – worry that it would mean handing a large chunk of the council land to a private developer. More than that, they fear that it would leave current tenants at the mercy of a body outside democratic control. In January, a key proponent of the plan, Haringey’s Labour leader Claire Kober, stood down, claiming she’d been forced out by the bullying of pro-Corbyn activists.

The scheme’s supporters argue that teaming up with a private developer would give the council access to both the capital and expertise required to tackle its housing crisis. The alternative to the HDV, they say, would be to put up with poor quality flats and growing waiting lists, and hope that the next government will be more interested in fixing them than this one.

Fair points, all. On the flipside, though, it’s nonsense to suggest that the only reason anyone could possibly oppose the scheme is a Momentum membership card. “Regeneration” is almost certain to involve demolition – and there is no estate so crumbling or so crime-ridden that there won’t be those who consider it a home worth protecting. To make matters worse, similar regeneration schemes in the past have had their social housing commitments whittled away, and promises to tenants repeatedly broken. Lendlease itself was a partner in the “regeneration” of the Heygate estate, six miles south in Elephant and Castle. That saw 1,194 social-rented flats pulled down, and just 74 built to replace them. The HDV’s supporters say that the alternative is to do nothing – but with that track record, Haringey tenants could be forgiven for thinking that “nothing” might sound preferable.

Decent homes for all… Has the social housing dream died? Read more

And there’s another reason for wariness. Even schemes that do provide new homes for existing tenants are also likely to involve far more private homes (this is, after all, how they fund themselves). That changes the social mix of an area, and in all likelihood the sort of businesses it attracts too. Existing shops and pubs get priced out; chic bars and cafes catering for the new residents move in, with all the craft gin and avocado toast that entails. Gentrification is controversial enough when it’s the result of market forces. When it’s imposed on a community by its elected representatives, it can be actively alienating.

Yet the pressures that led Southwark and Haringey to partner with Lendlease do not seem likely to abate. Local government has lost as much as half of its funding since austerity began in 2010, even as demand for services such as adult social care has soared. Councils have been scrambling to use whatever resources they can to plug the gap – and for many, their most valuable asset by far is their land.

The resulting regeneration schemes will always be controversial: you can’t demolish people’s homes or threaten their livelihoods and expect them not to push back. But there are perhaps things that could be done to drain the poison. Allowing councils to borrow against their housing revenues might make it easier to fund regeneration schemes in-house, without teaming up with the private sector. And tougher rules requiring developers to provide like-for-like replacement – to prevent social rented homes being swapped for “affordable” private housing that is not in any sense affordable – could make it easier to win community approval.

But developers are ultimately motivated by the need to make a profit, not by the need to provide enough housing, and borrowing will only get councils so far. If we really want councils to get building again, the only solution is a big investment of cash. National government is willing to invest in roads and railways without expecting a direct profit, because ministers understand that they are vital infrastructure that will boost the economy as a whole. Why should they not give councils money to build homes on the same basis? We’ve done it before.

• Jonn Elledge is editor of CityMetric