At about 10 o’clock yesterday morning, police officers and bailiffs began to swarm into the 12 Bar Club on Denmark Street in the West End of London to evict its occupants. In December, the management had been told to leave the premises by 14 January; soon after, this most cramped and rickety of places was squatted by a group of battle-hardened activists who called themselves Bohemians 4 Soho.

Each night there were events and performances; this Monday saw a gig by the arena-filling singer Frank Turner. “The spirit of Soho, even the spirit of London, is gradually being whittled away,” one of the protesters told my Guardian colleague Peter Walker, a few weeks before the authorities turned up. “Do we just want another load of chains, Tesco and Starbucks?”

Soho: home to an alliance of artistes, chancers, impresarios and dreamers

St Giles, the patch of urban space that nudges Oxford Street and Tottenham Court Road, is to be redeveloped. In the early 1750s it was the setting for Hogarth’s Gin Lane; two centuries later, it had become an outpost of creative Soho, with Denmark Street recast as Tin Pan Alley (a name borrowed from the New York haunt of songwriters and music publishers), and the alliance of artistes, chancers, impresarios and dreamers we now know as the music industry in full-time residence.

Those whose histories are bound up with the place include David Bowie, the Rolling Stones, and the Sex Pistols, who lived and rehearsed at No 6. But plans for new restaurants, shops and a hotel – and more – mean that this heritage is supposedly under threat, although Camden borough council insists it will vigilantly protect both Denmark Street’s history and what remains of its music-based economy. The landlords and developers in charge are at pains to emphasise the same things – as proved, they say, by their plans for an 800-capacity music venue.

Carnaby Street, 1968. The sartorial centre of bohemian London has already undergone the kind of gentrification Soho now faces. Photograph: Manchester Daily Express/SSPL/Getty

Viewed from a certain angle, then, the 12 Bar protesters might look like rent-a-mob opportunists. There again, to understand that they might have a point, one need only think of those dread modern words “regeneration” and “redevelopment”, and the proposed future of cities in both Britain and the wider world. What is planned for St Giles could conceivably happen in any number of places, and beyond its musical elements, would seem equally appalling – not least, something called Outernet, a complex of buildings and giant screens that will apparently “put the heart and soul back into St Giles” by allowing us to “interact with the brands we love in exciting new ways.”

And in any case, context is all. On the other side of Charing Cross Road, people with a profound attachment to Soho are watching a square-mile of the capital also being transformed at speed. The clip joints and sex workers of yore have largely been pushed out – somewhat controversially, the authorities have now decisively set themselves against Soho’s sex workers.

Meanwhile, independent businesses have been squeezed by rent hikes, and when the iconic burlesque club and music venue Madame JoJo’s was closed and marked for demolition last year, the symbolism was deafening. To sit and soak in the French House on Dean Street, or my own favourite the Blue Posts on Berwick Street – where only the regulars know that the toilets sit behind a door marked “Private”, and to which I once habitually escaped from the West End’s hurly-burly to let long afternoons wear off – is to witness a whole world increasingly hanging on for dear life. “You don’t go into Soho to see films, because Soho is a film,” wrote Colin MacInnes in Absolute Beginners. Now we may be reaching the closing credits.

The arrival of Crossrail – linking east and west London – has triggered this, disrupting Soho’s borders

The arrival of Crossrail – linking east and west London – has triggered this, disrupting Soho’s borders and offering landlords the prospect of hugely increased human traffic; not least the jet-setters who will travel from the new Tottenham Court Road station to Heathrow in 28 minutes. Since 2010, Soho has lost 30,000 square metres of offices and studios, and acquired twice that area of residential space. A one-bed flat on Dean Street goes for somewhere around £3.6m; if you want to live the dream with a family, a three-bed on Greek Street will cost in excess of £4.5m.

This all blurs out into the bigger story of London’s social revolution. It is of a piece with such social-cleansing sagas as the tale of the New Era estate in Hoxton, and the government’s attempted clearing of inner London via the benefit cap – which might seem light years away from the fate of music venues and fashion start-ups, but also reflects planning and politics being blankly aligned in the interests of “business”, and a systemic push towards profit at any cost. This week, there also came news that 25% of London’s LGBT venues have closed since the recession, something equally attributable to the advent of luxury city living and soaring rents. A case in point is the Yard, a gay bar in Soho’s Rupert Street now threatened by redevelopment: as the editor of the gay listing magazine QX puts it, it was probably doomed as soon as the owners realised they could “put a chain restaurant and a couple of floors of flats in there and make a lot more money.”

Soho fair, 1955: Save Soho, co-fronted by Stephen Fry and Benedict Cumberbatch, wants English Heritage to step in and honour Soho as a conservation area. Photograph: Fred Ramage/Getty

Money, money, money. The city London ought to be – turbulent, chaotic, fantastically creative, brimming with a sense of freedom – is under threat. The capital is in danger of hitting a cultural dead-end. Much the same thing has happened in Manhattan; you also get a sense of it in Berlin – though the absence of a financial centre in that city means bohemia is not nearly as squeezed – and Paris. It may yet amount to the evolution of the archetypal western city, away from the idea of a thriving metropolis and refuge-cum-outlet for talent that might founder elsewhere, towards something closed-off, and devoted largely to the most cold and clinical of human activities.

Since November, a group called Save Soho, co-fronted by Stephen Fry and Benedict Cumberbatch, has been pushing for something to change. Its founders want English Heritage to step in and honour Soho as a conservation area; they also want its warren of streets declared a Special Policy Area, an instrument already used to protect the tailoring trade in Savile Row and the art business in St James’s. The group’s co-founder, a musician called Tim Arnold, tells me that he is in conversations with the Greater London Authority; he has raised the latter proposal, only to be told Soho is “too diverse”. His bafflement is obvious. “So they’re telling me that what should be protected amounts to the reason it can’t be protected,” he says.

From the 1960s onwards, the legend of Swinging London, which still partly defines the way the city is seen, was traceable to the coming-together of working-class talent and loose-living bohemia – precisely the elements that are now in danger of being chased out of the centre of central London altogether. From the mods, through the punks and on to the New Romantics and creators of what was eventually called Cool Britannia, these people pioneered the subcultures that ensured so many of us were gripped by the London-obsessed mentality Julie Burchill memorably called capitalism. They also ensured that politicians would end up paying tribute to British fashion, music and drama, and then counting the tax take.

Ironically enough, their story also sits at the heart of why people who have more money than sense are buying up the most romanticised areas of town. But at this rate, the new arrivals will emerge from their urban condos, and realise they have been chasing ghosts.