It's a logical evolution for my home town. The media workers who'd been pushed to the edges of Penge were bound to question whether there was something better than this at some point. Realising that, for the amount of money they were spending on a bedsit in Broadway Market, they could probably afford two houses in Montpelier, the exodus began. Encouraged by Buzzfeed listicles, and enticed by rumours of thriving creative industries, casual drug use and good transport links to Glastonbury, Bristol became the promised land.

Every time I return to Bristol it feels like I've dragged more of London back with me. Pop-up bars in shipping containers are spreading like moss. The network of underpasses in town, which used to be haunted by a piss-stench so acrid it would make your eyes water, is now home to a taco stand. The graffiti's still there, only now it's worth more than the buildings it's sprayed on.

A few weeks ago a friend sent me a picture of a sticker that had been slapped against the side of a black-metal bin. It read "Make Bristol Shit Again", which struck me as a pretty good summation of the mystical something that is currently under threat. One of Bristol's crowning virtues always has been its shittiness. Its naffness. The unpolished, unwatched, natural lie of the land that has birthed trip-hop and drum 'n' bass. Crusties, jungletek, muddy cider, poi and sub-Banksy – the lazy cliches my London friends love to rinse – are all part of the crappy milieu. The grain of an earnestly unprofessional, improvisational space is in danger of being sanded away.

Yet, now what's happening is what always happens. The bohemian settlers and marketing execs have brought with them the developers, the shiny-facade billboard-raisers, the sky-high glass apartment builders and shopping precinct marble-pavers. Bristol is losing its edge. My trainers slide over pavements they used to scuff. Everywhere feels clean, neat – that touch less real.

Hamilton House is the largest functioning building in the area – a one-time office block that is now run by social enterprise Coexist. More than 200 organisations, businesses and artists operate inside: a community cooking school, a wellbeing space for therapy and massage, a bike project which repairs unwanted bikes and gives them to underprivileged people. Over 2,000 people work there, and it's estimated that over half a million people a year come through the doors, many for the bar and cafe on the bottom floor.

Stokes Croft sits on the cusp of the town centre. If anywhere is the centre of shit Bristol, it's here: harem pants and dreadlocks, left-wing graffiti, dubstep nights. The Guardian described it as having a "certain urban oomph" back in 2012, which tells you all you need to know.

It's true that Bristol needs housing – the city's population is projected to surpass half a million by the year 2029. Yet, as in London, all too often "affordable housing" targets in Bristol have been missed; of the past 36 large developments approved by the council, only 11 met the minimum requirement of 30 percent affordable homes as set out in the council's housing policy. Even then, for housing to be defined as affordable it must be 80 percent of the market value, which, in a city whose property prices are rising faster than London's, still leaves them far out of reach for most. Then there's the cultural cost of housing.

When I meet with Sean, an architect who has a studio in Hamilton House, he speaks of the upset the sudden seeming volte-face has caused. "There's been no consultation, no feasibility, no communication," he explains, shaking his head. "Hundreds of businesses, thousands of jobs, tens of millions of pounds for the local area. All of that is at risk now, and nobody has given a good reason."

Over the past few months it has been the focus of anxieties about the changing face of the city. When Coexist's 12-month lease came to end this summer, their landlords, Connolly and Callaghan, served them with a vacant possession notice – essentially notifying them that the building had to be in a fit state to be occupied by the owners at any time. Since then, the future of the entire project has seemed uncertain. Negotiations about renewing the lease have grown tense, Coexist have had a bid rejected and word has spread of Connolly and Callaghan's intentions to turn the part of the building into residential flats. "SAVE HAMILTON HOUSE" posters now block out the windows. Residents fear the worst.

"We need a lot of housing, nobody is arguing with that," Sean continues, "but there's a danger if we're not careful that we'll have houses, but there won't be a Bristol here. It'll be a suburb." If Hamilton House is turned into flats, then Stokes Croft – Bristol's cultural quarter – will be left economically and creatively rudderless. For a part of the city sustained and safe-guarded by the community inside the building, it's hard to envisage a particularly bright future for the area without it.

Additionally, they stated that they have always intended to develop some residential spaces on the site, and doing so will secure its future and that of the creative community hub. Andrew Baker, head of social enterprise at the company, was unequivocal: "Hamilton House is safe. We have put together a plan that will secure its future, and that of the creative community hub. C&C will not make any profit out of this plan as all money will be ring-fenced in a separate company for capital improvements to Hamilton House. Hamilton House is not going to be turned into expensive flats. It is not going to be sold to a developer for the highest price. The occupiers will not have to move out of the building, and will not have changes made to their rents."

C&C appear committed to protecting the project, but it's easy to see why its occupants remain scared – if not of the landlords themselves, then simply of recent history. All around, the city is changing. Bristol has already lost a clutch of studio spaces and workshops, while others have downsized. Next to Stokes Croft, neighbouring St Paul's – a historically majority Afro-Caribbean area in a city recently proven the most racially unequal in the UK – is currently a hotbed of "sold" signs and diggers, as developers have grown wise to its prime location next to a motorway, town centre and hipster quarter. The warning signs are hard to ignore. As noble as intentions might be, who ever heard of progressive regeneration?

Well, it's not as ridiculous as it sounds. If Bristol is currently passing through a moment of reckoning, then it is happening at a unique time. The logic of the market and the politics of austerity are being challenged in the mainstream. The ugliness of property development has never felt more pronounced. If anywhere has the experience, the political will and the inventiveness to upset the conventional narrative, it's here.

Labour Councillor Nicola Beech has recently become a cabinet member for Spatial Planning and City Design in Bristol. When we speak on the phone she admits that the challenges facing Bristol are big, compounded by being an attractive city with an affordable living problem. Yet she also speaks encouragingly of the role the Labour-dominated council want to play in the future, as well as the influence communities can have over long-term plans. "There are things that can happen in and around Stoke's Croft, like community development planning... that take a holistic view of how a city should evolve. These things by their nature are slow moving – but it's important for communities to know that if they would like to have more control over what is happening then there are lots of tools."