One of Britain’s biggest community and cultural hubs is facing partial demolition to make way for private apartments under a plan that places dozens of social enterprises, charities and arts organisations at risk.

Large parts of the Hamilton House complex in Bristol’s Stokes Croft area, the spiritual home of the street artist Banksy, will be turned into at least 50 flats with only a fraction of the community uses retained, according to plans unveiled by the building’s multi-millionaire owners.

Opponents to the plans include Grant Marshall, founding member of the group Massive Attack, who said it was “absolutely disgraceful”.

Grant Marshall, also known as Daddy G from Massive Attack: ‘I don’t know what the council is up to. They don’t support the arts.’ Photograph: Adrian Sherratt/The Guardian

It has sparked a bitter row over the direction of gentrification in the city, where property prices have risen faster than any other city, including London, over the last five years. After decades as a neighbourhood many Bristolians avoided because of problems with crime, drug use and rough sleeping, Stokes Croft’s reputation as a hub of creative dissent has seen small businesses starting to thrive and now housing developers are moving in.

“This was the first part of Stokes Croft,” Marshall, also known as Daddy G, told the Guardian. “Everything else followed. I don’t know what the council is up to. They don’t support the arts.”

Hamilton House is widely considered the catalyst for the wider development of the area. The former bank offices are sandwiched between two famous murals: Banksy’s teddy bear throwing a petrol bomb under the strapline “the mild, mild west” and a breakdancing Jesus by Cosmo Sarson. Its operator, Coexist, says that in 2017 500 artists, social enterprises and businesses occupied its spaces and it provided over 1,260 employment opportunities.

A third of the complex has already been emptied and 212 businesses, artists and community groups have been moved out by the building’s owners, Connolly & Callaghan. They paid low rents of just £3 per square foot after they colonised the redundant bank offices from 2008, but the owner is in the process of quadrupling the rent and insists the current operation is not financially sustainable.

The dispute pits two different visions against each other. Coexist believes it has created a place that sustains relationships across different parts of Bristol and creates a sense of belonging and unique Bristol identity, all of which is hard to measure.

Graffiti protesting about the plans for Hamilton House. Photograph: Adrian Sherratt/The Guardian

The owners argue they have subsidised the complex for a decade and that the community-led organisation that now runs it does not have a convincing business plan and is failing to reflect the full social and ethnic diversity of the area.

Groups based at the scruffy but lively complex include those that work with drug addicts, run dance groups for people with learning disabilities and Parkinson’s disease, graphic designers and artists. When the Guardian visited this week, Misfits, a group of adults with learning disabilities, were rehearsing a performance. At the weekly communal lunch for centre users, dozens of diners included a painter, a business coach and a community arts organiser.

“The UK is losing one of its biggest community and cultural bases,” said Danny Balla, a director of Coexist. “It’s not just a Bristol issue. We were a beacon and we have been studied all over the world. We put people with the least social capital, the homeless and the drug users, at the heart of what we did. There is a systemic issue about how we make sure our social spaces are kept in common rather than being taken into the private sector.”

Kabbo Ferdinand, who helps run a dance centre at Hamilton House, said: “It’s astonishing they should be getting rid of something that creates cohesion, especially in the current climate where there are forces to entrench age-old division.”

Connolly & Callaghan says it needs to refurbish the building and argues that it will retain community uses but it needs to be more focused on incubating small businesses.

It bought the complex before the financial crash of 2008 for £2.1m with a plan to turn it into flats. With the credit crunch it started turning it into a community hub. It has turned down a £7.9m offer from a developer of student flats and a £6.5m offer from Coexist. It says it will maintain part of the building as a community hub, with a large part of the rest turned into flats for private rent and possibly some offices.

“It’s not a question of gentrification,” the firm said in a statement. “C&C has subsidised Coexist, to the tune of £500,000 per year, for many years and continues to support it financially. While C&C is committed to the community hub at the site, the building as a whole has to be financially viable.”