“I know how I am going to be portrayed,” says Barrie Short, rubbing his face glumly. The landlord of the Old Farmhouse pub is not looking forward to watching Channel 4’s controversial new documentary, Immigration Street. With his bulldog tattoo and Fred Perry shirts, he fears being stereotyped as “the skinhead bastard at the end of the road”. In fact, the 43-year-old is so pessimistic about the scenes the crew shot at the pub’s St George’s Day celebrations that he won’t tell me the name of the children’s charity he was raising money for, in case it is tainted by association.

And he is not alone in his anxiety. On Derby Road, Southampton, there has been a groundswell of outrage since filming began on this sequel of sorts to Benefits Street, a series that attracted more than five million viewers and 1,800 complaints to Ofcom (although it was cleared of breaching broadcasting rules). Residents fear the same fate as the inhabitants of James Turner Street in Birmingham, who critics claimed the makers of Benefits Street had exploited, ridiculed and demonised, portraying them as lazy, “feckless scroungers”. Some neighbours claimed their community had been misrepresented or that they had been tricked into taking part.

After the programme aired, Birmingham politicians complained the road was inundated with “TV tourists” who egged homes, hurled insults and defaced the street sign. Ofcom ruled that the programme-makers had taken “due care” in filming the street’s children, but the local primary school said their pupils were being bullied and the whole community had been stigmatised.

Facebook Twitter Pinterest A protest against Immigration Street. Photograph: Mark Kerrison/Demotix/Corbis

The company behind both shows, Love Productions, promises the forthcoming series will be an “honest look at how immigration has changed one street”. But this has not stopped an alliance of anti-racist groups, local politicians and residents campaigning against it since filming began in April last year. Now, after protests, petitions and a poster campaign, a number of residents have pulled out of filming – and Channel 4 has announced it will not air the full six episode series it originally planned, but will broadcast a shorter run instead.

Walking down Derby Road, it is not hard to see why the programme-makers chose it. The handful of shops dotting the shabby but bustling residential street are testament to successive waves of immigration to the UK – from the sari and salwar kameezes of the south Asian stores to the Somali and north African cafes. The small supermarkets are stocked with Jamaican food, and the shopkeepers have learned how to greet customers in Lithuanian and Romanian. With children scampering in and out to buy sweets as their parents chat, it is not just the diversity but the friendliness that is striking – and it is easy to see why residents fear seeing their vibrant home being misrepresented.

Shopkeeper Nadia, who was born and brought up on the street, says this unified spirit has been hard won. The road was once a byword for illegal drinking dens and prostitution – until local families, fearful for the safety of their children, took a stand. “My father and his neighbours would put sofas on the street or stand in the road all night so the men were too embarrassed to come,” she recalls.

Facebook Twitter Pinterest A shop selling saris on Derby Road. Photograph: Laura Dale/BNPS

Anxious to protect its new respectability, the 33-year-old has backed out of appearing in the documentary, despite originally agreeing to be filmed. “Every area has good and bad, but people here work hard and are friendly. My neighbours are Sikh, and they always keep an eye out for me and ask me how I am. That’s the kind of neighbourhood we have.”

Satvir Kaur, a councillor who grew up in one of the adjacent roads, is one of many who object to the name Immigration Street, pointing out that many of the residents were born and brought up in the UK. “I am second- or third-generation, as are most people living on Derby Road, so I would not class myself as an immigrant,” she says. “At what point will someone like me be classed as British? Derby Road is like every other street, apart from the fact people may look and dress differently.”

Kaur says the programme-makers have already damaged the character of the area. “We have gone from growing up in an open community to feeling like animals in a zoo, so people have become suspicious of everyone, and particularly the media world. Why and how can Channel 4 justify debating immigration at the cost of ripping up a community?”

Facebook Twitter Pinterest Store owner Raj Banga. Photograph: Laura Dale/BNPS

The police, meanwhile, are worried the programme could make Derby Road a target for far-right groups, and have asked to view the series before it is aired. Neighbours, too, fear the series could spark racial tension. A customer in a Derby Road cafe worries the programme is being made “just to show white people that the foreigners are here to take their jobs – it’s making problems”. Local MP Alan Whitehead says 95% of the local people he has heard from are against the filming.

Even those taking part are nervous. In Banga Stores, a supermarket and off-licence at the end of Derby Road, the irrepressible Raj – surely a contender for Immigration Street’s breakout star – is holding court. Teasing and joking with a diverse stream of customers, he greets them all with outlandish nicknames, or at least a cheery “bruh”. But he grows serious when he contemplates the repercussions for the community. “I think when the show comes out it will all blow up,” he says, pointing out that when people hear what their neighbours think about different cultures, it could lead to trouble.

Kieran Smith, the creative director for the programme at Love Productions, says allegations that the documentary will be racist or exploitative, or that they are somehow pitting neighbours against each other, are just not true. And he says residents such as Short – who told the team that if they misrepresented Derby Road, he would “kick our arse” – have little to worry about. “The model was to take a street that is ethnically diverse, in an area transformed by immigration and which continues to be a place that immigrants come now, and film it for about six to nine months. It’s very simple … and people can choose to take part or not.”

He says that local people had felt pressured into pulling out. “There was an unpleasant feeling of low-level coercion.”

Facebook Twitter Pinterest Derby Road. Photograph: Laura Dale/BNPS

A bigger worry, he says, was the activity of a criminal gang who targeted his crew. While news reports stated the team had been pelted with flour and eggs by locals, the situation, he suggests, was much more serious: one local man who was being filmed was so badly beaten up he had to be hospitalised. Other incidents included tyres being slashed, threats to throw acid at the team, and film crews being pelted with rocks and stones. Security was hired to protect them, but ultimately filming had to be cut short.

James Fulton, a police commander for Southampton, says they are investigating the allegation of assault on the participant, but cannot investigate any other incidents because “Channel 4 has declined to provide any statements that would provide us with the evidence necessary to make followup enquiries against suspects”. When asked about criminal gangs in the area, the police say Derby Road is a low-crime area and they are not aware of any gangs operating there – something Kaur, the cabinet member for communities in Southampton, confirms.

Facebook Twitter Pinterest Barry Short, landlord of the Old Farmhouse pub. Photograph: Laura Dale/BNPS

Few of the people on Derby Road think the street is a perfect picture of harmony – there are moans about rubbish on the street, references to raids for illegal immigrants and the odd comment linking new immigrants with crime. A reported gang rape, for which six Derby Road men have been charged, has also created understandable concern about safety. But everyone I speak to insists this is far from the whole picture, and fears that the street’s positive side will be ignored in the bid for ratings. Which, according to Short, would be a disaster.

“When I was first looking at this place, people said: ‘Don’t move to the jungle,’ but I like places like this. There are so many different communities, but they all come together. It’s unsaid, but everyone looks after each other.

“If they showed it as it is, it’s an amazing place,” he says. “Coming here was the best move I ever made.”