An escalating confrontation between police and residents in a Madrid neighbourhood who are incensed by the treatment of immigrants has seen locals driving police patrols from their streets.

Inspired by the "indignant" protest movement that camped out in Madrid's Puerta del Sol square in May, residents in the multicultural neighbourhood of Lavapiés have reacted to recent attempts to detain immigrants by jeering and chanting at police.

Police have twice had to call for reinforcements in riot gear so that arresting officers can leave safely.

About 200 protesters gathered after an immigrant texted to say that police were asking African residents for identity documents. In another incident a crowd formed to drive police out after they arrested a Senegalese immigrant in the metro. Police said they had been chasing a drug trafficker on one occasion and, on the other, had been called to arrest a man who failed to pay for his metro ticket.

But neighbours say police frequently hassle immigrants for no reason at all. "This is a reaction to the police's habit of illegally rounding up immigrants and checking people's papers on grounds of race," said Olmo Calvo, a photographer at Diagonal, a leftwing bi-monthly newspaper based in Lavapiés.

Spain's interior ministry denied ordering police to stop Africans and Latin Americans, or launch special immigrant roundups. But Lavapiés residents say that is not true. "Police controls have increased. Sometimes you get three or four police cars pulling up and they start stopping anyone who looks like an immigrant – mostly Africans or Latin Americans," said 32-year-old As, one of a group of Senegalese men gathered outside the Baobab restaurant. "Even if you have proper residency papers, like me, they start asking whether you have a job and things like that, which have nothing to do with them," he added.

Lavapiés, where a web of narrow streets comes off the sloping Ribera de Curtidores that hosts Madrid's famous Rastro flea market, has long been a melting pot for immigration and a magnet for bohemians.

"The whole world is here," said Mercedes Solanilla, a 50-year-old doctor, who moved in five years ago. "That is what is so great about it."

Shopfront mosques and Western Union franchises vie for space with Chinese or Pakistani wholesale textile shops. Restaurants offering food from around the globe compete with traditional Spanish bars and newer, trendier art cafes. The "indignant" movement, which spread its popular assemblies into Madrid's barrios after raising camp from the Puerta del Sol last month, is firmly rooted here.

Some 150 people gathered in the Plaza Cabestreros over the weekend to debate, among other things, police behaviour. "For months we have looked on with horror and a sense of impotence at the police's racist roundups and ID checks," a working group on immigration reported. "We have decided to say, 'That's enough'."

"We've been tracking the police controls of immigrants for a long time," said Calvo at Diagonal. "Sometimes they simply pack up and go if they see us arrive with our cameras. Other times they threaten us, make us wipe our memory cards clean or think up reasons to arrest us." Fifty Madrid press photographers protested against police harassment in November after Diagonal photographer Edu Leon was arrested for a third time – and, for the third time, declared innocent by a judge.

The Lavapiés protests mimic those of another "indignant" activity: blocking access to houses being repossessed by banks. Police unions have complained that the neighbourhood threatens to become a no-go zone. "Officers have been seriously worried about whether they were in danger," Alfredo Perdiguero, a spokesman for the Federal Police Union told El País. "How on earth can we work if we can't demand ID cards from or detain people dealing in drugs or with criminal records?"

Some residents agree that police have a job to do. "You must bear in mind that there are drug dealers in the barrio too," said As.