Demonstrators gather outside office of one of the main property owners in the area calling for them to stand up to the police

A new crackdown on sex workers in London's historic red light district risks putting women in danger and changing the character of Soho, sex workers and activists said on Wednesday.

Sex workers from three flats in Soho were evicted late on Tuesday after police issued enforcement notices on landlords warning they could be prosecuted if they were found to be allowing "immoral activities".

Scotland Yard said police were working with Westminster council to tackle "all crime" in Soho.

On Wednesday sex workers and activists – some wearing sequinned masks and holding banners with slogans such as "I'd rather sell my body in Soho than sell my mind to a corporation" – gathered outside the offices of Soho Estates, one of the main property owners in the area , and called for them to "stand up" to police.

"Soho has always been one of the safest places in the country for women to work – it is transparent, well established and there has always been the support of the community," said Niki Adams from the English Collective of Prostitutes (ECP), who organised the protest. In the 19 "walk up" flats operating in Soho – where clients can come in off the street – women worked on their own, but were accompanied by a "maid" who helped with cleaning and also provided sex workers with more security in return for tips, she said. "If this continues I fear more women will choose to work on their own or on the street which will put them in much more danger." The actor Rupert Everett, who is making a documentary on prostitution, attended the protest, the ECP said.

In a statement Scotland Yard said police were working with the council to tackle crime which "includes using a range of tactics to tackle historic crime problems, but also ensures sex workers who are vulnerable or need assistance receive it and can access support networks and services". It went on: "Properties being used as a brothel do break the law and police will take action where appropriate, as we have done over a number of years."

A protest in Soho by sex workers and representatives from the English Collective of Prostitutes. Photograph: David Levene for the Guardian

It is not illegal to sell sex in Britain, but activities associated with prostitution – such as operating a brothel, soliciting and kerb-crawling – are outlawed. The ECP argue that by putting pressure on landlords to evict tenants, police are bypassing the need to prove that the flats are being used by more than one sex worker, and therefore operating as a brothel. Previous attempts by police to use brothel closure orders to close flats have been unsuccessful.

Paula, a 21-year-old Romanian sex worker who had worked in several different flats in Soho since she was a teenager, said police attitude towards sex workers in Soho had changed hugely in the past 12 months. "[Police] start raiding, saying lies, accusing us of things we don't do and then giving us a letter saying we are going to be deported or the flat is going to be shut down," she said. She said she had been attacked by punters in the past, but having a maid made her feel safer. "Sometimes [clients] can look very nice, very polite and very gentlemanly but if you say a wrong word it can turn out really badly," she said. "It happened to me about two times, everything was nice and I end up with a punch in my face for no reason. You need to be very careful who you let in, that's why it's very important to have a maid. If you scream she is the first one to walk into the bedroom and save you."

The raids had made her less likely to report violence to police, she added. "Most of the time when we call them and we've got trouble, they let the bad guys walk away and accuse us of prostitution."

A 52-year-old maid from the evicted flat in Romily street, who herself worked as a sex worker before becoming a maid, said: "The police have completely changed, they've dragged customers naked out of bed, searched the flat – the girls are scared, they are not criminals. As far as I have understood one girl and one maid is not illegal, it's not a brothel."

John James, the managing director of Soho Estates faced the loud-hailer wielding women outside the firm's offices, assuring them that he "had no problem with this type of work" but had no choice but to inform the leaseholder of the flats that they could lose their lease if they were to allow "immoral activities", after Soho Estates was issued with an enforcement notice by police.

"I have no angst with these girls, I was surprised last week by an enforcement notice from the police and we have to take that seriously," he told the Guardian. Asked what he thought of the police action he said: "I don't understand it, I really don't – it is part of Soho. Police are putting us in the firing line and turning us into the villains."

Members of the local Soho Society said the character of Soho was under threat from developers and risked becoming homogenous. A new Soho Estates development, Walker's Court, will see the company open offices, a restaurant and cabaret theatre – it will also mean the closure of several more "walk-up" flats and a sex shop. "They are chipping away at Soho, there is a community here and these women are part of it," said member Juliet Peston.

James said as a major landowner in Soho the company had a right to redevelop its property. "That will unfortunately mean some of the closure of the occupiers, but that is what development means," he said. "We believe it is to the benefit of the area – it's too short-sighted to say that any change is bad."