San Francisco police exclusively targeted black residents during undercover drug arrests as part of a pattern of racial profiling, according to a new lawsuit.

Police in the California city, where only 6% of the population is black, worked with federal authorities to arrest 37 people, all of whom were black, for selling small amounts of drugs, the American Civil Liberties Union of Northern California said in a complaint announced on Thursday. During the operations, one undercover officer was caught on camera declining to buy drugs from an Asian woman and waiting to buy from a black woman, who was later prosecuted.

The suit focused on San Francisco police collaborations with the US Drug Enforcement Administration and federal prosecutors in 2013 and 2014, but the ACLU has alleged that the discriminatory policing and harassment of black people in the city has continued.

“We’ve seen time and time again how racial bias has infected the San Francisco police department’s ability to administer equal enforcement of the law,” Novella Coleman, an ACLU staff attorney, said in an interview.

The allegations are striking given how rapidly the African American population has declined in San Francisco, a city that has a serious housing crisis amid a booming tech economy. The city and state have also developed a huge marijuana industry while reports over the years have repeatedly shown that black residents in San Francisco are disproportionately targeted for marijuana and other drug arrests.

In one video included in the complaint, an officer involved in the undercover operations was caught saying “fucking BMs”, an apparent reference to the police code for “black males”, while surveillance footage zoomed in on a group of black people. An undercover officer also approached a black woman to buy drugs and told the woman that he had purposefully avoided buying from an “Asian chick”, the suit said.

In the Tenderloin, the neighborhood where the arrests occurred, surveys have suggested that only half of the people who sell drugs are black, the ACLU said.

A federal public defender representing some of the accused people later argued in court that the police department had singled out black people, leading a district judge to conclude there was “substantial evidence suggestive of racially selective enforcement”.

By then, however, many of the defendants had already pleaded guilty.

The federal prosecutors, who were pursuing charges with one-year mandatory minimum sentences and potentially stiffer sentences, opposed attorneys’ efforts to uncover further evidence about the operations and eventually dismissed the remaining cases.

Some of the plaintiffs’ lives were derailed by the prosecution, said Coleman, adding: “There’s that sense of uncertainty about interacting with law enforcement, and the fear of being in San Francisco and the Tenderloin.”

The ACLU was not aware of any involved officers facing disciplinary action, and one was promoted, she said.

Despite a judge’s assertion of racial bias and the recorded comments of officers, the city of San Francisco has continued to defend its actions. The police department “did not engage in selective enforcement”, John Coté, the spokesman for the city attorney, said in an email Thursday. “The evidence will show that San Francisco police acted in accordance with federal directives.”

The statement also said the police department “prides itself on being one of the most diverse, forward-thinking and transparent law enforcement agencies in the country”.

The police agency in recent years, however, attracted national scrutiny for a scandal involving racist and homophobic text messages sent by officers. A 2016 justice department review also found “institutional bias” against people of color. The city’s police chief was further pushed to resign in 2016 after a high-profile killing of a black woman, which followed numerous controversial killings by police.

The ACLU suit was announced the same day a Los Angeles Times investigation revealed that officers involved in southern California drug enforcement efforts have stopped and searched thousands of Latino drivers without uncovering any drugs.