The removal of a federal judge from a long-running case against the New York police department's stop-and-frisk practices appeared to be "gratuitous and deeply flawed", according to a legal challenge filed on Monday by civil rights attorneys.

Attorneys from the Center of Constitutional Rights say that a ruling by a three-judge second circuit panel to remove US district judge Shira Scheindlin was "unprecedented and unwarranted" and have asked the entire court to reconsider it.

It is the second legal appeal in a week against the panel's unusual ruling removing Scheindlin from the case. A motion filed on Wednesday by Scheindlin's lawyers argued that the panel acted with unseemly haste, that they did so on the basis of a skewed reading of the evidence, and that they violated a rule that gives judges accused of misconduct notice and the opportunity to defend themselves.

The three-judge panel removed the judge on 31 October. It also stayed her earlier order requiring the New York police department to alter its stop-and-frisk policies. The panel said she misapplied a related case ruling that allowed her to take stop-and-frisk cases, and that she gave media interviews during the trial, thus calling her impartiality into question.

Baher Azmy, CCR legal director, said: “The removal of judge Scheindlin was done by a perfect storm of procedural irregularity. The appellate panel cast aspersions upon the professional conduct of one of the most respected members of the federal judiciary – and thus inappropriately cast doubt on her legal rulings – while itself taking an unprecedented step that no party requested, of which no party was notified, and without providing the parties an opportunity to be heard.”

He said: “The facts conclusively show that the district judge engaged in no unethical conduct whatsoever and that her decision finding the city liable for widespread constitutional violations and racial profiling is based on overwhelming evidence presented at trial.”

The filing argues that the plaintiffs in the case – four African Americans and "hundreds of thousands of New Yorkers they represent" – have already suffered substantial prejudice by the reassignment of a judge unfamiliar with the complex case.

"It is not apparent that the panel even considered this potential prejudice" the filing said. The decision was taken without "even the most basic procedures notice to the parties or the judge, without any request or complaint from the parties and long after the city waived an opportunity to seek removal, it said.

The new action follows a late appeal on Friday night by New York City, asking for a federal appeals court to vacate orders by Scheindlin requiring police to change its stop-and-frisk practices which critics say unfairly targets minorities.

Scheindlin ruled in August that the city violated the civil rights of tens of thousands of black and Hispanic people by disproportionately stopping, questioning and sometimes frisking them. She assigned a monitor to help the NYPD undergo training and change its practice with regard to the tactic.

Jonathan Moore, an attorney with Beldock Levine and Hoffman LLP and co-counsel in the case, said: “This unprecedented action by the second circuit should not detract from the central issue that has been resolved after six years of litigation: overwhelming evidence has conclusively proven what hundreds of thousands of New Yorkers already know well – that the city’s quota-driven stop-and-frisk program has produced systematic constitutional violations that must be remedied if the NYPD is to win back the trust of our communities.”

Mayor-elect Bill de Blasio, who takes office in January, has said he would drop the city's appeal of Scheindlin's ruling. He has also indicated a possible settlement with those urging reforms, which would eliminate federal oversight.

Recorded stops increased dramatically under Bloomberg's administration to an all-time high in 2011 of 684,330, mostly of black and Hispanic men. A lawsuit was filed in 2004 by four African Americans, who said they were targeted because of their races, and it became a class-action case.