Tucked in the FBI’s 2011 gang-intelligence report, between ominous write-ups on cross-border human trafficking and organized crime in the Caribbean, was a short section warning about the growing menace of the Juggalos.

The facepaint-wearing, Faygo-drinking followers of the rap duo Insane Clown Posse were a “loosely-organized hybrid gang,” the FBI said. Though their crimes were “sporadic” and “individualistic,” the report claimed, their “gang-like criminal activity” seemed to be on the rise. Law enforcement should be wary, the FBI said.

On Monday, Insane Clown Posse, better known as ICP, and a group of self-proclaimed Juggalos, who derive their name from an ICP song called “the Juggla,” experienced what could be a fatal setback in a years-long legal battle to shake the FBI’s designation.

The U.S. Court of Appeals for the Sixth Circuit ruled that the FBI’s decision to call the Juggalos a gang wasn’t a “final agency action” — government jargon for an official, legally binding rule — and therefore couldn’t be challenged in court.

The ruling threw out a lawsuit brought by ICP frontmen Violent J and Shaggy 2 Dope in 2014 alleging that the FBI’s findings trampled their constitutional rights and caused a venue to cancel one of their concerts at the request of law enforcement.

They were joined in the case by two Juggalos who claimed that the gang designation had gotten them in trouble with police and two others who said it jeopardized their military careers. All the plaintiffs said they had never knowingly been involved in a criminal gang.

The appeals court found that the Juggalos failed to show that the FBI’s label had resulted in legal consequences. The FBI report was just an annual gang-activity report presented to Congress, the three-judge panel wrote, and did not contain any marching orders for other law enforcement agencies.

“The various reputational and personal harms suffered by Appellants in the present case may be the practical consequences of the Juggalo gang designation,” the court wrote, “but they are not a direct or appreciable legal consequence of the Juggalo gang designation.”