ANAHEIM, California -- Five Ku Klux Klan members who were arrested following a vicious brawl in Anaheim were released because evidence shows they acted in self-defense, police said.

Seven people who remained in custody were seen beating, stomping and attacking the Klansmen with wooden posts, Sgt. Daron Wyatt said Sunday.

A police statement said the clash, which erupted after six Klan members arrived at a park Saturday for a planned anti-immigration rally, was started by a larger group of 10 to 20 counter-protesters who had "the intent of perpetrating violence."

Get Breaking News Delivered to Your Inbox

Police said the Klansmen stabbed three counter-protesters with knives and the decorative end of a flag pole.

"Regardless of an individual or groups' beliefs or ideologies, they are entitled to live without the fear of physical violence and have the right, under the law, to defend themselves when attacked," the statement said.

Mayhem ensued Saturday as soon as the Klan members pulled up in a black SUV for an anti-immigration rally they had advertised in advance and pulled out signs saying "White Lives Matter."

Dozens of protesters swarmed in and someone smashed a window. The SUV then sped away, leaving three Klansmen dressed in black shirts decorated with the Klan cross and Confederate flag patches outnumbered.

"(The counter-protesters) were so angry, they would have torn these folks limb from limb," said Brian Levin, who directs the Center for the Study of Hate and Extremism at California State University, San Bernardino. "I was afraid for their lives."

Levin, who went to Pearson Park expecting to record the rally for research, found himself protecting the Klansmen until police could intervene. On a video Levin shot and posted to Twitter, he later asked one of them, "How do you feel that a Jewish person helped save your life today?"

"I thank you. I thank you," said the Klansman, waving away the question with his blood-spattered arm. "I would have saved a colored man's life," he added.

Much of the clash was captured on video and posted online. In one, a man cries "I got stabbed," lifting his T-shirt to show a wound to his stomach. A fire hydrant where the man briefly sat was covered in blood.

Police told CBS Los Angeles they had determined that one Klan member was responsible for all three stabbings. They said he acted in self-defense.

Despite the change in tone from police, community activist Rev. James Stern told the station he is calling on the U.S. and California attorneys general to prosecute the Klan.

"We've known for a long history time, everywhere the KKK mobilized, they bring about violence, you don't bring a knife to a peaceful rally," Stern said.

Stern says the KKK -- or any other organized hate group -- should be indicted under the federal Rico Act, which is designed to combat organized crime and has been used in the past against L.A. street gangs.

"If it's not done, it's another form of racist, of justice for one group and not for the other," Stern says.

The reverend told CBS Los Angeles he is planning a news conference in Beverly Hills on Monday asking people of all races to come together against the KKK's hate.

As for Saturday's bloodshed, the DA will still need to review the police findings.

Like many other cities across the United States, Anaheim has a history intertwined with the KKK. What sets the city apart, however, is its decisive backlash after the Klan gained four of five City Council seats in 1924. Those Klansmen were ousted in a recall election after their affiliations with the Klan became public and following a nighttime KKK initiation rally that attracted an estimated 10,000 people to the city park where Saturday's violence erupted.

"The only reason we remember Anaheim for the Klan is because they fought the Klan so hard," said Phil Brigandi, an Orange County historian and author. "The more the Klan came out of the shadows, the more people became aware of it and the opposition grew."

In the near century since then, Anaheim has gone from 95 percent white to become 53 percent Hispanic and 27 percent white, according to data with the U.S. Census Bureau.

"We're a far cry from those terrible times and the Klan is really an anachronism," Levin said. "Anaheim is now a diverse community that is in many ways an amusement and sports capital of California. This is just an example of how a small group of people can tarnish the name of a community."

Rosa Madrigal, who was at the park Sunday with her husband and three children, said she was shocked to even hear about the KKK holding a rally in Anaheim, let alone the violence that ensued.

"I didn't even think it was true when I heard it," said Madrigal, 25. "It's crazy, especially in a park where you take your kids."

When the melee started, Levin said he saw no uniformed officers.

Wyatt said police were there and engaged with people at one end of the fight, and called for additional resources to deploy to the other end. The event stretched along an entire city block, he said.

Police Chief Raul Quezada said his officers were able to respond quickly enough to arrest all but one of the main participants, a counter-protester who remains at-large.

The Klan members were booked for investigation of assault with a deadly weapon before being released. The seven people who remained in custody were booked for assault with a deadly weapon or elder abuse for stomping on a Klan member who's older than 65 years old, Wyatt said.

Though the Klan members were released, prosecutors will review the case and decide whether to file criminal charges, he said.

Chris Barker, who identified himself as the imperial wizard of the Loyal White Knights of the Ku Klux Klan, told The Associated Press by phone from North Carolina that his members were holding a peaceful anti-immigration demonstration and acted in self-defense.

"If we're attacked, we will attack back," said Barker, whose organization lists Pelham, N.C., as its headquarters. Last year, the group drew headlines when it protested the removal of the Confederate flag from the South Carolina Capitol.

Nationwide, the number of active KKK groups increased to 190 in 2015 after falling in 2013 and 2014, according to the Southern Poverty Law Center, which tracks hate groups.

In January 2015, packets containing fliers from the "Loyal White Knights of the Ku Klux Klan" and condemning the Rev. Martin Luther King Jr. were left in the driveways of about 40 homes in Santa Ana, about 8 miles south of Anaheim.