Auburn, Alabama (CNN) Three people were arrested Tuesday at Auburn University amid mostly peaceful protests over the appearance of white nationalist Richard Spencer, whose speaking events have sparked an outcry at other schools.

A crowd of several hundred had gathered by early evening at the public university in eastern Alabama, monitored by dozens of police officers with police dogs. Student groups organized a concert before the speech in what they called a peaceful counter to Spencer's hate speech.

Elsewhere on campus, students told CNN they witnessed a fistfight between a Spencer supporter and a protester that ended with police arresting both men.

Auburn Kelton, 20, makes a play on Donald Trump's slogan as #AuburnUnites prepares to counter Richard Spencer's speech with pizza & music. pic.twitter.com/HfVE5X1AaD

Inside, hundreds of people packed Foy Hall, many of whom appeared too old to be traditional students, as Spencer delivered on his reputation for inflammatory rhetoric.

There were several attempts to shout him down as he extolled the virtues of being white and and called on whites to fight for their rights. People called him names and yelled at him to get to his point.

A protester is arrested after a fight at Auburn University's Foy Hall.

Spencer's supporters occasionally chanted, "Let him speak" when he was interrupted.

Reiterating his key talking points, Spencer denounced diversity as "a way of bringing to an end a nation and a culture" defined by white people.

"There would be no history without us," he said, prompting shouts from the crowd. "The alt-right is really about putting Humpty Dumpty back together again."

Though the auditorium was packed when Spencer began speaking, the crowd thinned considerably toward the end of Spencer's two-hour speech.

Crowd had thinned considerably toward the end of Richard Spencer's speech at #AuburnUniversity. pic.twitter.com/9tZa3OqXmR — Eliott C. McLaughlin (@EliottCNN) April 19, 2017

Hundreds of students and protesters congregated outside Foy Hall. Students in the crowd reported arguments and plenty of flags and protest placards during the speech but no real violence aside from the fight before the Spencer event.

But as several dozen Spencer supporters exited the auditorium following his remarks, the students and protesters spotted them and gave chase. The mob chased them off campus and through downtown.

Free speech or hate speech?

JUST WATCHED Spencer: 'Make white privilege great again' Replay More Videos ... MUST WATCH Spencer: 'Make white privilege great again' 01:08

It was an event that almost did not happen.

Spencer, 38, director of the white nationalist think tank National Policy Institute, has been a target for his radical beliefs. He has advocated a "peaceful ethnic cleansing," where people who are not of European descent voluntarily leave the United States.

Richard Spencer speaking in December 2016 at Texas A&M University.

He said his Auburn appearance was sponsored by AltRight.com, a white nationalist site. He paid $700 to rent the hall and an additional fee for security from the Auburn Police Department, according to The Plainsman , an Auburn student newspaper.

Citing safety concerns, Auburn canceled the event Friday. Then, a federal judge granted Spencer's request for an injunction, effectively ordering Auburn to host his speaking event as originally scheduled.

"While Mr. Spencer's beliefs and message are controversial, Auburn presented no evidence that Mr. Spencer advocates violence," U.S. District Court Judge W. Keith Watkins said in his ruling, noting that peaceable free speech is protected by the Constitution.

'Our goal is to not be the next Berkeley'

Debates over free speech on college campuses have flared up in recent months after appearances by such controversial speakers as Charles Murray and Milo Yiannopoulos. A scheduled speech by Yiannopoulos sparked violent protests in February at the University of California-Berkeley.

The injunction prompted Auburn administrators to call for peace Tuesday amid what it called attempts from uninvited, unaffiliated, off-campus groups to provoke racially divisive, disruptive conflict.

"Auburn University supports the rights and privileges afforded by the First Amendment. However, when the tenets of free speech are overshadowed by threats to the safety of our students, faculty and staff, we have a responsibility to protect our campus," the Provost's Office said in a statement.

Last week, the university issued a statement that said in part, "We strongly deplore his views, which run counter to those of this institution. While his event isn't affiliated with the university, Auburn supports the constitutional right to free speech."

But two days later, the university said it was canceling Spencer's visit "based on legitimate concerns and credible evidence that it will jeopardize the safety of students, faculty, staff and visitors."

The challenges seemed to embolden Spencer. That same day, Spencer told his 56,000 Twitter followers he was flying to Auburn and purchasing safety gear.

"They think they have shut this down, but they haven't," Spencer told The Plainsman last week. "I will give a speech on their campus. It is a public place. I think Auburn University is naive and has totally misunderstood who I am if they think that I am going to politely back out of this. I will be there 100 percent."

'Protesting is the best thing we can do'

Jakobi Bailey, a black sophomore from Pelham, Alabama, said he was initially "kind of mad" when he learned that his school had rented space to someone like Spencer, but once he realized that it was public space that anyone could rent, his anger waned.

"People started talking about it, and protesting is the best thing we can do since we can't run him out of here," the 20-year-old said.

#AuburnUniversity senior Shannon Arthur helped several minority groups organize a protest against Richard Spencer, but after violent threats to those groups & its members, they've decided on a Plan B. A post shared by Eliott C. McLaughlin (@eliottmc) on Apr 18, 2017 at 3:54pm PDT

Shannon Arthur, 22, a senior had helped organize several minority student groups -- including the Black Student Union, NAACP and Hillel, a Jewish group -- to protest Spencer. But the groups and their leaders were targeted, some by name, on social media before the speech so they opted to join a peaceful protest near the school's football stadium.

They informed the Auburn police that they'd be wearing orange ribbons on their wrists so that authorities could differentiate them in the event of violence, she said.

"You saw Berkeley," Arthur said. "Seeing people really get hurt and bloody is something we don't want for our school population."

Several student groups sought to counter the tension over Spencer's visit by holding an outdoor concert under the hashtag #AuburnUnites.

#AuburnUniversity grad Cassidy Kulhanek, 23, explains why #AuburnUnites wanted to hold a concert during Richard Spencer's appearance. pic.twitter.com/ctElsURbpU — Eliott C. McLaughlin (@EliottCNN) April 18, 2017

"I feels like it has less chance of going awry than marching down the street," said Cassidy Kulhanek, a 23-year-old Auburn graduate.

In the shadow of mammoth Jordan-Hare Stadium, bands played in the university green space, which was encompassed in metal police barricades.

Students handed out pizza and shirts with the group's hashtag, which had been made on site with a stencil and bleach.

"We're trying to block hate speech with music and positive ideas and unity," said organizer Jakob Geiger, 19, a sophomore studying political science and English. "We think this is a more effective message to the outside and to our minority students. Our goal is to not be the next Berkeley."