(CNN) Charlottesville, known as a progressive college town, is bracing for another white nationalist rally scheduled for Saturday.

Police anticipate as many as 2,000 to 6,000 people , including counterprotesters, to converge in the Virginia city, in an event that could be "the largest hate-gathering of its kind in decades in the United States," as described by the Southern Poverty Law Center.

On Friday night, white nationalists gathered at the University of Virginia and marched holding torches throughout the campus.

After seeing images of the march, Charlottesville Mayor Mike Signer released a statement referring to Friday's rally as a "cowardly parade of hatred, bigotry, racism, and intolerance march down the lawns of the architect of our Bill of Rights."

"Everyone has a right under the First Amendment to express their opinion peaceably, so here's mine: not only as the Mayor of Charlottesville, but as a UVA faculty member and alumnus, I am beyond disgusted by this unsanctioned and despicable display of visual intimidation on a college campus," he added.

So how did Charlottesville, which previously has been of little interest to white nationalists, become the rallying ground for what appears to be the third such event in the city this year?

Saturday's "Unite the Right" event would bring white nationalists, neo-Confederates and alt-right activists to the statue of Confederate Gen. Robert E. Lee at Emancipation Park, which had recently been renamed from Lee Park.

The same park was the site of a May demonstration in which a group carried torches led by white nationalist Richard Spencer around the Lee statue to protest its removal.

A group of torch-wielding protesters gathered in a Charlottesville park recently to protest the planned removal of a statue Gen. Robert E. Lee.

In July, a Ku Klux Klan rally in Charlottesville drew about 50 Klan members and supporters who gathered around a statue of Confederate Lt. Gen. Thomas "Stonewall" Jackson.

The historic city, home of Thomas Jefferson and the University of Virginia, has become the latest Southern battleground over the contested removal of Confederate monuments.

Why Charlottesville?

Like many cities, Charlottesville has grappled with what to do with its Confederate monuments -- symbols of the Civil War that represent heritage to some, hate to others.

In April, the city council voted to remove the bronze statue of Lee, CNN affiliate WVIR reported . It was one of several steps Charlottesville is taking to reduce its Confederate monuments. The removal is on hold pending litigation.

The city became an ideal protest site for white nationalists.

"This entire community is a very far left community that has absorbed these cultural Marxist principles advocated in college towns across the country, about blaming white people for everything," said Jason Kessler, who organized the "Unite the Right" rally.

Kessler, a Charlottesville resident, blames what he calls "the anti-white hatred that's coming out of the city."

Although other cities such as New Orleans, St. Louis and Austin have contended with the Confederate monuments issue, it is Charlottesville where the rallies are happening.

"Charlottesville has kind of been put on the map recently," Mayor Mike Signer said last month. "We want to change the narrative by telling the true story of race through public spaces. That has made us a target for groups that hate that change and want to stay in the past, but we will not be intimidated."

Lasting impact of May's torch protest

JUST WATCHED Spencer: 'Make white privilege great again' Replay More Videos ... MUST WATCH Spencer: 'Make white privilege great again' 01:08

Outraged by the decision to remove the Lee statue, protesters in May converged around the monument carrying torches led by Spencer, who had returned to his college town. He is a graduate of the University of Virginia and lives in the state.

The gathering was swiftly condemned by city leaders, including Signer, who said it was "either profoundly ignorant or was designed to instill fear in our minority populations in a way that harkens back to the days of the KKK."

But the fiery imagery from that event had a searing effect.

Kessler called it a "fantastic first event."

"Obviously that first event kickstarted public interest," he said.

The torch light rally was well-received by extremists, said Heidi Beirich, director of the Intelligence Project at the Southern Poverty Law Center, which tracks hate groups and extremists.

"They love what they consider to be a powerful image. A lot of people pointed out that it looked like Klan harassing black people. They didn't have a problem with that."

"It's partly because that went so well, that's why they keep returning to Charlottesville," Beirich added.

KKK rally in Charlottesville

Two months later, the Ku Klux Klan held a rally at another Charlottesville park, which had been renamed Justice Park from Stonewall Jackson Park.

Clad in Klan robes and carrying Confederate flags, members of the North Carolina-based Loyal White Knights of the Ku Klux Klan marched to the park and chanted "white power."

They were met with about 1,000 counterprotesters. The local chapter of the NAACP organized a counter-rally. Shouting "racists go home," the crowd drowned out the Klansmen's chants.

Officers clash with counterprotesters who turned out in reaction to a KKK rally.

Kessler: 'We're going to have bigger events'

The previous two rallies were "seen as an affront to the liberal population of this city and of the country," Kessler said.

The events "reopened the wound. It stuck a finger on the raw nerve of this population that has already failed to get over the election of Donald Trump," he said.

Charlottesville, which has a population of 47,000, had 80% of its voters choose Hillary Clinton last year.

The city has not been known as a hotspot for white nationalism. There aren't historical precedents to the current white nationalist gatherings in Charlottesville, local historian Rick Britton said in an email.

And it may not be over after Saturday if Kessler gets his way.

"We're going to have bigger and bigger events in Charlottesville," he said.

While Charlottesville was once named America's happiest city in 2014 by the National Bureau of Economic Research, some residents aren't very happy about what's happening. Many are tired of the white nationalist rallies and have demanded the city revoke their permit.

In July, more that 200 residents packed City Hall demanding that the city cancel Saturday's rally, CNN affiliate WVIR reported.

"I can very easily see this just eroding and erupting throughout the streets of Charlottesville and putting our citizens in danger," said one speaker, Don Gathers.

"A five-hour block with over 400 angry folks coming in spewing just hatred and vitriol can truly be a problem and I don't know how you can control that," he said.

Meanwhile, some businesses have declared their stores as safe spaces and posted signs of support for equality and diversity ahead of the weekend's event, reported WVIR.