NEW ORLEANS -- With dozens of demonstrators on both sides of a fierce debate over Southern history looking on from a distance, crews removed a statue early Thursday of Confederate President Jefferson Davis, reports CBS affiliate WWL-TV.

Workers labored under the cover of darkness to remove the statue, wearing bulletproof vests, masks and dark clothes to hide their identities in the wake of threats from thosewho opposed the statue's removal. City officials had refused to give advance public notice of the work because of threats of violence against contractors and workers involved in the effort. The statue was first unveiled in 1911.

As the statue was lifted from its perch on a grassy median along one of the city's main thoroughfares, a cheer went up from some of the dozens of protesters on the scene who have been pushing for the monument's removal. It was then lowered behind trucks encircled around the monument's base and out of view of media gathered on the scene.

Workers prepare to take down the Jefferson Davis statue in New Orleans, Thursday, May 11, 2017. Gerald Herbert / AP

It was the second of four monuments to the Confederacy slated for removal under a controversial city plan.

Get Breaking News Delivered to Your Inbox

On one side of the debate are those who argue the monuments honor Southern heritage and history. Others say the monuments are symbols of racism and white supremacy.

"The fight that we're waging is against the city fathers who have refused to bring about genuine equality and freedom for the black people here," said Malcolm Suber of the activist organization #TakeEmDownNOLA.

New Orleans Mayor Mitch Landrieu, who first urged the City Council to remove the monuments in 2015, praised the effort in an interview on "CBS This Morning."

"These particular monuments were put up by a particular group of people for a particular purpose," Landrieu said. "The purpose was really to whitewash history and to tell a sanitized version of the Confederacy."

Dozens of protesters both supporting the monuments and calling for their removal stayed up overnight to watch the proceedings which happened in early-morning darkness. The opposing groups of demonstrators, separated by barricades, got into some heated shouting matches, WWL says. There was a heavy police presence.

Demonstrators who support keeping Confederate era monuments protest before the Jefferson Davis statue was taken down in New Orleans, Thursday, May 11, 2017. Gerald Herbert / AP

A last-ditch effort to block the removal of a monument to a Confederate general was rejected Wednesday by a Louisiana judge who turned away arguments that the city doesn't own the statue or the land on which it sits.

"This has gone on an inordinate amount of time," Judge Kern Reese said as he outlined reasons for his refusal to grant an injunction protecting a statue of Gen. P.G.T Beauregard. It was a reference to state and federal court battles that delayed removal of the Beauregard monument and the other three for more than a year.

The huge bronze image of Beauregard on horseback sits in the center of a traffic circle at the entrance to New Orleans City Park. Those who don't want it removed argued that it belongs to a park board and, therefore, the city has no authority to remove it.

Reese's rejection of an injunction means the city can remove the statue pending further proceedings in his court. Richard Marksbury, a New Orleans resident and monument supporter, said he may go to an appeal court to block removal.

A statue of Gen. Robert E. Lee is also supposed to be taken down.

A fourth structure, the Liberty Place monument, was removed late last month. It honored whites who battled a biracial Reconstruction-era government in New Orleans.