This summer, a series of Confederate monuments were torn down in Maryland, North Carolina, Florida, Texas and Louisiana after the death of a civil rights activist at a far-right protest in Charlottesville. It looks like New York City might be next.

In a city with more than 800 public monuments, four in particular have irked artists and academics, who have signed a public petition. The 500 signatories are advocating for the removal of monuments of Christopher Columbus, James Marion Sims, Theodore Roosevelt and one adjoined honouring Philippe Pétain and Pierre Laval.

“For too long, they have generated harm and offense as expressions of white supremacy,” reads the petition, in a city which “preaches tolerance and equity”.

To the signatories, the Roosevelt monument, which shows the 26th American president with a Native American chief and an African man, glorifies racial hierarchies. “Roosevelt is a very salient symbol for us of white supremacy and it’s often cited as the most hated monument in New York City,” said Andrew Ross, an activist and culture professor at New York University. “It has no place outside of the museum.”

Ross first drafted the petition alongside other members of Decolonize This Place, which organized the Anti-Columbus Day Tours at the American Museum of Natural History.

New York’s Columbus statue, located at Columbus Circle in Manhattan, has come under fire for representing genocide. The statue, however, was a gift from the Italian American community, some of whom adopt Columbus as an icon. To Phil Folgia, counsel for the Italian American legal defense and higher education fund, the removal of the statue would be “an outrage to our community”.

The 70ft-tall column topped by a statue of Christopher Columbus at Columbus Circle in Manhattan. Photograph: Cameron Bloch/AP

Not all Italian Americans feel that way. Tom Angotti, an American Italian professor at Hunter College believes “the genocide of Christopher Columbus should not be revered”, adding that: “We ought to honour instead the Italian immigrants who faced discrimination and fought for economic and racial equality – people like Vito Marcantonio, once a congressman from New York City.”

To Roberto Borrero, a Taíno person who works for the International Indian Treaty Council, he explains: “Italians have many other role models to look up to, so their pride should not supersede my people’s tragedy.”

John Viola, the president of the National Italian American Foundation, said: “In the wake of the lynching of 11 Sicilian Americans in New Orleans, the statue at Columbus circle was gifted to the city of New York by the newly arrived Italian American community. Our community was able to gather small donations from thousands of Italian American families and turn them into that statue – a testament to their belief and confidence in the American dream. That statue is a thank you card to America.”

While New York City has seen several public art controversies – like Richard Serra’s Tilted Arc which caused an uproar for its inconvenience to pedestrians in the 1980s – for the first time the mayor’s office has responded to this issue by hosting public hearings and an online survey for New Yorkers.

“It has hit a high level,” said Gregory Sholette, an artist who signed the petition. “It’s reached a situation that’s now different because people are finally standing to say ‘this not acceptable.’”

According to the signatories, the monument honouring French politicos Philippe Pétain and Pierre Laval, which is part of the Canyon of Heroes on Broadway, needs to be removed, as the duo were historic markers of the Vichy France regime during the second world war and were “Nazi collaborators”.

Alan Singer, a signee and professor at Hofstra University, says a new system needs to be brought in. “A scale needs to be developed to evaluate which monuments are taken down and which place names are changed,” he said. “Criminals, including traitors, should be stripped of honours, as well as advocates of hatred.”

The monument of American physician James Marion Sims at Central Park near 103rd street is especially troubling, as Sims performed experimental gynaecological operations on enslaved African women between 1845 and 1849.

“By honouring Sims, New York City honours the American equivalent of Josef Mengele,” said Singer. “This statue should be removed.”

Could the monuments be replaced? One New York artist, Alicia Grullon, thinks there should be a statue of Kalief Browder, the African American man from the Bronx who was imprisoned on Rikers Island, and whose suicide is believed to be the result of prison abuse.

“He is an unfortunate martyr who has forced us to look at the prison-industrial system and the inequity that drives it,” she said. “His short life forces us to be better, that’s a worthy statue to have and let it be 500ft high.”

A series of public hearings were held in each borough last month by the Mayoral Advisory Commission on City Art, Monuments, and Markers, with its final hearing on 28 November. New Yorkers had the opportunity to voice their support or opposition of the monuments and the commission will bring the petition to the attention of New York’s mayor, Bill de Blasio, by the end of this month. The commission also had an online survey, which asked New Yorkers questions such as: “What is the best way to achieve public space that is open and inclusive?”

The commission’s co-chairs, Tom Finkelpearl and Darren Walker, recently published an op-ed last week in the New York Daily News, which says they are looking “to develop a set of guidelines that allows us to approach these issues in a coherent, transparent way that brings people together in dialogue, instead of driving them apart”.

While some of those guidelines includes adding labels or signs to change the meaning of the monuments, that isn’t the answer to one of the petition’s signees. “One of the most important functions of public space is to be a vibrant and safe hub for encounters of people from different backgrounds,” said Carin Kuoni, a professor at the New School. “An additional sign or label simply doesn’t carry the same weight as a monumental bronze sculpture that is placed in a prominent spot in the city.”