Emmanuel Macron has said the “battle scenes” in central Paris between police and protesters over the weekend risked unnerving foreigners.

The French president told ministers at a cabinet meeting on Monday that the government must respond after images were relayed around the world of police firing teargas and water cannon at protesters who set up barricades, lit fires and smashed restaurants and shopfronts on the Champs-Élysées.

France was still clearing up on Monday after the clashes, which were sparked by anger over fuel tax rises. The clean-up operation continued along what France calls the “most beautiful avenue in the world”, as city authorities mobilised 200 extra workers to repair the damage to streets and buildings.

Shopkeepers whose windows were smashed and tagged with graffiti during what was supposed to be a peaceful demonstration by the gilets jaunes (yellow vests), had been hoping for a busy weekend after the Champs-Élysées’s celebrated Christmas lights were turned on last week.

Instead, they were forced to shut their shops as the street turned into a battlefield when a small number of protesters built barricades between themselves and the police and set fire to them.

“We shouldn’t underestimate the impact of these images of the Champs-Élysées [...] with battle scenes that were broadcast by the media in France and abroad,” government spokesman Benjamin Griveaux told a news briefing, recounting Macron’s words. “Behind this anger there is obviously something deeper and which we must answer, because this anger, these anxieties have existed for a long time,” Griveaux said.

BFM TV reported that France’s intelligence services had identified 80 to 120 extreme-right sympathisers at the heart of the violence, while other media claimed tags and logos at the scene suggested extreme leftwing and anarchist organisations were involved.

Officials said it was too early to establish the cost of the damage, but one estimated it could be up to €1.5m (£1.3m).

Sparked by fuel price rises, the gilets jaunes movement has embraced wider dissatisfaction with Macron and his centrist government, who are seen as out of touch with ordinary people. Ministers have said there will be no U-turn on fuel tax increases, which are part of environmental measures. Macron is expected to respond to the protests in a speech on Tuesday.

The Paris police chief, Michel Delpuech, said forces had used 5,000 teargas canisters, “more than one a minute, which has never been seen before”, while fire services put out about 100 fires. Police arrested 103 people, including a handful of minors; 45 have been charged and are due to appear before a judge.

Thirty-one people were injured in the clashes, 24 protesters and seven police officers. One protester was accused of throwing metal nuts at police, leaving one officer at risk of losing an eye, and was due in court on Monday along with a prison guard reportedly carrying a hammer when he was arrested at the protests.

The interior minister, Christophe Castaner, blamed “seditious” and “ultra-right” hooligans for hijacking the gilets jaunes protest and sparking violence, but police said those arrested were mostly young men with no criminal record, many of them from outside Paris.

“We are well aware that it’s a very small minority who for several years have attached themselves to protests each time in order to smash everything up,” said Emmanuel Grégoire, Paris deputy mayor.

Officials said more than 106,000 gilets jaunes protested across France on Saturday, about half the number in demonstrations the previous week, in which two people died. Reports that further protests are planned in Paris on 1 December have been denied by unofficial representatives of the gilets jaunes, who have no official leaders or organisation.

In Italy, a protest group inspired by the gilets jaunes and unveiled on Facebook on Saturday has garnered thousands of supporters online. Alberto Nardozzi, who runs market stalls in Turin and started the Italian protest group, said Brussels was the focus of his ire.

“We are inspired by the French gilet jaunes,” he said. “But we are motivated by other issues. We, unlike the French, support our government. What we protest against is Europe. We want Europe to no longer interfere with Italian politics.”

Nardozzi said his group, which was planning a major rally in January, opposed the so-called Bolkestein directive, which liberalises cross-border services in the EU’s internal market, as well as taxes on business and motorway tolls.

“Brussels has busted our balls,” he said. “Without it I am convinced that Italy would do well. The time has come to stop austerity in Europe, which has created poverty among the Italians.”

Additional reporting by Lorenzo Tondo in Palermo