The European Union has warned France it could face sanctions over the treatment of its Roma community after the interior minister said the majority should be deported and that France was "not here to welcome these populations".

The European Commission spokesman Olivier Bailly warned France that "free circulation and the freedom to live in other countries" within the European Union were fundamental rights written into treaties and if they weren't respected the commission would use "all the means at its disposal to sanction violations".

He said: "Roma, like all EU citizens, have the freedom to circulate freely in all member states of the EU and to live in a country other than their country of origin."

The angry response from Brussels comes six months before local elections in which several political parties, ranging from the far-right to the Socialists in power, have seized on the issue of the Roma's illegal camps, portraying the ethnic minority's presence as threat to French people. The popular and outspoken interior minister Manuel Valls told the radio station France Inter on Tuesday that "these populations have lifestyles that are very different from ours, and are clearly in confrontation" with the lifestyle of the French. He said few Roma could integrate into French society.

Asked about the Socialist government's controversial dismantling of camps – continuing a crackdown begun under the previous rightwing administration of Nicolas Sarkozy – Valls said: "I approved the dismantling of these veritable slums that represent a danger both for the people of Roma origin, but also of course the people who live in working-class neighbourhoods" near them.

Critics say the policy is discriminatory against the roughly 20,000 Roma in France, most of whom trace their origins to Bulgaria and Romania and who have not been offered viable alternatives to squalid makeshift settlements.

While some Socialists, including the government minister Arnaud Montebourg, criticised Valls's comments as "excessive", the interior minister stood by his words, telling BFMTV the "majority" of Roma should be taken to the border. He said: "We must not discriminate, nor must we put our head in the sand."

In 2010, Sarkozy prompted criticism from the European Commission and the Vatican when he linked immigration to crime and promised to expel Roma migrants and destroy illegal camps.

France has been pushing to keep Romania and Bulgaria from gaining full access to Europe's Schengen zone, which allows passport-free travel. The two eastern European countries are set to accede to the 26-nation zone on 1 January.