Anti-Islam rallies in Melbourne and around the country on Sunday have been described as "ugly and disappointing" by Australia's Race Discrimination Commissioner.

There were violent scenes in Melton, in Melbourne's west, where six people were arrested and weapons including knives were seized when Reclaim Australia campaigners clashed with anti-racism protesters.

Police were forced to subdue some protesters with capsicum spray.

Race Discrimination Commissioner Dr Tim Soutphommasane said he supported people's right to protest, but he was concerned some are out to spread hate.

He said it was important to remember that the "vast majority" of Australian society would not endorse that kind of "ugliness".

"These anti-Muslim protests represent a fringe of our society that's seeking to promote hatred and division," he said.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Listen Duration: 3 minutes 31 seconds 3 m 31 s 'Ugly and disappointing': six arrested at anti-Islam rallies, weapons seized ( Lindy Kerin ) Download 6.5 MB

"It's clear as well that the so-called Reclaim Australia protests have featured some far-right nationalist extremists, including some who have been intent on promoting hateful physical violence.

"All these elements of these protests are things we should reject and repudiate.

"They're not part of the kind of democratic discourse we should be having in a multicultural society."

Federal Justice Minister Michael Keenan downplayed the influence of the protests that took place at the weekend.

Mr Keenan said the protesters were "fringe-dwellers" who did not share the views of mainstream Australia.

"I wouldn't classify that as a significant backlash, I mean the amazing thing about these rallies is how sparsely attended they are," he said.

"This isn't reflective of mainstream Australia, and it's not reflective of the broad community opinion around the country."

Extremists on 'both sides': Victoria Police

Protesters take part in the Reclaim Australia anti-Islam rally in Melton on Sunday. ( ABC: Patrick Rocca )

Victoria Police Acting Commander Alan Byrne said in excess of 800 people were searched on the day of the protests, with extreme behaviour from both sides.

"There's extremists on both sides, people who actually go along with intent to cause riotous behaviour and affray and bring weapons and cover their faces for nefarious purposes only as I can imagine," he told 774 ABC Melbourne.

He said police did not ask people to uncover their faces, despite public concerns.

"Our job there is keeping the peace, there's no legislation that I know of that requires people to have their faces uncovered," he said.

"I think the common man's point of view is that why would you cover your face if you're going to go to a public demonstration [but] the main thing for police is to keep the peace.

"We do use discretion in relation to many different things such as blocking the roads, noise levels, but I think for police to start to proactively go and try to remove the face coverings or masks that people wear would be counter-productive to a peaceful demonstration."

No Room For Racism organiser Mel Gregson blamed police for fanning tensions between the two groups.

"I think the reality is that the presence of police and particularly the riot police with all their gear and all that type of stuff actually creates more tensions," she said.

"I think that if the police think there are concerns and want to come out and try to keep things calm then that's fine, but that's absolutely not what they're doing in relation to these rallies."

Acting Commander Byrne said the day was "mostly peaceful" despite the scuffles.

Anti-racism protesters should 'exercise civic responsibility'

Mr Soutphommasane said he did not believe the rallies should be banned.

"People have a right to protest peacefully, but there's absolutely no excuse for anyone to be conducting a campaign of intimidation that may be directed at anyone because of their religion or because of their background," he said.

"If anyone has engaged unlawful activity, they should be held to account of the full force of the law."

His warning was for both anti-racist protesters and anti-Islam protesters.

"No-one wants to see violence of any kind on our streets, certainly not any violence that's justified by extreme nationalist ideas or that targets people because of their religion," he said.

"Those marching against Reclaim Australia also need to exercise their civic responsibility.

"People should not be responding to bigoted ugliness with any ugliness of their own.

"It's important that those who are attending counter-protest rallies aren't playing into the hands of extremists who want to attract attention to their divisive and hateful cause."