Seven people have been arrested amid violent scuffles between anti-racism and anti-immigration groups holding counter rallies in Melbourne's north.

Key points: Police attempted to keep anti-racism and anti-immigration groups apart

Police attempted to keep anti-racism and anti-immigration groups apart Officers used capsicum spray after scuffles broke out, five people arrested

Officers used capsicum spray after scuffles broke out, five people arrested Victoria Police said both sides were to blame for violence

Hundreds of police officers had to form a line between the two groups, while mounted police also patrolled the crowd in the multicultural suburb of Coburg.

But they failed to keep some protesters apart, and officers used capsicum foam as rolling scuffles broke out between the rival groups as they made their way down the street.

A group of officers in riot gear pounced on two protesters, pinning them to the ground and tying their hands with zip ties before leading them away.

Police had been preparing for an outbreak of violence. ( AAP: Julian Smith )

One of them yelled "What am I under arrest for?" as the incident unfolded.

Police said two people were arrested for riotous behaviour, one for assaulting police and one for hindering police. Another was arrested for robbery.

Two people were arrested before the rally for weapons offences.

The two groups hurled abuse at each other during the marches, with anti-racism protesters chanting "Nazi scum, off our streets".

Counter-protesters attacked a group with large flagpoles flying the Australian flag.

The two groups hurled abuse at each other as police kept them apart. ( AAP: Julian Smith )

Members of the United Patriots Front, which held a counter-rally in response to the planned anti-racism march, held a sign that read "No refugees. Our home, our future".

The leader of the United Patriots Front, Blair Cottrell, said the rally was an "initial success" but then "the police happened".

"They prevented us from doing what we wanted to do, which was to move up that street to counter the opposition," he said.

He said he wanted to counter the opposing group "in the same way that they do, with force and terror".

Police label protesters 'cowards', hunt for troublemakers

Commander Sharon Cowden from Victoria Police said a taskforce would be set up to identify those behind the violence at the rallies.

She said both sides were to blame and condemned those who covered their faces, calling them "cowards".

"I understand the need and the right to protest ... but what we can't have is this violence in our community," Commander Cowden said.

"We saw inappropriate and often cowardly behaviour, people wearing masks."

Commander Cowden said there were only minor injuries and the operation had been a success.

She said Victoria Police was well-prepared for an outbreak of violence and denied they had "failed" by not preventing the clashes in the first place.

"Some people come along ... and they're looking for trouble."

Multiculturalism one of our great achievements: PM

Police took two men away after holding them to the ground. ( ABC News )

A third group of people, who appeared to be those taking part in the original rally organised by local community groups on social media, walked in unison as they chanted "Say it loud, say it clear, Muslims always welcome here".

Loading

When asked about the rallies, Prime Minister Malcolm Turnbull said Australia's multicultural society was "one of our great achievements" and urged mutual respect.

"We are united, we have so much in common, and we are stronger and more prosperous as a nation because of that diversity," he said.

"My message today and every day, as your Prime Minister, is that mutual respect, respect for each other, giving the respect to others that we ask them and expect them to give to us, that is the key to our success."

In February, hundreds of police separated rival groups rallying in the central Victorian city of Bendigo demonstrating for and against the construction of a local mosque.