Masked "cowards" who commit violence in public places will face tougher penalties after a series of violent protests between rival groups in Melbourne.

Splinter groups of masked people dressed in black have caused trouble at recent rallies, although police generally managed to keep the anti-Islam and anti-racism groups apart at the latest protest on Sunday.

Victorian Premier Daniel Andrews has labelled masked protesters as "cowardly". (AAP)

Victorian Premier Daniel Andrews said the government will introduce new public order laws and penalties around affray and riotous behaviour to deal with people wearing masks.

"There will be additional penalties, additional specific offences and an aggravating factor, which means you'll get more time where you belong, behind bars, if you turn up and act violently with a mask on," Mr Andrews said.

"We are not having a situation where people turn our streets into some sort of battleground."

The move comes after violent rallies in Melbourne which saw masked protesters burn an Australian flag. (9NEWS)

Police were generally pleased with the behaviour of 120 True Blue Crew and 200 rival Campaign Against Racism and Fascism protesters at Sunday's rally, which was marred by flags being set alight and scuffles involving a splinter group.

But there was no repeat of the violent clashes between the opposing groups at a Coburg rally in late May.

The move comes after violent rallies in Melbourne which saw masked protesters burn an Australian flag. (9NEWS)

Yesterday Mr Andrews warned future offenders will "feel the full force of the law".

"If you commit a violent act and you are wearing a mask, then you will receive a significantly higher penalty because of the cowardly nature of wearing that mask," he said.

"You will feel the full force of the law because hardworking Victorians and their government are sick and tired of having to spend hundreds of thousands of dollars, indeed millions of dollars, to deal with this sort of riotous behaviour."