An anti-Islamic group is threatening to claim Albury, on the New South Wales-Victorian border, as its "territory" after being involved in violent clashes at protests in Bendigo and Melbourne.

The United Patriots Front (UPF), which is a splinter group of Reclaim Australia, staged a rally in Bendigo last month in protest against the town's first planned mosque.

It is now vowing to bring its protest power to the regional city of Albury where there are plans to redevelop an existing mosque.

The far-right group's spokesman, Shermon Burgess, has posted a video on social media vowing to come to Albury and the regional town of Orange in NSW after they "sort out this Bendigo problem".

"I will really enjoying marching straight into [Albury] and claiming it as my own," he said.

Recent rallies by the UPF in Melbourne and Bendigo have been met by rival protests by anti-racism groups and required police to use capsicum spray to subdue the crowds.

Meanwhile, police had to be called in to a Bendigo Council meeting last week when verbally abusive anti-mosque protesters shut down proceedings.

Albury councillor David Thurley said his council were concerned by the prospect of the group holding a rally locally.

"It's serious, it's hateful, it's disgusting and for people who think they can just walk into other people's communities, ferment hate and cause trouble - I don't understand where they're coming from or what it is they're trying to do," he said.

Cr Thurley said he had raised concerns with the local police after a Facebook page titled Stop the Albury Mosque was set up.

"There's not a lot we can do except to say to our community 'if you don't want this, you need to voice your opinion loud and clear that we don't want racist bigots here'," he said.

"We want people to be tolerant and to have a peaceful community that co-exists and shares cultures and music and food and so on."

Muslim leader confident community will reject far-right group

The Islamic Society of Albury-Wodonga is responsible for the plans to put $380,000 into redeveloping their existing community centre into a more modern facility, including a new mosque.

Its president, Yakub Muhammad, said the existing centre was crumbling and locals told him its condition was disrespectful to the area.

The United Patriots Front rallied in Bendigo last month against a planned mosque. ( ABC News: Stephanie Anderson )

"I thought, we don't have the money, but what we can do is get the locals together, raise as much as we possibly can, empty our wallets," he said.

"What we can do is demolish it and rebuild that and make it look brand new, you know, look respectable.

"At the moment you drive past it and it doesn't look respectable at all, it looks quite yucky actually."

Mr Muhammad has described the majority of UPF members as "lonely people" who were looking for a group to belong to.

"The other 2 per cent, they're the extremist ones and they're the people, you know they have this nonsense idea that we're going to take over Australia," he said.

"They're really obsessed with the idea that Islam is really bad.

"If they come, they'll probably come and just do a small protest or whatever.

"When they realise that Albury-Wodonga people, they don't really care about their garbage that they bring here, they'll just quickly move on because that's what they want - they're just after some attention."