The Islamic Society of Victoria is preparing to take legal action against ASIO for what it says is constant harassment and bullying.

Members of the Preston Mosque in Melbourne's north claim intelligence operatives are approaching worshippers on a daily basis, offering them jobs and demanding personal information.

The secretary of the Islamic Society of Victoria, Baha Yehia, believes sermons are being monitored and he wants the harassment to stop.

"We will go to Canberra if we have to. We will go to Canberra and we will complain directly to the Prime Minister," he said.

The Islamic group has sought the legal services of criminal defence lawyer Robert Stary, who also defended Jack Thomas, the first Australian to be convicted under anti-terrorism laws.

Mr Stary says those that support the Palestinian struggle feel vulnerable.

"They're at the pointy end. They've seen the many prosecutions, some of which have failed, some of which have been successful," he said.

In 2008, seven people connected with the Preston Mosque were found guilty of belonging to a terrorist organisation. Ringleader Abdul Nacer Benbrika is now in jail.

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Listen Duration: 2 minutes 43 seconds 2 m 43 s Listen to the AM story Download 1.2 MB

"None of these people used to practise or preach at the mosque. They used to attend the mosque. You know mosques have a different structure to churches," said Mr Yehia.

According to Mr Yehia, the harassment of mosque-goers is so constant, that they have recently held a public legal seminar on how to deal with intelligence agents.

"People are more aware now and people know now that they can say no to ASIO and we don't have to speak to you and we don't want to speak to you.

"And they know if they do get harassed they can come and report the incident ... and if they don't back off, we will be going to the Attorney-General and to David Irvine as well, the head chief of ASIO," he said.

Mr Stary believes ASIO has too much power.

"You've got to remember now of course that ASIO has trebled in its size since 9/11. It's a very large organisation, really unparalleled in its growth and its size and its power in the history of the democracy of Australia," he said.

Mr Yehia says the intelligence agency's interference is starting to divide the community.

"There's a bit of mistrust in the community, because nobody knows who is working for ASIO, because a lot of people have been approached and they've also been asked to work for ASIO," he said.

ASIO says it is inappropriate to comment on specific communities.