A sociologist and Anglican priest says atheists or people without a specific religion are creating the most problems for inter-faith tolerance in Australia.

Monash University Professor Gary Bouma says people without a specific faith are fuelling sectarian conflict and cause division in society.

"Conflict comes up when groups vilify, deny the right to build the mosques," he told the Studies of Religion in Focus conference in Sydney today.

"Or when the 'nones' - those who are anti-theist - [say] 'You're stupid', that religious voices should be driven out of the public policy area, that religion shouldn't be in schools, etc.

"That is conflict, and that is highly divisive in this society."

Professor Bouma says a growth in religious diversity in recent years has created problems for Australian schools.

He says schools have to work out to how to encourage respectful engagement between students and teachers of various religions.

"Schools have a whole variety of competing loyalties within the teachers and within the students," he said.

"It can sometimes go to conflict if there's a viewpoint that some don't want expressed.

"But how is it that you accommodate the diversity? How is it that you develop respectful engagement between diverse groups?"