Scott Morrison has called on Australians to “disagree better” just hours after Peter Dutton accused Greens politicians of being just as bad as the disgraced far-right senator Fraser Anning.

In a speech to the Australia-Israel Chamber of Commerce in Victoria, the prime minister also said he wanted the nation to “reject the thinking that one person’s gain is another’s loss”.

Earlier this month, the prime minister said it was “simple math” that medically transferred asylum seekers would take the place of Australian citizen patients in hospitals. He had spoken up in support of Dutton after his home affairs minister had made similar claims.

Dutton criticised for 'vile' claim that Greens 'just as bad' as Fraser Anning Read more

On Monday, Morrison said he wanted to “remove the demarcation lines between Australians”.

“I want us to reject the thinking that one person’s gain is another’s loss,” he said. “This is a doctrine of scarcity that betrays our social and economic prosperity and creates an environment for conflict and division.

“I see every Australian as an individual, not part of some tribal group to be traded off against another.”

While denouncing racism and “tribalism”, Morrison also said the community needed to be free to hold “legitimate policy debates” on issues such as migrant intake and border security without the conversations being “hijacked” by accusations of intolerance or racial hatred.

Morrison said he would be visiting the governor general in the coming weeks to call for an election, and ask Australians not to “join my side but to convince them … we are on theirs”.

The Christchurch terrorist attack dominated Morrison’s speech, with the prime minister saying he wanted to engage in a “broader reflection” on “how we see difference in our world and how we manage it”.

“The bonds between us all matter,” he told the audience in Melbourne. “The rainforests in north Queensland are older than the Amazon. Every part of this ecosystem reinforces itself. It doesn’t grow apart, it grows together.

“And so it is with countries and their peoples. But these ties that bind us are under new pressures and are at risk of breaking.

“If we allow a culture of ‘us and them’, of tribalism, to take hold; if we surrender an individual to be defined not by their own unique worth and contribution but by the tribe they are assigned to; if we yield to the compulsion to pick sides rather than happy coexistence, we will lose what makes diversity work in Australia.”

Morrison attacked “tribalism”, which he said was increasingly “taking over”, with people only reading the news they agreed with, interacting with people they agreed with and “having less understanding and grace towards others that we do not even know, making the worst possible assumptions about them and their motives, simply because we disagree with them”.

His words came hours after Dutton attacked the Muslim senator Mehreen Faruqi and the Greens leader, Richard Di Natale, for their criticisms of his rhetoric in regards to African “gangs”, asylum seekers and refugees in recent years.

“I’m hardly going to take morals lectures from the extreme left, who frankly are just as bad in this circumstance as people like Fraser Anning – they should equally be condemned,” Dutton told ABC radio on Monday.

In his speech, Morrison said tribalism was true of “the left and the right”.

“And even more so from those shouting from the fringes to a mainstream of quiet Australians that just want to get on with their lives,” he said. “Hate, blame and contempt are the staples of tribalism. It is consuming modern debate, egged on by an appetite for conflict as entertainment, not so different from the primitive appetites of the Colosseum days, with a similar corrosive impact on the fabric of our society.

“Contempt is defined by the philosophers as ‘the unsullied conviction of the worthlessness of another’. The worthlessness of another! That is where mindless tribalism takes us.

“It ends in the worst of places. Last week it ended the lives of 50 fellow human beings, including children praying in Christchurch.”

Morrison said the discussion around immigration intake was a “classic example” of tribalism at work and said “frustrations” about traffic and infrastructure did not mean people were “anti-migrant or racist”.

New Zealand mosque attack: alleged gunman bought four firearms online – live Read more

“For the overwhelming majority of Australians concerned about this issue, this is not and never would be their motivation. But that is how the tribalists seek to confect it, from both sides.

“The worst example being the despicable appropriation of concerns about immigration as a justification for a terrorist atrocity. Such views have rightly been denounced. But equally, so too must the imputation that the motivation for supporting moderated immigration levels is racial hatred.

“We cannot allow such legitimate policy debates to be hijacked like this.”

Morrison praised the New Zealand prime minister, Jacinda Ardern, for her words after Christchurch attack on Friday, when she said of the country’s Muslim community “they are us”.

“This is a powerful idea. No them but us … As prime minister, I want to continue to bring Australians together, not set them against one another.”

Morrison also announced a further $55m in community safety grants, with priority to be given to religious schools and places of worship and religious assembly, following the Christchurch attack. The money is to help community and religious groups upgrade security at their facilities.