Former Liberal prime minister Malcolm Fraser has labelled Tony Abbott's use of the term Team Australia "divisive" and "counter-productive".

Mr Abbott used the phrase when announcing plans for new anti-terrorism laws, raising concerns among community leaders.

The Islamic Council of Victoria (ICV), which represents more than 150,000 Muslims, boycotted Mr Abbott's meeting with Muslim leaders in Melbourne after he told a radio station that "everyone has got to be on Team Australia".

Mr Fraser said the Prime Minister's choice of language was "terrible".

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Listen Duration: 3 minutes 6 seconds 3 m 6 s Listen to former prime minister Malcolm Fraser talk about the term Team Australia. Download 5.7 MB

"It resonates very, very badly with Australia's ethnic communities," he told AM.

"If I were still a member of the Liberal Party I would be totally distraught. I am interested in policy and very much interested in what's going to happen in the next week, next month, next year.

"You have got to be able to debate, and using that term is almost an attempt to stop or to shut down debate."

The Race Discrimination Commissioner has warned the Prime Minister and Federal Government should be careful not to cause divisions among the Arab and Muslim community.

Australia's relationship with US comes at a price: Fraser

Mr Fraser said the larger question before Australians right now is what the country's future will be.

The former prime minister's recent book, Dangerous Allies, which won this year's John Button literary prize for political commentary, argues that Australia's close relationship with the United States could lead to conflict with China.

"How many politicians talk about that in a realistic sense? There's no direct threat to Australia. We do not need to be a frightened nation," Mr Fraser told AM.

"You know, if America goes to war with China because Japan does something stupid – which is the most likely scenario – Australia, at the moment, on current policy settings, does not have the capacity to say 'No'," he said.

But Mr Fraser said the situation was no different under the Rudd or Gillard governments.

"You can't pick much difference between the political parties," he said.

"It was Gillard who put those American troops in Darwin. Julie Bishop said, 'We're going to stand up to China'.

"What are we standing up to China about? We don't have an argument with China unless they create one."

Mr Fraser said that, in many ways, it is in Australia's DNA to sustain these relationships.

"It made sense in the early days when we were small colonies – few people, few resources. It has also made sense during the days of the Cold War," he said.

"It does not make sense in the changed strategic context. The world has changed; America has changed."

Mr Fraser said new political voices are needed.

"And the level of disenchantment with today's politics is greater than it's ever been, I think, in our history," he said.

"That's not just an octogenarian saying that; I get young people saying it to me.

"We want a young man's revolution or a young people's revolution – men and women saying, 'We've had enough of this nonsense. We want a party that will stand for and work for Australia'."