Prime Minister Tony Abbott has been accused of effectively declaring Australia "terra nullius" before British settlement, after remarking that Sydney was "nothing but bush" prior to the arrival of the First Fleet.

During a breakfast for British prime minister David Cameron in Sydney this morning, Mr Abbott made a speech about infrastructure and noted the "extraordinary partnership" between the two countries since the arrival of the First Fleet in 1788.

"As we look around this glorious city, as we see the extraordinary development, it's hard to think that back in 1788 it was nothing but bush," Mr Abbott said.

"The marines and the convicts and the sailors that straggled off those 12 ships, just a few hundred yards from where we are now, must have thought they had come almost to the moon.

"Everything would have been so strange. Everything would have seemed so extraordinarily basic and raw, and now a city which is one of the most spectacular cities on our globe."

Kirstie Parker from the National Congress of Australia's First Peoples said the comments do tremendous damage to the relationship of the Prime Minister and Aboriginal people.

"I'd say they were a blunder except this is becoming a habit for the Prime Minister," she said.

"On several occasions just in the last couple of months, he has made comments that have erased Aboriginal and Torres Strait Islander people from the landscape."

Ms Parker said it was not a case of reading too much into the remarks, or taking them out of context.

"For the Prime Minister to say there was nothing here but bush is incorrect; there were people here with sophisticated systems and societies and rules," she said.

"We were here."

Labor's Indigenous affairs spokesman Shayne Neumann said Aboriginal people have a right to feel that the Prime Minister "owes them an apology and he should express regret at the form of words he used today when he was honouring his own heritage but denying theirs".

"It's a denial of their culture, their language, their heritage and their custom and basically it shows the Prime Minister has a sort of terra nullius type approach to the continent," he said.

"Language counts. Words have meaning, words can be like bullets, words are symbolic.

"They drive people's thoughts and can influence people."

Greens Indigenous affairs spokeswoman Rachel Siewert said Mr Abbott's comments were "another example of the Prime Minister ignoring the reality of colonisation and the people's flourishing culture and languages that were here at the time of European settlement".

Space to play or pause, M to mute, left and right arrows to seek, up and down arrows for volume. Listen Duration: 4 minutes 3 seconds 4 m 3 s Sydney nothing but bush before First Fleet: Abbott Download 7.4 MB

Mr Abbott made a point of acknowledging Indigenous history in his speech to the Parliament on Friday as he welcomed Mr Cameron to Canberra.

"Modern Australia has an Aboriginal heritage, a British foundation and a multicultural character," he said.

Mr Abbott also attracted criticism in August when he described the arrival of the First Fleet as the "defining moment" in Australian history.

Subsequently Mr Abbott conducted an interview with British newspaper The Telegraph during his trip to a remote Aboriginal community in Arnhem Land.

According to the report he said the arrival of British settlers on the First Fleet proved devastating for Aboriginal people.

"Initially the impact [of British settlement] was all bad, disease, dispossession, discrimination, at times wanton murder," he said.