Aboriginal Labor Senator Nova Peris claims bipartisan efforts to recognise Indigenous people in the Australian Constitution have been set back by the Prime Minister's comments on British settlement.

In comments after an economic speech in Melbourne on Thursday night Tony Abbott said Australia had been "unsettled" before the arrival of the First Fleet.

"I guess our country owes its existence to a form of foreign investment by the British government in the then unsettled or scarcely settled great south land," Mr Abbott told the Melbourne Institute Economic Conference.

In a statement, Senator Peris hit out at Mr Abbott's comments, saying they had offended Aboriginal Australians and many other people around the country.

"The comments were highly offensive, dismissive of Indigenous peoples and simply incorrect," Senator Peris said.

"British settlement was not foreign investment. It was occupation.

"Current foreign investment in Australia can be defended, promoted and debated without such insensitive statements from the Prime Minister."

Mr Abbott's chief Indigenous adviser Warren Mundine, who was in the audience for Mr Abbott's speech, has also described the comments as "silly".

Opposition Leader Bill Shorten said: "This was an offensive thing to say, especially from someone who proclaims to be Prime Minister for Indigenous Australia".

Mr Abbott's office declined to comment, as did Reconciliation Australia.