Prime Minister Malcolm Turnbull and senior cabinet ministers including Barnaby Joyce and Scott Morrison have dismissed calls to change the date of Australia Day, despite comments by the Prime Minister's Indigenous advisor and a former cabinet minister.

Key points: Ian Macfarlane says he has had a change of heart, wants date changed to March 1

Ian Macfarlane says he has had a change of heart, wants date changed to March 1 Warren Mundine says date marks Indigenous people losing their land and massacres

Warren Mundine says date marks Indigenous people losing their land and massacres Barnaby Joyce says people who want date changed are "miserable, gutted people"

Ian Macfarlane, a former resources minister who resigned from Parliament last year, wants the date changed and admits he has only had the change of heart in the last few months.

"I started to think about my forebears, and the way I'd feel if in England I had to celebrate United Kingdom Day on the anniversary of the Battle of Culloden when my mother's forebears were cut in half by English grapeshot and then hunted down and murdered along with their wives or children," he told AM.

"Or on the day when the Vikings invaded Arrochar where my father's family is from, and I thought yeah, look, this really isn't something that we can continue with."

Head of the Prime Minister's Indigenous Advisory Council, Warren Mundine, believes January 26 has a devastating and emotional impact for many Australians.

"Most Aboriginal Indigenous people want to celebrate Australia Day and they feel they are Australians, that they are part of the Australian community," he said.

Mr Mundine said the date marked Indigenous people losing their land, massacres, people being driven onto reserves, and the slow destruction of languages and culture.

"It was just like the end of the world for a lot of communities," he said.

Mr Turnbull said he did not support changing the date and said Australia was the most successful multicultural nation in the world.

"There are many bigger and more profound issues — including Constitutional recognition — to deal with than the date of Australia Day," he said.

"I know Ian — he is a good friend of mine and I respect his point of view.

"It is a debate everyone is entitled to have but it is not a change the Government supports."

'Out of control political correctness'

The Deputy Prime Minister said it was "annoying" to hear people protesting over Australia Day.

Mr Joyce told the ABC that calls to change the date was "out of control political correctness".

"I'm not going to be politically bullied into changing the name of Australia Day because it's the raison d'etre of a certain group for a certain period of time," he said.

"It's annoying to wake up and listen to the radio and hear people talk about how its morally repugnant to celebrate."

Ian Macfarlane said Australia Day could be moved to March 1. ( ABC News: Loretta Florance )

When asked on Invasion Day protests, Mr Joyce cited his own ancestors and stressed they were not invaders.

"There's nothing evil about [Australia Day]," he said.

"People say we're going to change things around because it was invaded — give me a break."

Treasurer Scott Morrison said he did not support moving the date and said Australia Day was a time to reflect on achievements and further reconciliation.

"I think all Australians can embrace all of our stories and I think that's one of the most wonderful things about the country," he said.

"But that doesn't mean we need to deny any parts of our heritage and parts of our history, whether it's our colonial history, our settlement history, our deep and long Indigenous history, our post-war migration history and refugees coming to Australia."

Labor's Chris Bowen supports keeping Australia Day on January 26 saying "it is a good day for many in the community".

"While I respect the views that are presented right across the board, it's a debate which I think comes up from time to time," he said.

"It is a good day for many to celebrate the achievements of our nation and I think it should remain so."

Tasmanian Liberal Senator Eric Abetz has blamed what he called "latte-sipping apologists" for bullying Australians into not celebrating Australia Day.

But Tasmanian Governor Kate Warner said Australians should consider moving the date to better reflect the concerns of Indigenous people.

"Australia Day has been evolving — what it means has been evolving for a long time, and I think will continue to evolve — and I can quite understand how Aboriginal people feel about it and think that it's invasion day rather than Australia Day," she said.

Macfarlane: Move Australia Day to March 1

Mr Macfarlane said his change of mind was prompted by triple j considering changing the date for the Hottest 100 and the Freemantle Council's decision to boycott Australia Day.

"In the end, the Government will have to act on the date, whether that's the current Government or the next Government, because this issue isn't going to go away," he said.

"It's going to continue to divide Australians at a time when we need to be united."

Mr Macfarlane, who is now the chief executive of the Queensland Resources Council, said Australia Day could be moved to March 1, as it was the day the Commonwealth started functioning as one nation under one Government.

"It can't be the day of Federation (1 January) because people will be hungover from New Year's Day, quite frankly," he said.

"It does need to be in the warmer months so we can continue the Australian tradition of celebrating in the great outdoors, so I've gone for 1 March — and let the debate begin."

Mr Macfarlane spoke at the Australian Unity Great Australia Day Breakfast.

"Today is the first time in a quarter of a century of public life that I'm going to deliver what I term a bleeding-heart speech," he said ahead of the event.

"I ran a pretty big business portfolio, I'm a pretty matter-of-fact guy and I say it like it is and I don't do bleeding heart speeches."