It is “impossible” for Australia Day to be an inclusive holiday while it remains held on 26 January, Reconciliation Australia has said.

The independent organisation, which is the national expert body on reconciliation with Aboriginal and Torres Strait Islander people, says changing the date of Australia Day is “a relatively small task” that would demonstrate a willingness to address past wrongs.

“Asking Indigenous people to celebrate on January 26 is like asking them to dance on their ancestors’ graves,” its chief executive, Karen Mundine, told Guardian Australia. “We’ve changed the date before – in fact January 26 has only been a national public holiday since 1994 – and will have to do so again if we want to achieve a national date that unifies all Australians.”

Linda Burney warns Greens push to change Australia Day 'more divisive than helpful' Read more

Reconciliation Australia was joined by the National Congress of Australia’s First Peoples, the peak body representing Indigenous people in Australia, and the Healing Foundation, the peak body representing survivors of the stolen generations, in pushing for a date change.

A poll released by the Australia Institute on Thursday found that 56% of Australians don’t mind on what date Australia Day is held. While 77% incorrectly believed Australia Day had always been held on 26 January (it was only gazetted in 1994), only 38% correctly identified the landing of the first fleet as the origin of the day.

Notably, 49% said Australia Day should not be held on a day which is offensive to Indigenous Australians.

Neither the federal government nor the Labor opposition support the push to change the date of Australia Day.

The assistant immigration minister, Alex Hawke, has dismissed calls to change the date as a “top-down” campaign from the Greens, claiming it did not have support from either Indigenous or non-Indigenous members of the wider community.

The prime minister, Malcolm Turnbull, said he was “disappointed” in those who wanted to change the date, saying to do so would be a denial of history.

But these three leading Indigenous organisations said most Aboriginal and Torres Strait Islander people found it “impossible to celebrate” on 26 January.

“The National Australia Day Council has acknowledged the varying perceptions of the date and attempted to reframe Australia Day as an occasion where we can reflect and celebrate at the same time,” Mundine said. “While these efforts are appreciated, it remains impossible for Australia Day to be an inclusive occasion as long as it is held on a date that some Australians celebrate, while others grieve.”

Australia Day row: minister says no 'reasonable' case for changing the date Read more

The Healing Foundation CEO, Richard Weston, said 26 January, the anniversary of the arrival of the first fleet, was “a reminder of the pain and loss caused by 230 years of dispossession, dislocation and mistreatment”.

“It is impossible to celebrate when it brings to mind the deep hurt borne by our ancestors and how that suffering continues to impact today,” Weston said. “Changing the date won’t make the problems in our communities disappear but acknowledging the wrongs of the past is important for healing to occur.”

National Congress said the current date of Australia Day was “not compatible with reconciliation”.

Quick guide History of Australia Day Show Hide Why is Australia Day held on 26 January? 26 January 1788 was the day the first fleet pulled into Sydney Cove and planted a British flag in the soil. They arrived in Botany Bay about a week earlier. It was first celebrated as a public holiday in 1818, on the 30th anniversary of that landing. The day was known variously as “foundation day,” “anniversary day” or “first landing” until 1946, when commonwealth and state governments agreed they should all celebrate the anniversary of British colonisation on the same day, and that day should be called “Australia Day”. The public holiday was not consistently held on 26 January until 1994, but was generally used to create a long weekend within that week. It has been recognised as a day of protest by Aboriginal and Torres Strait Islander people since at least 1938, when a national day of mourning was held during the sesquicentenary celebrations in Sydney. Photograph: Chris Hopkins/Getty Images AsiaPac

“‘There are 364 other days that would be more suitable for reconciliation, provided our history is understood and respected,” its co-chair Rod Little said.

Labor MP Linda Burney, the first Indigenous woman in the federal lower house, said 26 January was “extremely painful” for Indigenous Australians but the push from the Greens “could end up being more divisive than helpful”.

Burney said Australia Day should be an opportunity for truth-telling and reflection.She called for the government to enact the reforms proposed in the Uluru Statement, which included a truth-telling and treaty-making commission.