More than 150 community and human rights organisations around Australia have called on federal attorney general George Brandis to reconsider his intention to repeal critical aspects of the Racial Discrimination Act.

The groups, including Oxfam Australia, the Australian Council of Social Services and the Lowitja Institute, argue that a repeal of section 18(c) of the act, which makes unlawful any public conduct that offends another person on the basis of their race, would increase “the likelihood of racial discrimination and racist violence”.

In an open letter to the attorney general, the groups continue: “The Racial Discrimination Act has long played a critical role in combating racial hatred and protecting individuals and groups against discrimination and hate speech based on race, colour, descent or national or ethnic origin.”

It adds: “Unfortunately racism remains widespread in our Australian community. Racial vilification complaints to the Australian Human Rights Commission increased 59 per cent last year.”

Brandis pledged during the 2013 election to repeal elements of the Racial Discrimination Act and last month said the plans had not changed since.

“You cannot have a situation in a liberal democracy in which the expression of an opinion is rendered unlawful because somebody else . . . finds it offensive or insulting," he told the Australian in November.

Section 18(c) of the Act has come to be known as the “Bolt law” after right wing columnist Andrew Bolt was found to have breached it in 2011 in articles in which he argued nine high-profile, fair-skinned Aboriginal people had used their heritage to gain entitlement.

In Bolt’s case, section 18(d) of the Act, which allows for exemptions in cases of public interest and “fair comment”, was found not to apply as the offending articles were not written in good faith and carried factual error.

“Many Aboriginal and Torres Strait Islander peoples experience racism and hate speech on a regular basis,” said Rodney Dillon of Amnesty International Australia, another of the 155 groups to co-sign the letter.

“Strong legal protections send a clear message that racist hate speech is not acceptable and that people who experience such treatment will be protected by law.”

Joe Caputo, chair of the Federation of Ethnic Communities Councils of Australia, another of the groups to sign the letter, added: “The repeal of section 18(c) would be incredibly damaging, sending a signal that hate speech is acceptable and allowing racism to get a foothold in our proudly multicultural nation.”

On Monday Brandis met representatives from eight groups, including the National Congress of Australia's First Peoples, representing minority communities. They expressed lukewarm support for a review of the Act.

"The balance struck by the existing law in Sections 18(c) and 18(d) of the Act was carefully reached after years of national inquiries and debates in Parliament and the general community,” the group said.

"Any attempt to restrike that balance is not to be undertaken lightly.

"We encourage the government to continue its consultations with us and the wider community.''