Federal Attorney-General George Brandis is reportedly preparing to water down his controversial plan to change racial discrimination laws.

Fairfax Media says the change of heart follows "an avalanche of submissions" citing concerns over the Abbott government's proposal to reform section 18 of the Racial Discrimination Act on the basis that it unreasonably restricts freedom of speech.

Two Liberal MPs who initially supported Senator Brandis' plan have also admitted there needs to be a rethink, the Sydney Morning Herald reports.

Several MPs believe the law changes could be "parked" for months as the government grapples with a fierce budget backlash, the paper says.

Senator Brandis has already been forced by cabinet to soften his original approach amid protest from coalition MPs in marginal electorates, some of whom represent large ethnic communities.

The Herald cites Senator Brandis as saying there is a large variety of views from all points of the opinion spectrum on the law changes, all of which will be taken into consideration.

The proposed repeal of the act would effectively remove its prohibition on offending, insulting or humiliating individuals or groups because of their race or ethnicity.