Confidence in the nation's building sector has fallen so low that it must be "rebuilt", the Morrison government minister with responsibility for construction says.

"There is solidarity that what we need to do is rebuild confidence in Australia’s building and construction sector," federal Industry Minister Karen Andrews said, after a meeting of Australian building ministers in Sydney on Thursday.

The ministers also agreed to fund a new taskforce to pursue nationally consistent building standards, amid the flammable cladding crisis plaguing Australia's construction industry.

Premier Daniel Andrews this week announced a $600 million fund to repair private buildings with flammable cladding.

The state asked the federal government to pay half, but Ms Andrews on Thursday reiterated Treasurer Josh Frydenberg's refusal earlier in the week, saying cladding was a state issue.