Central to the public debate about the National Energy Guarantee (NEG) has been the numerical forecasts of its effects – in particular how much it will reduce power prices. In a democracy whose households pay some of the world’s highest electricity bills, it is obvious why this measure should shape the narrative on energy policy.

But Plato tells us that good decisions are based on knowledge, not numbers. What’s more, electricity markets are incredibly complex, and therefore not amenable to straightforward predictions.

The Energy Security Board has put numbers at the centre of its NEG proposal, but the basis of these numbers is not clear. With 22 colleagues at 10 other Australian universities, we are calling for state and territory ministers to ensure that the ESB’s modelling is available for proper scrutiny. I explain here why I support this request.

Read more: Infographic: the National Energy Guarantee at a glance

On October 17, 2017, the newly created ESB claimed in a letter to federal energy minister Josh Frydenberg that annual household bills would ultimately be A$100-115 lower under the NEG as a result of the NEG being introduced.

The ESB said this calculation was based on its estimate that wholesale electricity prices under the NEG would be 20-25% lower than under business as usual, and 8-10% lower than under the Clean Energy Target proposed by the Finkel Review.

No analysis or modelling was provided to justify these claims. But five weeks later the ESB had altered its forecast, releasing a report claiming that wholesale electricity prices would typically be 35% lower with the NEG than they would be without it. Underlying this claim was the assumption that only 597 megawatts of renewable generation would be developed between 2020 and 2030 if the NEG was not implemented.

Since more renewable generation than this was installed just on the roofs of Australia’s households and businesses in the first five months of 2018, the ESB was effectively suggesting that without the NEG investment in renewable generation would all but dry up.

This stands in stark contrast to the verdicts of other analysts. Bloomberg New Energy Finance predicted that 24,000MW of renewable generation (40 times more than the ESB’s figure) would be built between 2020 and 2030 without the NEG. Bloomberg also predicted less new renewable capacity with the NEG than without it.

Final design on the table

The ESB last week released its final design for the NEG to policymakers, but not the public. It now claims that the policy will reduce household electricity bills by A$150 a year relative to business as usual. It also now says that without the NEG around 8,000MW of new renewable generation will be installed (13 times more investment than it predicted eight months ago).

But the ESB says all of this will be installed behind the meter on the roofs of Australia’s homes and businesses and it persists with the assumption that no new large-scale renewable capacity will be built without the NEG.

But once again this seems to contrast vividly with what others are saying and doing. Several major companies have signed contracts for large-scale renewables, including Telstra, Carlton & United Breweries, Orora, and BlueScope Steel. The ESB’s assumption that all large-scale renewables development will grind to a halt without the NEG is even less plausible now than it was in November 2017.

Others have previously noted that the ESB’s estimate of renewable investment from 2020 to 2030 bears no relation to the estimates from the Australian Energy Market Operator (AEMO) of around 18,000 MW of additional renewable generation between 2020 and 2030, despite the ESB’s claims to the contrary.

However, the ESB’s final design has now helpfully clarified what several other analysts have already pointed out: that meeting the government’s target of reducing the electricity sector’s greenhouse emissions by 26% will require emissions reductions of just 2% between 2020 and 2030 beyond what is already set to be achieved. That is a meagre 0.2% cut per year that the NEG policy will be required to deliver.

I estimate that this will in fact be achieved several times over just with the 8,000MW of new rooftop solar capacity that the ESB predicts will happen even if the NEG is not implemented. To be clear, on the ESB’s numbers, Australia’s electricity sector greenhouse gas emissions will be lower than the government requires them to be, even if the NEG is not implemented. So how then can it be plausible to predict that the NEG will stimulate additional investment in renewable capacity beyond what would happen anyway?

Read more: Explainer: why we shouldn't be so quick to trust energy modelling

You can’t have your cake and eat it. If a policy is intended to make no difference to what would happen anyway, how can it be expected to drive down household bills by A$150?

And without putting its modelling into the public domain where it can be subjected to wider expert scrutiny, how will we know whether the ESB’s assumptions actually hold water?

The NEG will be a massive administrative change to Australia’s energy market, and a potentially substantive change if future governments set much higher emissions reduction targets. State and territory energy ministers are being asked to accept the ESB’s promise that household electricity bills will decline by 30-40% in the next few years, and that the NEG will account for a fair part of this. Those ministers should scrutinise this rosy projection carefully before accepting it. After all, the public will be looking to them, and not the federal government, to make good on these price pledges.