Australia’s energy markets got a big shock in July this year, when wholesale electricity prices spiked in South Australia, alarming the state government and major industrial customers. Commentators rushed to find the immediate culprits. But the real issues lie elsewhere.

As shown by the Grattan Institute’s latest report the market worked. Having soared, prices fell back to more manageable levels. The lights stayed on.

Yet South Australia’s power shock exposed a looming problem in Australia’s electricity system – not high prices or the threat of blackouts, but an emerging conflict between Australia’s climate change policies and the demands of our energy market.

A perfect storm

On the evening of July 7, the wind wasn’t blowing, the sun wasn’t shining, and the electricity connector that supplies power from Victoria was down for maintenance. This meant gas set the wholesale price, and gas is expensive these days, especially during a cold winter. At 7.30pm wholesale spot prices soared close to A$9,000 per megawatt hour. For the whole month they averaged A$230 a megawatt hour. They were closer to A$65 in the rest of the country.

Australia has committed to a target to reduce greenhouse gas emissions by 26-28% below 2005 levels by 2030. Despite this well known and significant target, the national debate on climate change has been so toxic and so destructive that almost no policy remains to reduce emissions from the power sector in line with that target.

By 2014 the much maligned renewable energy target (RET), a Howard government industry policy to support renewable energy, remained as the only policy with any real impact on the sector’s emissions.

Wind power has been the winning technology from the RET, and South Australia has been the winning state. Wind now supplies 40% of electricity in South Australia due to highly favourable local conditions. Because wind has no fuel costs it suppressed wholesale prices in the state and forced the shutdown of all coal plants and the mothballing of some gas plants. But wind is intermittent – it generates power only when it is blowing, and the night of July 7 it barely was.

A report by the Australian Energy Market Operator noted that the market did deliver on reliability and security of supply in July. It reviewed the behaviour of market participants and concluded there were “no departures from normal market rules and procedures”.

The events of July do not expose an immediate crisis, but they have exposed the potential consequences of a disconnection between climate change policy and energy markets. If it is not addressed, the goals of reliable, affordable and sustainable energy may not be achieved.

The bigger problem

Climate change policy should work with and not outside the electricity market. With a fixed generation target of 33,000 gigawatt hours of renewable electricity by 2020 and a market for renewable energy credits outside the wholesale spot market under the RET, the conditions for problems were established some time ago.

The specific issues that arose from the design of the RET would have been far less problematic if one of the attempts over the last ten years to implement a national climate policy had been successful. A rising carbon price would have steadily changed the relative competitiveness of high and low emissions electricity sources and the RET would have quietly faded.

The first lesson for governments is that we need to establish a credible, scalable and predictable national climate change policy to have a chance of achieving emissions reduction targets without compromising power reliability or security of supply. A national emissions trading scheme would be best, but pragmatism and urgency mean we need to consider second best.

While such an outcome is the first priority, it will not provide all the answers. The rapid introduction of a very large proportion of new intermittent electricity supply creates problems that were not foreseen when traditional generation from coal and gas supplied the bulk of Australia’s power needs.

All of the wind farms in one state could be offline at the same time – a far less likely event with traditional generation. The problem can be solved by investment in storage and in flexible responses such as gas and other fast-start generators. Commercial deals with consumers paid to reduce demand could also contribute.

Lower average prices combined with infrequent big price spikes are not an obvious way to encourage long-term investors. The market may find solutions with new forms of contracts for flexibility or the market operator could introduce new structures or regulations to complement the existing wholesale spot market.

Much uncertainty exists, no easy fixes are in sight and the consequences of failure are high. Getting it right will provide clear signals for new investment or for withdrawal of coal plants as flagged by speculation over the future of the Hazelwood power station in Victoria.

Josh Frydenberg, as the new minister for the environment and energy, and his fellow ministers on the COAG Energy Council would be unwise to waste a near crisis.