Well, you can say one thing about the Australian Energy Market Commission’s release of its annual market performance review: They got the headlines they were looking for.

“Renewables put grid under stress”, cried The Australian.

““Unprecedented’: Energy operator in daily fight to keep lights on”, said the Sydney Morning Herald, and The Age.

“Stop gap measures propping up the power grid,” said the Australian Financial Review.

It even got a tweet from Nine’s Chris Ulhmann.

Yet another report details the many challenges of integrating large amounts of variable renewable energy on to the electricity grid. Voltage. Frequency. The forced curtailment of wind power. Directing SA thermal generators to switch on in at a cost of $34m https://t.co/flugb5ejH1 — Chris Uhlmann (@CUhlmann) April 4, 2019

And these stories all had a common theme. The influx of “intermittent” and “asynchronous” wind and solar was causing massive headaches for the Australian Energy Market Operator. It’s a miracle they manage to keep the lights on.

And that’s probably a fair reflection of the media release and five page summary distributed by the AEMC to journalists two days ago – under a “strict” embargo and on a “seek no response” basis. In other words, the journalists who were given the document under embargo were not supposed to ask anyone else what they thought of the report.

Clever. That’s the way things often work nowadays when you are looking to craft a particular message.

The problem is that those headlines – and the reporting within them – don’t reflect the depth and breadth of the actual report itself, put together by the AEMC’s Reliability Panel, and which in its 300 and something pages goes into great detail about the issues facing the grid.

Those problems are numerous. And they are broad. But the role of renewables is just one part of it. The report notes – in great depth – AEMO’s concerns about the reliability of the country’s thermal coal fleet, as highlighted in the biggest wave of blackouts that occurred last August and their failures in this January’s load-shedding.

Frequency control , and the lack of it, is a major issue with the coal fleet, and even in hydro-dominated Tasmania, and with the BassLink interconnector. There are also problems with the transmission and distribution networks.

The fragility of the coal fleet is further highlighted by this graph pointing to the increased risks of forced outages in the country’s thermal fleet, which played a key role in the January load shedding that captured so many headlines. Black coal generators are three times as likely to trip in coming years. So too are gas generators. More outages are expected in all gas and other coal categories.

There are also issues with network constraints and network strength, and marginal loss factors. These are all issues that have arisen from a lack of proper planning. Regulators have allowed networks to spend so much money making the grid bigger, but not smarter.

But none of this made the media release prepared by the AEMC, or the summary. They gained not a single mention in the lengthy opinion piece published in The Australian by Anna Pearson, the chief executive of the AEMC.

Those who have been paying attention over the last few years are not surprised. This is not so much the technology disaster the AEMC would have us believe, so much as a regulatory and cultural failure. The AEMC has rarely missed an opportunity to paint renewables in a bad light.

As much as anything, the details of this report are evidence the regulators and rule-makers in this market have simply not been able to keep up with the technology changes, partly because they have refused to believe that such changes were inevitable or that they would emerge as quickly as they have.

As many have pointed out, including the head of China’s State Grid, the biggest challenge in the clean energy transition is not technology, but culture. Some people refuse to believe it can be done.

The AEMC’s reliability review highlights numerous rule changes and market initiatives that could be adopted to solve the problems at hand now. The 5-minute settlement rule is just one example.

This is a rule that will encourage battery storage and other fast-response technology, to replace the clunky and slow moving generators accused of using the 30-minute period to game the market, and many of which are now all but useless useless in a crisis, as they proved last August in a major transmission failure.

It was the AEMC that at first refused to entertain the switch to 5-minute settlements, and then – when they did – impose an unconscionable delay in its introduction: it won’t come in until mid 2021.

Like many proposals, particularly the “demand side” options that regrettably are barely touched on in this report, the AEMC has been slow to act.

Partly, people suggest, because of its own ideology, and partly because of the immense pressure from incumbents – many the same organisations who sit on the reliability panel – who fought the changes because of the risks it put to their own business models.

And the relatively scant attention in this report to the possibilities of demand side management is, to be frank, a scandal.

The AEMC has to get serious about this. The NEM continues to operate as a supply only market, despite demand always having been half of the equation – and despite so many submissions that urged the AEMC to pull its head out of the sand.

Rather than seeing demand response as an expensive “market intervention”, demand management needs to be treated as equal to supply in the market. It would lower costs to consumers, and the peaks. It would also mean that energy efficiency gets to have a market value and we get to a better balance all round.

To its credit, the AEMC has belatedly had its “oh, shit” moment, and understood that it has to move quickly to try and keep up, even if it does want to fire a pop-shot over the bows of the wind and solar revolution on the way and is still stalling on demand response.

It even recognises, deep in the report, the power of rooftop solar to provide significant grid services, such as reducing the levels of peak demand in heatwaves, as it did in Queensland last summer.

It now claims it is fast-tracking efforts to make the rules of the market reflect new technologies, and not the ageing clunkers that are on the way out. The AEMC is now trying to convince people it is on the case.

So, what of the AEMC reliability panel report? Well, it doesn’t actually tell us that much that we didn’t already know, because it is largely a summary of various AEMO reports, including its Integrated System Plan, the Electricity and gas statements of opportunities, and other detailed studies.

The AEMC does make a really big deal of the fact that AEMO is now having to intervene on a daily basis to “keep the lights on” in South Australia – and the mainstream media seized on this.

But here’s the thing. This is not so about technology, but market practices, and the behaviour of the fossil fuel incumbents. AEMO has to intervene because it reckons it needs a certain amount of thermal generation to keep the system secure (around 300MW if there is more than 1295MW of wind, with some variations).

The problem is, the thermals in that state, the gas and diesel generators, don’t like to run their expensive machinery unless they are getting paid a lot of money. And wind and solar is bringing down the price.

So there is a big cat and mouse game going on. The gas generators wait until AEMO tell them to switch on their plant – that way, the gas generators get a more generous “intervention” price. As more wind and solar comes into the grid, the less the gas plants want to operate, and the more that AEMO needs to direct.

It is just a game, but it will soon be over.

The state’s transmission company, ElectraNet, is installing four “synchronous generators” in key parts of the state. When they are switched on in the next 12 months, deliver significant cost savings and another pricing party will be over for the gas plants. AEMO won’t need the gas generators so much.

These directions have been about “system strength”. And soon they won’t be needed. Has AEMO had to intervene for the sake of reliability? No, says the report: “During 2017/18 no directions or instructions to market participants were issued to maintain the system in a reliable operating state,” it says

Were there any losses from un-served energy? No, says the report. What was the share of “reliability” events in supply interruptions? 0.29 per cent. (Just to be clear, that is, nought point two nine per cent).

And this is the remarkable thing. For all the AEMC’s hand-wringing about the influx of wind and solar, and the consumer switch to rooftop installations and batteries, which state is likely facing the least problems in terms of reliability?

If the events of last August are anything to go by, it will be South Australia. The state that has already surpassed a 50 per cent share of wind and solar, and is charging towards 60, 70, and 80 per cent renewables, and will likely be generating more than 100 per cent of its demand from wind and solar within a decade.

It is also because it has a strong grid. Yes, it has issues, and emerging ones, particularly when the amount of rooftop solar pushes minimum demand to negative territory, something that could happen within three or four years.

But these problems can be addressed and resolved, particularly if the regulators and rule-makers are not asleep at the wheel. And South Australia has already become home to innovation and adaption.

It had the first big batteries, the first wind farm that can provide frequency response, and it will have the first major virtual power plants and hydrogen electrolyser that can boost stability and reliability to the grid. It should still get the first large scale tower plant with storage, and it has business – such as the Whyalla steel-works genuinely excited about the shift to renewables.

South Australia is also succeeding because the previous government got tired of market-makers holding its grid to ransom and went about ensuring its own security – most notably through the Tesla big battery, or the Hornsdale Power Reserve.

Unlike the big lumbering coal and gas generators, the battery – as the AEMC notes – does what it is supposed to do. It is fast, accurate, and flexible. It holds the grid together, as AEMO noted in that major load-shedding that hit other states in August, and in other events.

More than, it points to the future that the AEMC for so long refused to see. The new Liberal government in South Australia says it is very comfortable with the pace of transition that will see the state become a significant leader in the switch to the clean energy future.

But there’s a lot to be done, particularly in recognising the value of these new technologies. As AEMO has noted, batteries only get payment for one third of the services they can offer to the grid.

That’s a failing of the rule makers to keep pace. There are many other examples. Within two decades, this “consumer led revolution” will see more battery storage in homes and the widespread adoption of electric vehicles, nearly half of all the market will be supplied by distributed resources installed by consumers.

Can the AEMC catch up? Or will it just continue its pursuit of cheap headlines?