Australia's National Electricity Market and power generators are struggling to come up with a coherent plan "to keep the lights on" due to policy and pricing limitations, according to a major independent study of the sector.

Key points: New capacity is being driven by construction and deal making rather than what is needed

New capacity is being driven by construction and deal making rather than what is needed Black coal has been replacing brown coal while solar and wind is pushing out gas and hydro

Black coal has been replacing brown coal while solar and wind is pushing out gas and hydro Price mechanisms have encouraged 'bare minimum' rather than reliable generators

The report compiled by industry analysts Global-Roam and Greenview Strategic Consulting found the "obsessive focus on fuel types" and an "us and them schism" between fossil fuels and renewable energy posed significant risks to a successful transition to a modern energy market.

The more than 600-page Generator Report Card is a deep dive into the 20 years of the NEM and every generator supplying the market.

The report was not sponsored by an organisation or interest group, nor does it take sides on fuel type or technology, but was prepared for commercial energy users and industry analysts.

The study's starting point is that 75 per cent of the "bulk" electricity grid in 2018 still came from the combustion of coal.

"Despite the Mandatory Renewable Energy Target [MRET] commencing 18 years ago, the NEM has moved from approximately 85 per cent [coal] in 2007 to its current level in a much slower manner than many people wish to see," the report said.

"The root cause of the increasing polarisation of the energy debate has been the structure of the incentives put in place by various governments over the past 20 years leading to a degradation of holistic systems thinking."

Despite wind and solar dominating the new capacity coming online in the past five years, the aggregate level of coal-fired generation in recent years has remained relatively steady.

Black coal has largely replaced closed brown coal plants, while wind and solar have displaced hydro and gas generation.

Black coal generation has replaced brown coal closures, while wind and solar have displaced hydro and gas ( Supplied: Global-Roam, Greenview Consulting )

Higher risk, higher prices

The commentary on the retirement of coal-fired plants has largely overshadowed looming problems in gas generation, the report found.

"For gas-fired generation, the prospect of reduced capacity factors and the potential for lower revenue than historical levels is likely to challenge their business model significantly."

Global-Roam managing director and report co-author Paul McArdle said the problem started when Australia "essentially contracted the domestic gas supply into the global LNG market".

"It quadrupled the size of the drain on Australian gas by shipping it out, offshore."

Mr McArdle said whichever way you look at it, the grid is moving into a higher risk environment and higher underlying prices.

"They [energy policies] have made it easy for new entrants to build wind and solar, which has been good for extra capacity, but not so good at delivering it in the form and location that is most needed."

"Some of developers have been very naive. Some have been former property developers looking for the next big thing. They've learnt a little, bought some land from a farmer next to a transmission connection and off they go."

Mr McArdle said it is vital construction and deal making is not renewable energy's main game, but part of an integrated energy supply.

NEM schism

State and parochial rivalries have subsided over the two decades of the NEM's operations.

However, the report has identified the new battleground as "anytime/anywhere energy", or wind and solar, versus "keeping the lights on services", or traditional synchronised generation from the big fossil fuel utilities.

The declining cost of wind and solar farms has made them the default choices for additional capacity, however the new generation is seldom integrated with "keeping the lights on services".

"We have structured market incentives in such a way that it promotes this increasing divergence. A rapidly increasing number of commercial businesses have 'followed the money' to what was disproportionate value due to scheme design," the report said.

The rapid build up of wind and solar generation in relatively remote locations has led to a number of problems such as the market operator, AEMO, effectively slashing their values by cutting the prices paid in a major recalibration of Marginal Loss Factors, or transmission losses. Transmission congestion resulting from largely their distance from the main energy markets has compounded the problem.

Almost all new supply — generators and batteries — being installed are inverter-based, meaning they cannot be plugged directly into the market they way traditional generators do with their turbines synchronised to the grid's frequency.

With no market mechanisms or government incentives in place, the Generator Report Card argues new entrants only have the incentive to install the "bare minimum" to keep costs as low as possible.

"Power systems require more than just energy and capacity," it noted.

Storage 'a toe in the water'

However, the study does not suggest more coal is the answer.

"It is not a rational scenario to replace this [aging coal-fired power stations] with new coal-fired capacity," it argued.

Battery storage is one solution, however it is still a tiny fraction of supply. Even existing pumped-hydro from the Snowy scheme accounts for less than 1 per cent of energy consumed in the NEM.

Currently battery storage is better suited to short, sharp bursts of energy or emergency responses, the report found.

Historically, storage facilities are net consumers of energy, with the value derived by shifting supply to when it is needed — but even then the market rewards have not been great.

"There is a glimmer of hope there with people putting in batteries, but it still just a 'toe-in-the-water' exercise," Global-Roam's Paul McArdle said.

"It make sense wind and solar farms should invest in some form of battery storage, but there is still a fair bit of commercial risk without greater incentives [to build them]," he said.

Overhaul needed

The NEM started in December 1998, growing out of a decade long federal/state government collaboration through the 1990s.

While it has served the generation and distribution of electricity well, the Global Roam-Greenview study found the policies of the past 20 years have created many unintended consequences.

"Policy makers would do well to remember that generators will respond to incentives provided to them — sometimes in ways that would not have been envisaged if viewing the policy altruistically. "The form of external incentives that have been layered on top of NEM operations has helped to created a landscape where a significant number of new entrants might look to provide 'anytime/anwhere energy' with scant knowledge, or focus on 'keeping the lights on services'. "This has created an environment where these new entrants belated become aware of, and impacted by, unseen risks — but there are also systemic risks being promoted for the system as a whole."

While the Report Card found the unavailability (in aggregate) of generation through 2018 was higher than 2017, overall the NEM's reliability was pretty much in line across the 17 years for which the data exists.

Power prices and generation costs: A quick explainer . Short run marginal cost (SRMC): These are essentially the fuel costs, the cost of the coal or gas that must be burnt (measured in megawatt hours of MWh), but also the wear and tear incurred in turning the unit on. Wind and solar don't have fuel costs.

Long run marginal cost (LRMC): This is an estimate of all of the costs incurred over the life time of an electricity generator divided by its expected production. This includes not just the fuel costs, but the annual fixed costs such as management, and maintenance, the cost of rehabilitating the site at the end of its life and the recovery of the capital investment. As well the costs include the annual finance costs. For wind and solar plants the capital investment is by far the biggest part of costs. National Electricity Market (NEM): Is a "gross pool" of power available to the market. All electricity produced, except behind the meter rooftop solar, has to be sold into the pool and all electricity consumed, except rooftop solar, has to be bought from the pool. Australian Energy Market Operator (AEMO): The regulators runs the market, estimating demand and running continually runs auctions for supply. Generators submit bids at which they are willing to supply into the pool, basically a quantity of power they are willing to supply at a price. Bid stack: AEMO arranges the bids in order of cost (the bid stack), lowest cost first, until the total supply quantity matches estimated demand. The price is set at the price of the last bid accepted. All successful bidders receive the price of the last bid accepted, not the price they bid in at. Half hourly prices set in the pool in this way can be volatile. Source: ITK, ABC News

However, the risks are mounting in the face of the massive energy transition that is underway.

Thermal plants are aging and are highly unlikely to be replaced by new coal plants.

The last coal-fired plant to be built at Kogan Creek is now 12 years old, the oldest Liddell is almost 50. It is slated for closure and it was unavailable for generation around 40 per cent of the time last year.

The NEM is becoming increasingly dependent on the weather, with the extremes of drought, heat and storms stressing generation.

Supply is being more geographically dispersed, transmission stretched and pricing policies are not helping in designing the most efficient and reliable system.

Changes in 'bid patterns' for power are seeing an increasing volatility and a concentration of either extremely low (below $0/MWh) or high (above $300/MWh) prices.

Occasionally "cheap" power may sound good for consumers, but they are bids from price-takers who find it either cheaper to keep plants going, or are happy enough to take whatever price is going — but average prices across the curve keep creeping up.

Then there are ever-present political and social responses to climate change, as well as technological change.

"We have focussed on what 20 years of history can tell us, so we can plan for next 20 years," Mr McArdle said.

"We are living in a period of grace when we still have synchronous generation and we need to plan for the future.

"The only thing for certain is there will be significant changes coming in the next 20 years."