The energy sector was supposed to be the showcase for privatisation and market deregulation. Yet in 2017, this premise is being sorely tested – no more so than in electricity retailing, where competition has failed to deliver on its promise of lower prices for customers.

The latest Grattan Institute report, Price shock: Is the retail electricity market failing consumers?, provides evidence that in the electricity retail sector, the anticipated price reductions have not happened, and innovation has been very slow in coming.

On the contrary, the markets with the least regulation have the highest prices. Australia’s experience is mirrored in the UK, the United States and Canada, and all are struggling to find solutions.

Mixed success

The privatisation of Australia’s electricity retail markets dates back to the 1993 Hilmer Report on national competition policy. The ensuing decade saw a raft of reforms that initially delivered increases in productivity, lower prices and business innovation. But in the decade after that, this progress became much harder to sustain.

The idea was for states to create regulated monopolies in electricity transmission and distribution (poles and wires), while deregulating the retail side (the supply of gas and electricity to customers).

The competition in electricity generation largely delivered lower wholesale prices through the National Electricity Market (NEM). But a mess has since been created by poor or absent climate policy at a federal level, which failed to match the enthusiasm of (some) states for clean energy. The resulting surge of investment in wind and solar happened without due consideration of the consequences for security and reliability of supply. Generating more renewable energy is essential, but failing to integrate it properly with the NEM was negligent.

Meanwhile, in Victoria – the state with most electricity retailers and the longest history of full competition – retail prices have been increasing without apparent justification and retail margins are higher than they should be. The cost to Victorians could be as much as A$250 million a year.

Lazy customers?

Customers are unhappy, and yet we are not seeing a surge of consumer action to get the best deals. So if it’s just a matter of lazy consumers, why should governments care?

Part of the answer to this conundrum lies with the product and its relationship with consumers. First, electricity is an essential service that underpins our daily lives, and switching off is not a realistic option for most consumers.

Second, the products offered by retailers are often complex and the advertising is confusing, if not downright misleading. It is hardly surprising that consumers feel stuck and eventually give up trying to find the best deal, when all too often an advertised 30% discount on their electricity bill doesn’t necessarily mean their bills will be 30% cheaper.

So far there have been few genuine innovations in electricity pricing – even in Victoria which has had full deregulation since 2009. The most common tactic has been a discount for paying on time or by direct debit, although consumers are often frustrated when they discover that at the end of their contract they lose the discount even if they continue to pay the same way.

Meanwhile, products that offer different prices for electricity use at different times of the day have been slow to appear. These products have the potential to deliver major savings, yet the industry has failed to deliver them in a way that makes them easy for customers to understand and adopt.

What to do

When faced with a market failure, governments should consider action. Yet, as with the Australia’s domestic gas market and South Australia’s power “crisis”, they should proceed with caution.

In Britain, the partial re-regulation of retail electricity competition delivered unexpected and perverse outcomes, such as the removal of the cheapest deals. A move to re-regulate prices here could stifle emerging innovation, and would most likely leave some consumers worse off without the guarantee of a better outcome overall. We seem to be driven to a choice between free markets and central planning. Yet neither is a panacea.

There are government interventions that could fix the worst problems without stifling effective competition. They include requiring clearer and simpler advertising, and more transparent and fairer contracts. Requiring retailers to provide data on their profit margins to an independent agency could also help, and could even be in the best interest of the retailers if it fends off more heavy-handed regulation.

The retail electricity market may be fixable, and the benefits of competition may ultimately exceed its costs. We may yet see fairer prices and real innovation. But if not, governments will have no choice but to return to price regulation. The electricity retailers who are used to the current free market certainly won’t want that.