The Australian Competition and Consumer Commission (ACCC) has launched a public inquiry into the National Broadband Network (NBN). Already, Telstra and Optus have come forward, offering to compensate customers whose expectations of internet speed delivery were not met.

But it’s four years too late. The ACCC dropped the ball when it allowed Telstra, Optus and its competitors to launch their NBN services without specific rules concerning misleading and deceptive advertising, or an enforcement mechanism to hold the NBN company to account for connection delays, missing appointments and fault rectification.

Now we’re in a position where the problems with the NBN run deeper than what can be solved through an investigation or more monitoring. Bad decisions about the technology, lack of hard data on performance and a war of words by competing providers makes it hard to untie responsibility for the mess.

Read more: Like it or not, you're getting the NBN, so what are your rights when buying internet services?

The stated aims of the ACCC public inquiry are to determine whether NBN wholesale service standard levels are appropriate, and consider whether regulation is necessary to improve consumer experiences.

The data released this week by the ACCC Wholesale Market Indicators Report shows that:

NBN Co is supplying more than three million wholesale broadband access services

there are 26 NBN Co retail and wholesale internet service providers (ISPs - also known as “access seeker groups”) directly connected to NBN Co’s network

the top four ISPs are Telstra, Optus, TPG Group and Vocus.

Optimism bias, and how we got here

In behavioural economics, optimism bias refers to the phenomenon that we think markets are going to work out well even when there is no rational basis for supposing so. A similar optimism bias is often seen in infrastructure projects.

In the case of the NBN there were a few strategic opportunities to reduce the chances of market failure, but optimism bias by policymakers let the opportunities slip by.

And now we’re in the position where – in common with energy retailers, deregulated providers of vocational education and financial services firms – ISPs have taken internet consumers and their regulators for a ride.

Read more: 5G will be a convenient but expensive alternative to the NBN

In 2013, when the NBN provided a Special Access Undertaking (setting out prices and terms of access to ISPs) to the ACCC, there was a great opportunity to require objectively verifiable disclosure of wholesale operating speeds. However, the government allowed NBN Co to negotiate the service level agreement with ISPs directly, and left the ACCC to watch from the sidelines.

Similarly, there should have been truth in advertising rules imposed on Telstra, Optus and their competitors to prevent false service speed claims.

The absence of enforceable regulation was taken as a green light for ISPs to continue to promise unrealistic and unattainable internet speeds, as they had done for years before in relation to price and package data allowances. Connection speeds have become the new wild west in communications.

In the case of the NBN, the failure by the government and the regulator to require an adequate transparency measure, or even to monitor wholesale services standards has allowed the deterioration to the current state of affairs.

Not all is lost

In relation to false and misleading claims about internet service speeds, the ACCC has established a broadband monitoring program to generate data about broadband speeds and performance.

Under the scheme (which is based on similar ones operating elsewhere), the ACCC will install monitoring equipment in 4,000 homes across Australia that are connected to fixed-line NBN services – including fibre to the node, fibre to the basement, fibre to the premises, and hybrid fibre-coaxial.

FTTP = fibre to the premises; FTTN/FFTB = fibre to the node/basement; HFC = Hybrid Fibre-Coaxial

Real-time data will determine average fixed-line NBN speeds at various times, and is expected to be published at least in part by the end of 2017.

A trickier task will be to sort out the real culprit in cases of service failure or performance shortfall. At present we see a culture of passing the buck, where complainants to ISPs are shunted off to the NBN, while the NBN just as vigorously points the finger at ISPs.

NBN argues (with some justification) that the ISPs are scrimping on the amount of capacity or bandwidth they are buying to on-sell to consumers. According to NBN Co, they are being criticised for something that’s just not their fault. Sorting of this provisioning stoush will not be easy even with the panel data.

Read more: The NBN: how a national infrastructure dream fell short

Following the ACCC inquiry into the NBN’s wholesale service standards, possible regulatory changes would include imposition of fines, and establishing a process through which consumers could seek compensation at a retail level when minimum whole service standards are not met.

Finally, in an attempt to control deceptive claims about the speed of networks, the ACCC has issued Broadband Speed Claims – Industry Guidance. This advises retailers how to advertise speeds for NBN broadband services, including clearly identifying typical minimum speeds during peak periods.

It is unclear whether the proposed measures are intended to solve the problems or just to be seen to be doing something and so reduce the clamour.

Sadly, it’s something we could have avoided if policymakers and regulators were just a little less optimistic.