Australia's welfare agency Centrelink is embroiling the government in a hugely damaging controversy, raising incorrect debts and auto-issuing threatening letters to citizens.

The “pay up” letters are genuinely terrifying when the sums involved are often thousands of dollars, especially when even government officials admit that the “error rate” is as high as 20 per cent.

It's pitched as an IT story, and there is an IT story there – but it's not the mainstream “Centrelink IT failure” it's presented as. Rather, it demonstrates that the government's blind faith in big data analysis is completely misplaced.

The agency was given a political instruction after the 2016 budget: claw back money from welfare recipients in what governments typically pitch as a “crackdown on cheats”.

(That language ensured that the Australian Labor Party, the country's national opposition, would not have the backbone to oppose the so-called “omnibus savings bill” that implemented the crackdown.)

Part of the crackdown – a part repeatedly trumpeted by the government as a success – was to use “data matching” to identify “cheats”. If someone received a Centrelink payment, the income they reported would be cross-checked automatically with the Australian Tax Office; if there was a discrepancy, a debt would be created representing the alleged “overpayment”, and a letter sent to the recipient.

And so, with much preening by government ministers like Christian Porter, the crackdown commenced, 160,000 Australians received their “debt letters” from Centrelink, and all hell broke loose.

When all of this first emerged, it looked like an IT story, and for good reason: the government is famous for chronic incompetence in managing IT systems; the hapless IBM is in charge of Centrelink's core systems; those systems are positively ancient and need replacing; IBM was the village idiot that ruined the 2016 Census; and people assumed that if Centrelink is creating incorrect debts it's because the IT system's a dog.

In a desperate search for IT people with credibility on the topic, The Guardian got Paul Shetler, former boss of the Office of Eternal Beta (not Google, the Digital Transformation Office) to give it an interview.

It's not an IT story, because the computer systems, as geriatric as they are, are calculating exactly what they're asked to calculate.

The “errors” are very simple to understand:

If you earn over a set threshold in a fortnight, your benefit is reduced. Earn enough, and the benefit should be zero;

The time unit is, however, a fortnight, but the Australian Tax Office doesn't care about fortnights, only years;

Centrelink's developers divided the ATO earnings data by 26 because they were told to write the software that way ;

; When “yearly / 26” yielded an amount over the benefit threshold, a debt was created and recipients could only change this by challenging the calculation, with records to back them up, as much as six years back.

This isn't a bug in an IT system: it's an executive giving developers instructions to implement a malicious system.

Before this blew into a social media sensation, Centrelink already knew that averaging annual earnings isn't a good guide to point-in-time entitlements.

With thanks to Twitter user @drzax – the ABC's Simon Elvery – we find this in the Department of Human Services' online manual explaining debt calculation:

Assessing verified employment income where fortnightly details cannot be obtained

“Employment income for workforce age customers should be assessed for the fortnight the income is earned. Sometimes it is not possible to determine a fortnightly breakdown and the only means to assess the income is as an annual amount …”

“If employment is for a part of a year only, averaging over 12 months will not result in a correct result if the customer should have received a full rate at other times of the year”

(Emphasis added.)

The government admits that 20 per cent of the “debts” are incorrect, it persists in treating its estimated savings as already realised. Christian Porter – social services minister, handling the government's attack-dog role while human services minister Alan Tudge enjoys his holiday – says the system has “saved” the government AU$300 million.

The whole thing is incompetence at the top. The devs did what they were ordered to do – and probably hated doing it.

The IT angle

That said, there is a government IT angle – it's just not a Centrelink IT fail.

Since before he was prime minister, Malcolm Turnbull has consistently promised that cross-agency data-sharing will improve government services, deliver efficiencies, and all the usual shiny stuff Big Data believers promise.

Part of the rationale for the Office of Eternal Beta – including its frankly-astonishing $90 million-plus annual budget – is to create a single digital identity for Australians. This is central to cross-agency data matching.

The reason the Australian Bureau of Statistics wants to keep names and addresses? To become the repository for national-scale data matching.

The reason the Australian government looks to New Zealand for its welfare models? Because it believes “big data” will help it slash the welfare spend.

However, the complete mismatch between just two agencies – the Australian Tax Office and Centrelink – shows that blind faith and political blather don't translate into results.

In the case of the ATO/Centrelink data matching, it's a simple matter of timescales: the ATO's heart beats once a year, Centrelink's once a fortnight, and in spite of what Turnbull (apparently), Tudge and Porter believe, they are completely incompatible.

And that's just two agencies: citizens' data touches dozens of agencies and probably hundreds of systems.

Each of those systems will have a view of the citizen that's relevant to the interaction between agency and individual.

It's no wonder the political dreams have borne so little fruit: they're borne of a simplistic view of the world. It's time the government listened to people with an understanding of operational detail, rather than visionaries. ®