There’s no way of knowing if the cashless debit card is having a positive impact due to poor evaluation, says an ANAO report — but the minister believes the report shows everything is “on track”.

It is impossible to say whether the government’s controversial cashless debit card trial has improved lives or saved money, according to the Australian National Audit Office.

“The Department of Social Services largely established appropriate arrangements to implement the Cashless Debit Card Trial, however, its approach to monitoring and evaluation was inadequate,” says the report, released Tuesday.

“As a consequence, it is difficult to conclude whether there had been a reduction in social harm and whether the card was a lower cost welfare quarantining approach.”

The cashless debit card trial started in December 2014, in the remote communities of Ceduna and East Kimberley, both of which have high Aboriginal populations. It works by “quarantining” 80% of a recipient’s payment on a special debit card that cannot be used on alcohol, gambling or to withdraw cash. The other 20% is deposited into a regular bank account as normal.

Participation is mandatory for all working age payment recipients who live in the selected trial sites. Wage earners, age pensioners and veterans pensioners can opt in.

The trial is to continue in its original locations up to the end of this financial year after being extended, and is being implemented in another location Western Australia, based on what the government called “strong independent evaluation results”.

Auditor General Grant Hehir isn’t convinced, however.

There were plans for analysis, monitoring and evaluation for the cashless debit card trial, but much of it was not completed, the auditor notes. This includes the planned cost-benefit analysis and post-implementation review.

The department also failed to properly measure baseline data, making it difficult to know what impact the trial had.

“Social Services did not build evaluation into the CDCT design, nor did they collaborate and coordinate data collection to ensure an adequate baseline to measure the impact of the trial, including any change in social harm,” says the report.

Key performance indicators did look at the effectiveness of the trial, but did not consider efficiency or operational aspects, ANAO says.

In addition, the costs of ORIMA Research, which carried out the evaluation, “were greater than originally contracted and ORIMA did not use all relevant data to measure the impact of the trial, despite this being part of the agreed evaluation framework.

But it wasn’t all bad news: the report also notes that “Social Services conducted an extensive consultation process with industry and stakeholders in the trial sites”, and that its communications strategy was “largely effective”.

The department has agreed to all six recommendations.

Everything is on track, says the minister

Minister for Social Services Dan Tehan was decidedly more upbeat, arguing the report “had confirmed the trials are on track”.

Tehan noted the auditor had found the department had “established appropriate arrangements for consultation, communicating with communities and for governance of the implementation of the Cashless Debit Card trial” and that “Social Services was responsive to operational issues as they arose during the trial”.

But he conceded ANAO had also “identified areas of improvement, which are being addressed”. As a result, the department has:

Appointed a chief evaluator;

Implemented a new data monitoring strategy, which uses card data to monitor program performance and identify issues;

Improved contract management, risk management and procurement practices.

Work has already begun on a second independent evaluation “that will satisfy the ANAO recommendations and fully address concerns around monitoring and risk management”, he stated.

“The government has extended the cashless debit card trials in Ceduna and East Kimberley and expanded to the goldfields, based on strong independent evaluation results and extensive consultation with local communities,” Tehan said.

“Independent evaluation by Orima Research found gambling, alcohol and drug consumption were reduced in Ceduna and East Kimberley. This finding was supported by feedback from service providers and the communities. “The cashless debit card trial is an important element of the government’s work to reduce welfare-funded social harm, and to help Australians escape welfare dependency. The cashless debit card is making a real difference in the communities where it operates. People are using the cards to pay for every day essential items such as food, clothing and energy bills instead of spending welfare money on alcohol, drugs and gambling.”

‘Flawed’ approach

West Australian Greens Senator Rachel Siewert called on the government to end this “flawed” approach.

“The government has extended the current cashless welfare trials sites and established a new site, and have plans for a fourth when they are unable to show the card has had a positive outcome,” she said.

“When the government first announced these ‘trials’, I continually pointed out that the baseline data was not being collected, nor was there a comparable site without the card, to actually gather evidence about the card’s effectiveness.

“Now the auditor general has unsurprisingly reported that the evidence base behind performance reports to the minister has been lacking. Communities have been through enough of this social experiment which the government cannot claim is working,” Siewert said.

“We need to be resourcing strong wraparound services for those struggling with drug, alcohol or gambling addiction in Australia. It needs to be treated like a health issue, time and again punitive measures like income management have failed in reducing harm and will continue to do so.”