IBM’s chief engineer has revealed the simple trick that could have avoided the national online Census fail altogether.

IBM Australia managing senior engineer Michael Shallcross said turning the router “off and on again could have detected the problem earlier, which could have avoided the 40-hour shutdown.

“If we had our time again we would have tested a hard power it off, power it on that router, that would have discovered earlier that we had that reboot and configuration loading problem,” he said.

Special Advisor to the Prime Minister on Cyber Security Alastair MacGibbon said one of the biggest issues was the Telstra router had been incorrectly coded by IBM.

“When it was turned off, the coding fell out,” Mr MacGibbon said.

“It made it a dumb unit ... had the router been properly configured and when it was turned off fired back up again we wouldn’t have had that problem.”

IBM and the Australian Bureau of Statistics (ABS) officials faced a grilling in front of a Senate committee in Canberra to explain what went wrong with the bungled national Census.

ABS head statistician David Kalisch opened his evidence with an apology to the Australian people for the inconvenience of the Census website shutdown.

IBM chief engineer claims router needed a "hard reset" to avoid Census issues IBM chief engineer claims router needed a "hard reset" to avoid Census issues

“The ABS has tested the patience of many households,” Mr Kalisch said before laying blame on IBM.

The IBM system, he said, should have been “robust” enough to withstand the events of August 9.

He then blamed Bill McLennan, who gave evidence earlier, for the “misunderstandings” that arose about privacy.

“The ABS of today is more complex and advanced than the ABS of 15 year ago,” he said.

On a positive note, Mr Kalisch said the ABS “will have high quality Census data”.

More than 96 per cent of households contributed data — 4.9 million online and 3.5 million in paper form.

Some 10,500 Australians have refused to complete the 2016 census, a decrease on the 13,000 people who refused in 2011.

Mr Kalisch promises the 2021 Census will adopt a “more rigorous approach” following the lessons learned from this year’s one.

“Ultimately the ABS has responsibility for the Census”, Mr Kalisch said.

Mr Kalisch said there were about 40 meetings between the bureau and relevant ministers in the lead up to the Census night. But there was never any concerns raised about the bureau’s preparedness.

He said the bureau first met with the minister newly in charge of the Census, Michael McCormack, two weeks before August 9. In that meeting, they just discussed the “nature of the Census process”.

It followed ABS boss Bill McLennan saying the Bureau was not authorised to collect people’s names on a compulsory basis.

“We believe our approach was consistent with the policy,” Mr Kalisch told the committee.

Mr MacGibbon confirmed his review of the 2016 Census had been completed and submitted to the Prime Minister’s Office on October 14.

Mr MacGibbon said he understood IBM were the natural choice to deliver on the project, but said he was concerned there was a degree of “vendor lock-in” associated with the ABS’ decision.

“I wouldn't be before you today if those risks (relating to vendor lock-in) weren’t real,” he said.

He said the Commonwealth “was not well served” by IBM in terms of the service delivery.

MacGibbon said the incident was an “attack” rather than a “hack” that was not dealt with effectively.

“It certainly should not have caused the damage it did,” he said.

CENSUS BUNGLE A ‘MALICIOUS ATTACK’

Senator Xenaphon also asked whether Mr Kalisch acknowledged he had coined the Census failure to be the result of a “malicious attack”.

“There was certainly a DDOS event, and those were the words I used in the news conference we had at about 10.30 in the morning,” Mr Kalisch said.

But Mr Kalisch said he couldn’t recall the words he used earlier that morning. He said he had no recollection of using the words “malicious attack”, but confirmed there had been a “malicious event”.

“It certainly was a malicious event and it was a DDOS event. It wasn’t a benign attack, if I can draw that comparison,” he said.

The statistician was also grilled on whether he consulted relevant ministers prior to making the decision to shut the Census down.

Mr Kalisch said he had advised Mr McCormack and Treasury secretary John Fraser after the site was shut down.

“As I say this is something the ABS has control over. It was not a ministerial decision,” he said.

IBM SORRY FOR CENSUS FAIL

Kerry Purcell, managing director of IBM in Australia and New Zealand, earlier said the company too full responsibility for the Census fail.

He offered an “unreserved apology” to the Australian public and the government as “a valued customer”, adding the shutdown “does not sit well with us”.

Mr Purcell assured the hearing “no personal information was compromised”.

He was adamant the site was not “hacked”.

Instead “it was unavailable to the public for a period of time but nothing more”.

Mr Purcell said it’s “akin to someone parking a large truck in front of your driveway ... Not someone breaking into your house and taking your goods”.

He refused to comment further on who the culprits were as IBM was assisting the AFP in its investigation.

When asked what went wrong, Mr Purcell said: “In short, the geo-blocking protocol was not properly applied by one of the ISPs.”

Like an electronic fence, geo-blocking prevents traffic identified as coming from outside of Australia from accessing a site.

With the Census, the ‘Island Australia’ protocol is the term used by IBM to describe how traffic from overseas should have been blocked.

He said IBM was ready to relaunch the Census website after three hours but it remained offline for 40 hours at the request of the ABS.

IBM OFFERS TO PAY COMPO

IBM were given the ABS Census contract, which was worth $9.7 million. The value of the IBM contract for the 2011 Census was $9.2 million.

Mr Purcell said IBM offered to pay compensation of any extra costs incurred on the night of August 9.

Asked how much IBM was prepared to offer the Commonwealth he said the matter was “commercial in confidence” at the moment.

He said no IBM staff were sacked or reprimanded over the incident.

IBM Australia managing senior engineer Michael Shallcross said lots of traffic from overseas — the majority from Singapore — indicated the ‘Island Australia’ protocol should have been activated.

He confirmed the fourth attack at 7.47pm on August 9 brought down the Census site. That attack came from Singapore.

Mr Shallcross said the first action was to restart the website and clear the queues but the attack traffic was still coming in and they were unable to restart the site.

The team then initiated a restart of two IBM routers linked to two ISPs — one was located in Singapore. One started correctly, but the second did not.

Asked if they would do anything differently if they had their time again, Mr Shallcross said he would make sure other contractors understood their role in the implementation of ‘Island Australia’.

Asked whether a third back up router would have prevented the crash, the IBM executive said “it would have been overwhelmed as well”.

IBM is standing by the view that geoblocking is an effective DDoS (a distributed denial of service) attack defence mechanism.

“The geoblocking approach was particularly well suited to the Census,” Mr Shallcross said.

But Special Advisor to the Prime Minister on Cyber Security Alastair MacGibbon said to rely solely on geoblocking to counter the attack was “clearly a failure”.

​

CENSUS: ‘I CRINGED WHEN I SAW IT’

Earlier, former ABS chief Bill McLennan opened his evidence by saying the “social contract” of the Census was broken when the ABS decided to ask people for their names.

He said he doubts it is covered by the relevant legislation.

Mr McLennan was asked about evidence submitted by the ABS that says it has been collecting people’s names since 1910.

“I personally think that is wrong,” Mr McLennan said.

“In our knowledge it’s always been collected on a voluntary basis.”

When asked about the 2016 Census public campaign he said: “I cringed when I saw it.”

The 2016 Census was controversial because it sought the names of all respondents, prompting concerns about privacy of information.

That apparently prompted a series of four cyber attacks on the Census website which led to it being shut down.

ROW OVER CENSUS ATTACK CONTINUES

The blame game over the national survey continues within IBM, with the contractor facing a spat with its own subcontractors over the August 9 distributed denial-of-service (DDoS) attacks.

A DDoS attack is not a hack where someone breaks past network security to access data. It simply floods serves with incoming messages, overloading it which prevents it from operating properly.

The fourth DDoS attack which struck the Census website on the evening of August 9 was foreign-sourced and came when IBM had already directed NextGen that geo-blocking was to be put in place, it claims.

“Had NextGen (and through it Vocus) properly implemented Island Australia, it would have been effective to prevent this DDoS attack and the effects it had on the eCensus site,” IBM said in its submission to the committee.

Vocus denies the fourth DDoS attack caused the site to become unresponsive.

“The fourth attack comprised of attack traffic which peaked at 563Mbps which is not considered significant in the industry, and lasted 14 minutes ... such attacks would not usually bring down the Census website,” its submission read.

The cause was IBM workers falsely identifying normal traffic patterns as data exfiltration.

“Vocus was not informed of IBM’s DDoS mitigation strategy, Island Australia or its specific requirements, until after the fourth attack.”

Census website crashes due to foreign hackers Census website crashes due to foreign hackers

Nextgen says it wasn’t privy to “Island Australia” until July 20, just six days before the eCensus site went live.

Also giving evidence to the inquiry will be Alastair MacGibbon, the special adviser to the prime minister on cyber security.

MacGibbon, who is conducting a review of the events, hasn’t yet finalised his findings. But he has already concluded there was a failure in the geo-blocking service during the fourth denial-of-service attack.

Simultaneously a monitoring system indicated there was outbound traffic from the website, which was feared to be malicious and is now known to have been a “false positive”.

In its submission, the ABS said the attacks should not have been able to disrupt the system.

In Senate estimates committee hearings last week, Australian Bureau of Statistics chief statistician David Kalisch acknowledged it made a number of poor judgments ahead of the Census. He revealed the shutdown cost taxpayers up to $30 million.

However, the ABS remains confident the census will still produce high-quality data, despite the problems.

— with AAP