It was a comedy of errors at the Commerce Department's Economic Development Administration (EDA) last year, when employees destroyed $170,000 worth of IT equipment that they mistakenly believed was infected with malware - a move that ended up costing U.S. taxpayers in excess of $2.7 million.

In the end, only two systems out of 146 were actually infected, but that didn't come to light until after desktops, printers, TVs, cameras, computer mice, and keyboards were all destroyed, according to a June report from Commerce's inspector general, which is basically a master class in why you should closely read your emails.

In Dec. 2011, the U.S. Computer Emergency Response Team (US-CERT) notified Commerce's Computer Incident Response Team (CIRT) about a potential malware infection on the Commerce network. Commerce determined that the issue affected the Herbert C. Hoover Building (HCHB) network, which was used by EDA and the National Oceanic and Atmospheric Administration (NOAA).

NOAA checked it out, fixed the problem, and had its systems back up and running by Jan. 12, 2012. It was a different story over at EDA, however.

On Dec. 6, 2011, the DOC CIRT's incident handler requested network logging information. Unfortunately, that handler requested the wrong information, which it then passed along to EDA the next day.

"Instead of providing EDA a list of potentially infected components, the incident handler mistakenly provided EDA a list of 146 components within its network boundary. Accordingly, EDA believed it faced a substantial malware infection," the IG found.

Yes, the handler sent EDA a list of its systems, rather than a list of affected systems.

An HCHB employee actually recognized that the handler had provided EDA with the wrong information, and sent another email to EDA to tell them that only two - not 146 - systems were affected. According to the IG, however, that second email was "vague" and didn't clearly explain that EDA should discard the first email and rely on this second notice. As a result, EDA continued to think it had a major malware problem.

DOC CIRT later asked EDA to fix the affected systems, but - under the impression that the problem was on 146 systems - EDA told DOC CIRT that it couldn't handle such a big job. Rather than questioning this logic, though, DOC CIRT assumed EDA knew what it was talking about.

"DOC CIRT assumed EDA performed an independent analysis to identify additional infected components (even though EDA lacked the necessary capabilities) and assumed EDA was now dealing with a widespread malware infection," the IG report said.

As a result, systems were taken offline and equipment destroyed. EDA hired an outside cyber-security contractor to help contain the almost non-existent problem - the tune of $823,000, plus another $688,000 for "contractor assistance for a long-term recovery solution." EDA also paid $4,300 to destroy $170,000 worth of IT equipment, while the department spent more than $1 million to provide replacement equipment. All told, the incident costs $2.747 million - over half of EDA's FY 2012 IT budget.

That contractor, by the way, couldn't actually locate any persistent malware. Still, "EDA's management and CIO remained convinced that there could be extremely persistent malware somewhere in EDA's IT systems," prompting the property destruction.

That destruction, meanwhile, could have been a lot worse - to the tune of $3 million. After $170,000 worth of gadgets were destroyed, "EDA had exhausted funds for this effort and therefore halted the destruction of its remaining IT components, valued at over $3 million," the IG found. "EDA intended to resume this activity once funds were available. However, the destruction of IT components was clearly unnecessary because only common malware was present on EDA's IT systems."

Further Reading

Security Reviews