During a hearing held yesterday by the House Oversight Committee, Committee Chairman Darrel Issa said that it was “unbelievable” that the IRS had lost the e-mails of former IRS official Lois Lerner. While Congressman Issa is not generally ignorant on tech issues, he’s clearly not familiar with just how believable such a screw-up is.

The IRS claims that many of Lerner's e-mails were lost when the hard drive on her desktop computer crashed in 2011. In a Monday night hearing, IRS Commissioner John Koskinen told Issa and the Oversight Committee that there was no way to recover these e-mails. “If you have a magical way for me to do that," he told Issa, "I’d be happy to hear about it.”

The IRS is not the only federal agency to lose e-mails over the past few years. In fact, despite efforts at many agencies to standardize and improve e-mail by moving to services like Google Apps for Government and Microsoft Office 365 Government, many agencies still run their e-mail like it’s 1999. It’s not just a technology issue—it’s an IT policy issue, a staffing issue, and a cultural issue within government, one that the federal government shares with many private corporations.

Mailbox options

Exchange and Outlook (at least as the IRS implemented them until 2011) are at the heart of the IRS e-mail fiasco. Prior to the Lerner e-mail episode, the agency’s IT department placed a 150MB limit on each users’s mailbox (today, IRS employees have a 500MB cap on their e-mail). To keep mailboxes within size limits, employees were told to archive e-mails locally in Outlook .PST mailbox files—including messages that they thought met the definition of “official correspondence” for retention purposes.

To make sure those important messages were preserved and reviewed for retention, IRS employees were told to print out and file a hard copy of those messages. Lerner printed out at least some of her e-mails, so she appeared to be following this policy at some level.

As contrary to common sense as "printing out e-mails" may sound, it’s an entrenched part of government workflow, particularly with more senior officials. Some career government managers and military officers I’ve dealt with actually have administrative assistants print out their correspondence for them and place it into physical inboxes.

Before the whole e-mail controversy exploded, the IRS e-mail servers were regularly backed up—but the agency’s IT department recycled those backup tapes every six months, thus wiping out older records that had since been deleted.

Bad backups have resulted in congressional controversy before—the Bush Administration lost millions of e-mails from its first three years because the White House recycled backup tapes. More recently, the Army—which began migrating its soldiers and civilian employees to a Department of Defense cloud-based e-mail system in 2010—lost a Medal of Honor nomination in e-mail, largely because the workflow for handling awards used a shared mailbox that e-mails were removed from as they arrived and stored in local .PST files.

Local storage is always risky because predicting how long a set of hard drives will last on a desktop or laptop is something of a crapshoot to begin with. A study by Backblaze found that 22 percent of hard drives fail within four years, with half failing within six. In 2011, the last major hardware upgrade the IRS had made to laptops and desktops was a buy from Hewlett-Packard in 2002, so desktop drives were likely dying like flies.

The IRS also had a contract with Sonasoft to provide document retention services for the IRS’ Counsel Division. Some reports have claimed that the contract included archiving of IRS e-mail. But a company spokesperson said that Sonasoft never had access to IRS e-mail—backup of e-mail was all in the hands of the IRS User and Network Services department and its team of contract IT workers.

Bad history

Some will argue that the IRS could have easily prevented the loss of records with regular enterprise desktop backups and that it’s inconceivable that a government agency would have such incompetent IT management practices. To which the only response is: yes, the IRS has lots of IT management problems. For starters, the IRS missed the deadline to get off Windows XP and is paying Microsoft for extended support while it completes a migration to Windows 7. Back in April, Koskinen told Congress that the IRS needed another $30 million to finish the upgrade to Windows 7—money that it was diverting from the IRS' tax enforcement budget.

A Treasury Department audit last year found that the IRS wasn’t properly performing even fundamental system management functions like software license management for desktops and laptops—something that was potentially causing the IRS to spend millions of dollars unnecessarily. “During the audit period, the IRS did not have an overarching approach for enterprise software governance,” wrote Michael McKenney, the Office of the Treasury Inspector General for Tax Administration (TIGTA) acting deputy inspector general for audits, in a memo attached to the report.

According to TIGTA’s findings, the IRS was “not adhering to Federal requirements and recommended industry best practices. The IRS does not have enterprise-wide or local policies, procedures, and requirements for software license management. The User and Network Services organization was unable to provide us with essential licensing records for properly managing licenses on 24 of 27 software products reviewed during this audit.”

It doesn’t help any that the IRS budget has been cut further and further over the past four years. Despite a pledge by the Obama administration to reduce reliance on contractors, a significant portion (if not a majority) of the day-to-day IT operations and support at IRS are in the hands of contractors hired under the IRS’ broad Total Information Processing Support and Services (TIPPS-4) contract. Much of the IRS’s management of its support services has been passed off completely to contractors under small business set-asides. Meanwhile, the IRS is seeing a huge turnover in staff, as a significant percentage of its workforce retires and is not replaced.

These are problems agencies across the federal government are facing—lack of internal IT expertise, dependence on contractors, reduced money to pay for contractors due to the ongoing budget wars, and the mass departure of baby boomer employees. Combine that with “best practices” that have long passed their shelf life, and you have a recipe for more than just a few missing e-mails; you have a blueprint for total chaos, of which missing e-mails are only a single symptom.