With the total number of people affected by the data breach at the Office of Personnel Management now estimated to be as many as 18 million, OPM Director Katherine Archuleta has mounted a public relations counter-attack, defending the agency's efforts to improve security during her tenure and crediting those efforts with finding the malware at the heart of the breach in the first place. But the news of the exposure has caused a wave of fear and distrust among federal employees—with some who work in the intelligence community now concerned for their families' safety.

Archuleta defended her tenure before a Senate hearing on June 23. "I'm as angry as you are that this is happening," she said in a message to federal employees and retirees during her testimony. "I am dedicated to ensuring that OPM does everything in its power to protect the federal workforce, and to ensure that our systems will have the best cyber security posture the government can provide.” And she insisted that no one at OPM was to blame for the breaches, saying, "If there is anyone to blame, it is the perpetrators."

Archuleta also acknowledged for the first time that the breach was at least partially related to the breach last year of an OPM investigative contractor, KeyPoint. Attackers used credentials stolen from a KeyPoint employee to access OPM's network initially, gaining access to the EPIC background investigation software tools.

Today, OPM e-mailed an eight-page document outlining OPM's "Actions to Strengthen Cybersecurity and Protect Critical IT Systems" to members of the media. In the document, OPM officials asserted, "Upon Director Archuleta’s arrival, OPM engaged in an end-to-end review of its IT systems and processes. Based on that review, the agency developed a Strategic Plan for Information Technology to guide its efforts to protect its legacy systems to the maximum extent possible as it replaced them with more modern and secure systems. This plan laid out a multi-phase strategy to bolster security through realignment of professional staff, adherence to relevant laws, policies and best practices, and investments in modern tools."

The OPM statement also promoted how much the agency was doing right.

"In an average month, OPM thwarts millions of...confirmed intrusion attempts targeting our network," the OPM spokesperson wrote. And Archuleta and OPM should get credit for effort, the spokesperson noted, because "It was only because of OPM’s aggressive efforts to update its cybersecurity posture, adding numerous tools and capabilities to its network, that the recent cybersecurity incidents were discovered."

Breaking the law, breaking the law

But according to an October 2014 OPM Inspector General report, issued a year after Archuleta took over at OPM, the agency's adherence to relevant laws, policies, and best practices at that point was severely lacking. Systems related to EPIC and support of other OPM applications had long been operating without essential security certification required under the Federal Information Security Modernization Act of 2014 and its predecessor, the Federal Information Systems Management Act (FISMA). The Office of the Inspector General called on Archuleta to shut the systems down until they were given official "Authority To Operate" (ATO), because they posed a risk to national security. Continuing to operate them was, essentially, breaking federal law.

"All Federal agencies are required to complete annual information security audits to comply with FISMA laws/regulations," Vinny Troia, director of risk and security consulting at McGladrey, told Ars. "In order for any system to operate on a federal network, it MUST have an Authority To Operate. An ATO is granted on a per system basis following the completion of a successful FISMA audit. And information systems under FISMA are also required to re-certify every three years. If an agency does not re-certify (or does not pass re-certification), that system loses its Authority to Operate."

Archuleta chose to let the systems continue to operate based on a promise from OPM CIO Donna Seymour that efforts were underway to bring the unauthorized programs into compliance over the next fiscal year. But within a month of the publication of the Inspector General report, the systems had already been compromised by attackers who had penetrated OPM's network, stealing user and administrator credentials and using them to tap into millions of sensitive records.

Why could a federal agency essentially break the law and continue to run systems that put information about employees—including those working in the intelligence community—at risk? As Troia said, at OPM, "There was no consequence for systems breaking the law." The OPM Inspector General report specifically cited the lack of any consequences for not complying with FISMA as a contributing cause to delays in getting the systems up to specifications. And the reason there were no consequences was because the persons responsible for deciding what consequences would be for breaking the law were Archuleta and Seymour.

It's not a huge surprise that Archuleta did not shut down EPIC and other systems that were out of compliance with the law—EPIC is essential to OPM's whole background investigation system, and shutting it down would have caused epic delays in processing new requests for security clearances and determinations of whether contractors and potential federal employees met "suitability" standards for access to federal facilities. And, in fact, it's fairly common for systems across the federal government to run without ATO for months, or even years, based on waivers given by agency leadership despite federal regulations. Plus, EPIC had enough problems without OPM having to worry about getting the program's security audit done—the whole system is in turnaround.

Michael Esser, the OPM Assistant Inspector General who led the 2014 security audit, told the Senate Appropriations Committee that not only had EPIC and the other systems not been audited, but that Seymour had ordered a freeze on system audits until all modernization work on systems was done. And while the Office of Management and Budget has suspended requirements for reauthorization every three years for systems with "mature continuous monitoring", he added, none of the programs at OPM fit that description.

EPIC trouble

The EPIC system has been in flux for some time. In 2010, OPM launched a modernization program called "EPIC Transformation"—the modernization effort for EPIC's mostly mainframe-based components. The program issued partial releases and pilots of new capabilities for the system in 2012 and 2013. But there were serious problems with the program, and it was "rebaselined" in February of 2014—sort of the project management equivalent of declaring bankruptcy and starting over.

The contract for the coding work given to Dynanet of Elkridge, Maryland, was extended in March of 2014. To get a feeling for why the project was rebaselined, here's how a Dynanet spokesperson described the work in progress as of last March under EPIC Transformation: "Specifically, Dynanet will implement the EPIC target architecture incrementally, first by transforming the Natural language programs and implement imaging as a service, followed by replacing Adabas by transforming the data management layer to point to a relational database; and finally modernizing the remaining components and automating business processes."

According to OPM procurement documents, OPM's strategy for EPIC Transformation is to move key parts of the system from the Software AG Natural and Adabas-based technology that much of the system has been based on to a collection of Java "portlets" and Web services based on Oracle WebLogic. OPM already has a significant Oracle installed base, with Oracle databases running on the agency's IBM mainframes. The blending of mainframe and modern web elements is a complicated and potentially dangerous task, especially from a security perspective.

"The worst things that happen to software happen when different technologies start talking," said John Chang, director of worldwide product marketing at software development management tools vendor CAST Software. "Some of the old legacy programs, they work surprisingly well, and are efficient for their original purpose. But when you add new technology on top, problems happen." That's an issue both at government and commercial companies that CAST does business with, Chang noted, "but we do see that there is some 'duct-taping' of integration at federal agencies we work with."

High anxiety

This is a common issue with federal IT programs, which are often designed to minimize risk by leveraging existing systems as much as possible. But the irony is that in doing so, the projects are often more risky, because the integration required to bridge the gap to older systems frequently fails or works so poorly that it makes applications incapable of carrying out basic tasks. In the meantime, agencies are forced to apply "duct-tape" solutions to keep things running—and managers are put in a position where their careers are only secure if they tell subordinates to shut up about breaking security regulations.

In the wake of the OPM breach, a number of current and former federal employees and contractors have spoken to Ars about the resulting cultural and professional dysfunction of federal IT security that results from those issues. One federal IT employee told us that when he "tried to report security issues and mismanagement of IT systems, [his managers] proposed to remove me from the federal government and then placed me on paid leave for over a year." He said he has learned "not to speak up or blow the whistle," and that the situation is so bad at his agency that he has considering quitting "out of frustration."

But the anger and frustration over the OPM breach may, at least, generate a bit of a change in view regarding security compliance. If anything, it has created an army of advocates for change that could number 18 million strong.