What GAO Found

U.S. Citizenship and Immigration Services (USCIS) created a reliable updated estimate to project the Transformation Program's cost, but has experienced program management challenges. In particular, the program's cost estimate was well-documented and substantially comprehensive, accurate, and credible. However, among other things, software development and systems integration and testing for USCIS's Electronic Immigration System (USCIS ELIS) have not consistently been managed in line with the program's policies and guidance or with leading practices.

Regarding software development, the Transformation Program has produced some software increments, but is not consistently following its own guidance and leading practices. The software development model (Agile) adopted by the USCIS Transformation Program in 2012 includes practices aimed at continuous, incremental release of segments of software. Important practices for Agile defined in program policies, guidance, and leading practices include ensuring that the software meets expectations prior to being deployed, teams adhere to development principles, and development outcomes are defined. The table below lists the program's status in addressing eight key Agile development practices. For example, the program has committed to a specific framework for software development, referred to as Scrum, but has deviated from the underlying practices and principles of this framework.

Table: Implementation of Key Agile Practices for USCIS ELIS

Practice Rating Completing planning for software releases prior to initiating development and ensuring software meets business expectations prior to deployment ◐ Adhering to the principles of the framework adopted for implementing Agile software development ◐ Defining and consistently executing appropriate roles and responsibilities for individuals responsible for development activities ◐ Identifying users of the system and involving them in release planning activities ◐ Writing user stories that identify user roles, include estimates of complexity, take no longer than one sprint to complete, and describe business value ◐ Prioritizing user stories to maximize the value of each development cycle ● Setting outcomes for Agile software development ○ Monitoring and reporting on program performance through the collection of reliable metrics ◐

Source: GAO analysis of USCIS documentation. I GAO-16-467.

Note: ● yes ◐ partial ○ noc

The Transformation Program has established an environment that allows for effective systems integration and testing and has planned for and performed some system testing. However, the program needs to improve its approach to system testing to help ensure that USCIS ELIS meets its intended goals and is consistent with agency guidance and leading practices. Among other things, the program needs to improve testing of the software code that comprises USCIS ELIS and ensure its approaches to interoperability and end user testing, respectively, meet leading practices. Collectively, these limitations have contributed to issues with USCIS ELIS after new software is released into production.

Why GAO Did This Study

Each year, USCIS processes millions of applications for persons seeking to study, work, visit, or live in the United States, and for persons seeking to become a U.S. citizen. In 2006, USCIS began the Transformation Program to enable electronic adjudication and case management tools that would allow users to apply and track their applications online. In 2012, to address performance concerns, USCIS changed its acquisition strategy to improve system development.

In May 2015, GAO reported that USCIS expected the program to cost up to $3.1 billion and be fully operational by March 2019. This includes more than $475 million that was invested in the initial version of the program's key case management component, USCIS ELIS, which has since been decommissioned.

This report evaluates the extent to which the program is using information technology program management leading practices. To perform this work, GAO identified agency policy and guidance and leading practices in, among other things, cost estimation, Agile software development, and systems integration and testing, and compared these with practices being used by the program.

What GAO Recommends

GAO is making 12 recommendations to improve Transformation Program management, including ensuring alignment among policy, guidance, and leading practices in areas such as Agile software development and systems integration and testing. DHS concurred with the recommendations.

For more information, contact Carol C. Harris at (202) 512-4456 or chac@gao.gov.