Reproducibility is essential for scientific progress and engineering advances. Even so, many published computational results lack sufficient capture and description of companion information that would enable subsequent confirmation and extension of the results. Certainly, most scientists intend to publish correct results, but without sufficient rigor in computational processes and practices, risk is unnecessarily high that results will occasionally be wrong and will always be costly to confirm and extend.

The reasons for inadequate reproducibility are fundamentally matters of incentives and costs. In recent years, because of the availability of improved software platforms from GitHub, GitLab, Atlassian and others, and container environments such as Docker, the cost of capturing and describing the computing environment used to produce scientific results has dramatically decreased. Furthermore, new workflows and skill-building opportunities are available for those who are interested in improving their practice. What needs further attention is our incentive system.

In this presentation, we discuss efforts to improve computational reproducibility by fostering and promoting changes to our incentive systems. We talk about efforts to increase reproducibility expectations by publishers, funding agencies, employers and the broader computational scientific community. By improving incentives to produce reproducible results, providing recognition for those who lead the community and providing conduits for effective exchange of best practices, we can expect and make reproducibility indispensable.

Bio:

Mike Heroux is a Senior Scientist at Sandia National Laboratories, Director of SW Technologies for the US DOE Exascale Computing Project (ECP) and Scientist in Residence at St. John’s University, MN. His research interests include all aspects of scalable scientific and engineering software for new and emerging parallel computing architectures. He leads several projects in this field: ECP SW Technologies is an integrated effort to provide the software stack for ECP. The Trilinos Project (2004 R&D 100 winner) is an effort to provide reusable, scalable scientific software components. The Mantevo Project (2013 R&D 100 winner) is focused on the development of open source, portable mini-applications and mini-drivers for the co-design of future supercomputers and applications. HPCG is an official TOP 500 benchmark for ranking computer systems, complementing LINPACK. Mike's most recent interests are focused on improving scientific software developer productivity and software sustainability. He leads the IDEAS project, dedicated to engaging scientific software teams to identify and adopt practices that improve productivity and sustainability.