The researchers say their findings cast doubt on whether the model should be used for decisions around hospital resourcing. Credit: Pixabay

An international group of renowned statisticians from the University of Sydney, Northwestern University and the University of Texas have collaborated to fully investigate the predictive performance of the COVID-19 model developed by Institute for Health Metrics and Evaluation (IHME) – which provides forecasts for ventilator use and hospital beds in the United States.

Published on pre-publication server arXiv, the researchers found that the IHME model substantially underestimates the uncertainty associated with COVID-19 deaths.

Seventy percent of US states had an actual death rate outside the 95 percent prediction interval for that state, casting doubt on whether the model is suitable to inform COVID-19 resource allocation.

The model, which provides forecasts on a state-by-state basis across the United States, has been circulated widely by the media and on social media, and has informed policy decisions at the highest levels, having been cited at a White House press conference on 31 March 2020.

“The discrepancy between predicted deaths and the actual death rate in the US has serious implications for the United States’ government’s future planning and provision of ventilators, PPE, and the staffing of medical professionals equipped to respond to this pandemic,” said University of Sydney statistician and Director of the Centre for Translational Data Science, Professor Sally Cripps.

“The level of uncertainty implied by the model casts doubt on its usefulness to drive the development of health, social, and economic policies,” she said.

Professor Martin Tanner from Northwestern University also said the model did not have the capacity to make long term projections.

“I am concerned that if the UW-IHME model has had difficulty in predicting the next day, how will the predictions fair over the long term,” said Professor Tanner.