As computer-aided diagnosis gains popularity in medical imaging, the use of artificial intelligence (AI)-powered computer system frameworks like convolutional neural networks (CNNs) are following suit.

However, researchers at the Icahn School of Medicine at Mount Sinai in New York City found convolutional neural networks (CNNs) trained to detect pneumonia on chest x-rays performed poorly when tested on data from outside health systems, according to a study published online Nov. 6 in PLOS Medicine.

The research suggests that AI must be tested for performance across a wide range of populations and data sets to ensure accuracy in medical diagnosis.

“If CNN systems are to be used for medical diagnosis, they must be tailored to carefully consider clinical questions, tested for a variety of real-world scenarios and carefully assessed to determine how they impact accurate diagnosis,” first author John Zech, a medical student at the Icahn School of Medicine at Mount Sinai, said in a prepared statement.