Wiro J Niessen is Professor of biomedical image analysis at Erasmus MC in Rotterdam and Delft University of Technology, and Director of the biomedical image analysis platform of the European Organization for Imaging Research. For the past 20 years, he has been working to integrate AI in radiology. He believes the technology can help radiologists in countless scenarios from automating image analysis to picking up disease at pre-clinical stage and predicting treatment outcome. But a number of obstacles must be overcome before it fully enters clinical practice.

Myriad applications for AI in radiology

One of Niessen’s key responsibilities has been to apply AI to support the Rotterdam study, a population cohort that was launched 15 years ago at Erasmus MC, and in which thousands of images have been generated to collect information on disease development. Niessen has been using machine learning (ML) for the past 15 years, first to automate the analysis of large imaging data sets, and now to find patterns with a prognostic value.

“We started to make the clinical researchers’ and clinicians’ jobs easier by automating the extraction of quantitative information from imaging data, which is tedious and time consuming to do by hand.

“Now we also try to predict from a CT or MRI scan whether someone is going to respond to a certain therapy or if they are likely to at an early stage in neurodegenerative disease. If you collect a lot of data and follow the patients over time, you see whether they develop a disease or not, and whether they respond to therapy. You can always dive back into the data and see whether there are patterns that can predict disease,” he said.

Niessen, who talked to Insights at the European Congress of Radiology in Vienna, is involved at every stage of AI technology development, from research all the way up to commercialisation. He notably founded the Erasmus MC spin-off company Quantib, which brings solutions that were initiated in research to the market.

For a long time, he has only used conventional ML, first selecting relevant features in an image, and then assessing their diagnostic or prognostic value. In oncology imaging, that means for instance capturing the boundaries of a tumour and computing features such as tumour texture, volume, intensity patterns and density, to build classifiers that can help in diagnosis and prognosis. Niessen’s group has worked with common ML tools such as support vector machines, decision trees and random forests.

Deep learning hype is set to continue

For the past four years, his group has been increasingly working with deep learning (DL), to train systems to perform tasks on their own by optimising efficient networks. DL originates from the vision field, but breakthroughs like neural network architecture or convolutional neural network can perfectly be adapted to medicine, Niessen explained. “This is very much a field in which you can benefit from all these advances. In daily clinical care and research, we are collecting a large number of cases and data, and it is now our challenge to exploit this wealth of information with AI techniques,” he said.

DL has very much become a trend in medical imaging and the industry is increasingly presenting new solutions. The hype is set to continue in the near future, as more and more DL-based technology will emerge, Niessen predicted.

One important aspect is generalisation, he said. “You need to make sure that the dataset on which you train the algorithm is representative of the environment in which you employ it. For example, population studies provide large opportunities to design proper algorithms for both ML and DL, and therefore are very good to understand preclinical stage of disease,” he said.

Obstacles to clinical integration

Clinical practice is also a minefield of data ready to be exploited by AI. In prostate cancer, ML could help to know cancer severity and grade, and to base a prognosis on MR images and potentially spare the patient a biopsy, Niessen suggested. “In oncology, it would be of great benefit if we could predict disease progression or therapy response based on imaging data alone. It’s already being used as an early screening and surveillance tool.” Besides cancer, DL can also help to detect white matter lesions in the brain, with the algorithm learning from examples.

Conversely, not many of the ML or DL tools developed in research are ready for clinical practice yet. “A lot of the developments are still in the research stage. Integration into clinical practice is still a challenge,” Niessen said. Perhaps even up to 95% of information used in daily practice is still qualitative information, for instance the radiology report, Niessen estimated. But things are changing in that aspect as well. “Step by step it’s getting better, and structured reporting is becoming more important in radiology, to provide more objective information. If efforts continue in that direction, more data will be available to train ML and DL algorithms, and the structured report could increasingly be filled by algorithms. We’re still in the early days,” he said.