What if I told you that we can now identify babies who are going to develop autism based on a simple brain scan? This, in essence, is the seductive pitch for a study published last week in the journal Nature, and making headlines around the world.

Early identification and diagnosis is one of the major goals of autism research. By definition, people with autism have difficulties with social interaction and communication. But these skills take many years to develop, even in typically developing (i.e., non-autistic) children. Potential early signs of autism are extremely difficult to pick out amidst the natural variation in behaviour and temperament that exists between all babies.

A brain scan for autism would be a major step forward. But is the hype justified? Are we really on the brink of a new era in autism diagnostics? Without wishing to detract from the efforts of everyone involved in the study, it’s important to look at the results critically, both in terms of the scientific findings and their potential implications for clinical practice.

The study, led by Heather Cody Hazlett at the University of North Carolina, was part of a larger research program investigating the development of babies who have an older sibling with autism. Because autism runs in families, these babies are much more likely to develop autism than babies from the general population.

The babies were given MRI brain scans at 6, 12, and 24 months of age and were then assessed for autism. As expected for this “high risk” sample, around 1 in 5 met the diagnostic criteria. The researchers were then able to look back at the brain scans to see if there were any differences between the autistic and the non-autistic babies.

Hazlett and colleagues first looked at three measures of overall brain size: the total volume of the brain; its total surface area; and the average thickness of the cortex (the brain’s outer layer). The figure below shows growth curves for individual children. Consistent with previous studies of older children, the autistic babies (in red) had slightly larger brain volume and greater surface area. However, these effects were only statistically significant for the last scan at 24 months.

Brain growth curves for individual children. Photograph: Jon Brock

Importantly, the figure also shows considerable variation in brain size in both groups. Most autistic infants had brains that wouldn’t set them apart from non-autistic infants. In other words, overall brain size isn’t in itself a very good predictor of whether or not an individual baby will go on to an autism diagnosis.

So Hazlett and colleagues tried a different approach, calculating the volume and surface area for 78 different regions within each infant’s brain. They did this twice: once for the 6 month scan and again for the 12 month scan, giving them 312 datapoints, or “features”, for each baby.

Next, they fed that information (plus the sex and skull volume of each baby) into a computer that they trained to differentiate between the autistic and non-autistic babies. Importantly, they only trained it on 90% of the babies at a time. They then fed in the brain features from the remaining 10% and asked the computer to predict the diagnosis of each baby. They did this 10 times, leaving out a different subgroup of babies each time.

The results shown in the figure below were encouraging. The computer correctly diagnosed 30 of 34 autistic babies in the sample and incorrectly flagged just 7 of 145 non-autistic babies. So the excitement is understandable.

Of 34 babies with autism, 30 were correctly identified. False positives occured for 7 out of 145 non-autistic babies. Photograph: Jon Brock

However, as the researchers themselves note, the study really needs to be replicated. Because it was a first-of its-kind, the researchers would necessarily have been feeling their way, making decisions as they went along. This tweaking inevitably biases the outcome towards a more compelling result. But having learnt the lessons from this first study, researchers are now in a position to preregister any replication attempt, nailing down all the details before they begin. If the current results are robust, they should replicate even without the tweaking.

Assuming the results do hold up, the next big question is whether this approach actually translates to real life clinical applications. Will we really see the everyday use of MRI scans to predict whether or not babies have or will develop autism?

An important practical consideration is the requirement for brain scans to be acquired at both 6 and 12 months. MRI scanners are noisy and claustrophobic. Any movement and the scan is ruined. The researchers scanned the babies while they were asleep but, despite their best efforts, only around half of the babies had two useable scans. Once we add the babies with incomplete data to the picture, the results start to look less useful. In particular, only 30 of the 70 autistic babies in the study could be identified based on their brain scans.

Including babies with incomplete data, only 30 out of 70 babies with autism were correctly identified. Photograph: Jon Brock

As a final point, the use of MRI scans for autism detection is unlikely to be of much practical benefit beyond high-risk populations. This is simply an issue of numbers. In the general population, it’s estimated that one person in 68 has autism. In the figure below, I’ve assumed that the computer maintains the same ability to differentiate between autistic and non-autistic brains but is now faced with 67 non-autistic babies for every one autistic baby.

Assuming an estimate of 1 in 68 people having autism, in order to identify the 30 babies with autism in the original sample, we would need to scan a total of 4760 babies. Photograph: Jon Brock

The figure suggests that, in order to identify our 30 autistic babies, we’d need to scan a total 4760 babies (on two occasions each). We’d still miss the other 40 autistic babies. And because of the scaling up, 132 non-autistic babies would incorrectly test positive. In other words, 81% of babies who tested positive would not actually be autistic.

These are, of course, inexact back-of-the-envelope calculations. The computer algorithm may perform much better when it is trained to differentiate between autistic and low risk babies. And there are, no doubt, ways of improving the success rate of scanning. But it illustrates the profound challenges in translating the research finding into widespread clinical practice. For now at least, it’s time to dial back the hype. We are still a very long way from autism diagnosis by brain scan.

But from a scientific point of view, I remain excited by these findings. They’re part of a growing body of evidence for subtle differences in the brains of young infants who go on to be diagnosed with autism. Some of these findings are perhaps more robust than others, but each represents an important step towards a greater understanding of the developmental origins of autism in the brain.