Just do as it does PR Michel Zanca, ISM /Science Photo Library

ARTIFICIAL intelligence is taking image recognition tips from a real expert: the human brain.

Using fMRI brain activity scans as a training tool has boosted the ability of machine learning algorithms to recognise objects. The technique could improve face recognition systems or help autonomous vehicles better understand their surroundings.

Machine learning is still a long way behind humans when it comes to tasks like object recognition, says David Cox at Harvard University. So his group trained algorithms to process images more like we do.


They analysed how regions of the brain’s visual cortex responded to images containing four different types of object: humans, animals, buildings and food. The data came from a volunteer who viewed more than 1200 images while an fMRI machine measured their brain response.

The different objects had their own corresponding pattern of brain activity, and the strength of the signals indicated how difficult each image was to classify.

The team used this information to train its machine learning algorithms. If an algorithm made a mistake on an “easy” image, it was more heavily penalised than if it made erred on a “difficult” image. This feedback essentially told the system what information it should base its classifications on to minimise errors. As a result, it performed better on images easily recognised by the brain, effectively making decisions in a more human-like way (arxiv.org/abs/1703.05463).

“If an AI makes mistakes that a human would make, humans will continue to trust that system”

Training a basic image classifier with the fMRI data improved its accuracy by 10 to 30 per cent across the different categories, says team member Walter Scheirer at the University of Notre Dame, Indiana. The method could allow relatively basic machine learning models to approach the accuracy of state-of-the-art neural networks, he says.

Algorithms that make decisions in a similar way to us could also be easier to understand and trust, says Cox. Computer systems sometimes make mistakes that humans wouldn’t – like Tesla’s Autopilot system failing to notice a white trailer against a bright sky. Systems trained on brain data would make mistakes in a more human way. “And if you make mistakes that a human would make, humans will continue to trust that system,” says Cox.

“These are preliminary results, but they’re impressive and they suggest that this is a new line of work that could be really fruitful,” says Thomas Naselaris at the Medical University of South Carolina in Charleston.

Next, the researchers will look at how cells in rat brains react to different images. The hope is that understanding how the brain works at this level could lead to systems that more closely mimic human decision-making. “The really exciting stuff is going to come from looking at the fine-grain detail of how individual cells are connected and how they’re firing,” says Cox.

This article appeared in print under the headline “Brain scans aid machines to see more like us”