A new technique developed by University of Toronto Scarborough neuroscientists has, for the first time, used EEG detection of brain activity in reconstructing images of what people perceive.

The new technique “could provide a means of communication for people who are unable to verbally communicate,” said Dan Nemrodov, Ph.D., a postdoctoral fellow in Assistant Professor Adrian Nestor’s lab at U of T Scarborough. “It could also have forensic uses for law enforcement in gathering eyewitness information on potential suspects, rather than relying on verbal descriptions provided to a sketch artist.”

For the study, test subjects were shown images of faces while their brain activity was detected by EEG (electroencephalogram) electrodes over the occipital lobe, the visual processing center of the brain. The data was then processed by the researchers, using a technique based on machine learning algorithms that allowed for digitally recreating the image that was in the subject’s mind.

More practical than fMRI for reconstructing brain images

This new technique was pioneered by Nestor, who successfully reconstructed facial images from functional magnetic resonance imaging (fMRI) data in the past.

According to Nemrodov, techniques like fMRI — which measures brain activity by detecting changes in blood flow — can grab finer details of what’s going on in specific areas of the brain, but EEG has greater practical potential given that it’s more common, portable, and inexpensive by comparison.

While fMRI captures activity at the time scale of seconds, EEG captures activity at the millisecond scale, he says. “So we can see, with very fine detail, how the percept of a face develops in our brain using EEG.” The researchers found that it takes the brain about 120 milliseconds (0.12 seconds) to form a good representation of a face we see, but the important time period for recording starts around 200 milliseconds, Nemrodov says. That’s followed by machine-learning processing to decode the image.*

This study provides validation that EEG has potential for this type of image reconstruction, notes Nemrodov, something many researchers doubted was possible, given its apparent limitations.

Clinical and forensic uses

“The fact we can reconstruct what someone experiences visually based on their brain activity opens up a lot of possibilities,” says Nestor. “It unveils the subjective content of our mind and it provides a way to access, explore, and share the content of our perception, memory, and imagination.”

Work is now underway in Nestor’s lab to test how EEG could be used to reconstruct images from a wider range of objects beyond faces — even to show “what people remember or imagine, or what they want to express,” says Nestor. (A new creative tool?)

The research, which is published (open-access) in the journal eNeuro, was funded by the Natural Sciences and Engineering Research Council of Canada (NSERC) and by a Connaught New Researcher Award.