A group at Berkeley has just published (£) the first successful attempt to reconstruct colour video imagery from the mind using an fMRI brain scanner. The results are startling, encouraging and a little bit scary.

The method used, called fMRI, is known for its high spatial resolution (3D imaging ability) but notoriously low temporal resolution (measurements with respect to time – effectively a slow shutter speed). In the past, this has been a barrier to research on the visual cortex because of the incredibly high rate of information processing in the visual system. However, scientists have recently developed a new MRI encoding method that allows for the modelling of brain activity in the visual system at a faster rate. In the experiment conducted by the Berkeley group, participants were shown 7,200 seconds of random colour video clips one time only, while their brains were scanned using the novel fMRI sequence. From these scans, researchers were able to create a “dictionary” of brain activity in the visual system.

After a dictionary of brain activity in the visual system was created, the participants watched a fresh unseen video from YouTube while undergoing a brain scan. This resulted in video outputs that resembled the new YouTube video shown to participants. The output appears as a collage of flickering pixels that reminds me of a cross between the paintings of prosopagnosia sufferer Chuck Close and the imagery in A Scanner Darkly.

The correlation between the videos shown to the participants and the output imagery in the collage-like videos (below) is pretty astounding especially when considering there is zero overlap between the clips used for calibration and the clips used to test the system.

The study authors suggest that the method used in this paper could eventually lead to the generation of video output from participants experiencing dreams or hallucinations. Watch this space! What was once a field reserved firmly for science fiction may fast be becoming a reality.

Listen to an NPR interview with the researchers here:



Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, & Gallant JL (2011). Reconstructing Visual Experiences from Brain Activity Evoked by Natural Movies. Current biology : CB PMID: 21945275

Follow Simon on Twitter RSS , or join the mailing list