The pupil* of the eye in a photograph of a face can be mined for hidden information, such as reflected faces of the photographer and bystanders, according to research led by Dr. Rob Jenkins, of the Department of Psychology at the University of York and published in PLOS ONE (open access).

The researchers say that in crimes in which the victims are photographed, such as hostage taking or child sex abuse, reflections in the eyes of the photographic subject could help to identify perpetrators. Images of people retrieved from cameras seized as evidence during criminal investigations could be used to piece together networks of associates or to link individuals to particular locations.

By zooming in on high-resolution passport-style photographs, Jenkins and co-researcher Christie Kerr of the School of Psychology, University of Glasgow were able to recover bystander images that could be identified accurately by observers, despite their low resolution.

To establish whether these bystanders could be identified from the reflection images, the researchers presented them as stimuli in a face-matching task. Observers who were unfamiliar with the bystanders’ faces performed at 71 per cent accuracy, while participants who were familiar with the faces performed at 84 per cent accuracy. In a test of spontaneous recognition, observers could reliably name a familiar face from an eye reflection image.

“The pupil of the eye is like a black mirror,” said Jenkins. “To enhance the image, you have to zoom in and adjust the contrast. A face image that is recovered from a reflection in the subject’s eye is about 30,000 times smaller than the subject’s face.” In the research, the whole-face area for the reflected bystanders was 322 pixels on average.

Forensics implications

High-resolution face photographs may also contain unexpected information about the environment of the photographic subject, including the appearance of the immediate surroundings, Jenkins explained to KurzweilAI.

“In the context of criminal investigations, this could be used to piece together networks of associates, or to link individuals to particular locations. This may be especially important when for categories of crime in which perpetrators photograph their victims. Reflections in the victims eyes could reveal the identity of the photographer.

“Also, around 40 million photographs per day are uploaded to Instagram alone, he pointed out. “Faces are among the most frequently photographed objects. Our study serves as a reminder to be careful what you upload. Eyes in the photographs could reveal where you were and who you were with.”

Although Jenkins did the study with a high-resolution (39 megapixels) Hasselblad camera, face images retrieved from eye reflections need not be of high quality in order to be identifiable, he said. “Obtaining optimal viewers — those who are familiar with the faces concerned — may be more important than obtaining optimal images.”

In addition, “in accordance with Hendy’s Law (a derivative of Moore’s Law), pixel count per dollar for digital cameras has been doubling approximately every twelve months. This trajectory implies that mobile phones could soon carry >39 megapixel cameras routinely.”

It would be interesting to see what hidden information is buried in law-enforcement (and other) photo archives — some of which could even help exculpate innocent persons.

*Technically, the image is reflected by the cornea.

Abstract of PLOS ONE paper

Criminal investigations often use photographic evidence to identify suspects. Here we combined robust face perception and high-resolution photography to mine face photographs for hidden information. By zooming in on high-resolution face photographs, we were able to recover images of unseen bystanders from reflections in the subjects’ eyes. To establish whether these bystanders could be identified from the reflection images, we presented them as stimuli in a face matching task (Experiment 1). Accuracy in the face matching task was well above chance (50%), despite the unpromising source of the stimuli. Participants who were unfamiliar with the bystanders’ faces (n = 16) performed at 71% accuracy [t(15) = 7.64, p<.0001, d = 1.91], and participants who were familiar with the faces (n = 16) performed at 84% accuracy [t(15) = 11.15, p<.0001, d = 2.79]. In a test of spontaneous recognition (Experiment 2), observers could reliably name a familiar face from an eye reflection image. For crimes in which the victims are photographed (e.g., hostage taking, child sex abuse), reflections in the eyes of the photographic subject could help to identify perpetrators.