The US government is challenging researchers to use cameras to ID people by unique features like their gait or the shape of their ears

Not just a face in the crowd (Image: Chris Schmidt/Getty)

CAMERAS are strewn around our environment, catching glimpses of our faces everywhere we go, yet even the best facial recognition technology still has a hard time picking us out of the crowd.

So the US government’s Intelligence Advanced Research Projects Activity (IARPA) has called for a new approach. The agency announced a contest on 8 November, challenging teams of the country’s top researchers to revolutionise how machines recognise people. Those entering the competition already know that conventional facial recognition won’t cut it.

The usual approach to identifying people is to sift through camera footage frame by frame, find a few that offer the best chance of an ID, and then attempt to match them to a database of known images. Ideally, this will mean the subject is facing the camera, with a neutral expression, and without any shadows on their face. All other frames are discarded, and only then do the facial recognition algorithms get to work.


“One of the goals of the IARPA challenge is to see what you can do with the discarded data,” says Mary Ann Harrison of the West Virginia High Technology Consortium Foundation in Fairmont.

That data contains “soft” biometrics, information which does not explicitly identify a person, but narrows the range of possibilities. This could be height, size, gait or other features.

“Our main focus has been ear recognition,” says Harrison. “The evidence is that the structure is unique to each person. There is a whole science of the structure of ears.” People can be categorised according to details such as whether the ears have lobes, or ear size in comparison to the head.

Bir Bhanu at the University of California, Riverside, who leads another team, says this type of work will result in faster, more flexible tracking that will help law enforcement after events like the bombing of the Boston Marathon in April. Partial shots of the suspects’ faces, as well as their gait would have been abundant from CCTV cameras, but it took an army of trained professionals several days of poring over footage to unearth their images.

There are other applications too. Airport security could be streamlined to allow passengers to walk freely from check-in to the gate, their movements monitored and identities verified automatically by cameras.

“Ultimately the goal is to be able to recognise a person in natural motion through any scene,” says Jack Ives of machine vision company, CyberExtruder in Newark, New Jersey, who stresses that the benefits are not only for the military and government.

“A department store or bank could have a system which is able to recognise each person as they approach the counter,” he says. “Wouldn’t it be nice to walk in and get the service you want without having to say a word.”

This article appeared in print under the headline “Soft surveillance”