First Facial Recognition Hack for Google Glass

Google’s new wearable computer, Google Glass, is bringing unheard of capabilities into new situations. A popular question that comes up on shows like TWiT.TV is about what’s going to happen once facial recognition technology reaches Glass. Well here’s the first hack for that!

My team wrote an app at a medical hackathon called MedRef for Glass. The app lets you find and create patient folders by voice, add photo and voice notes, view previous notes, and also find patient folders by facial recognition! Very exciting.

Some people I talked to said hospitals are full of very busy people, often with their hands full, working with a lot of information - so Google Glass making it wearable is especially looked forward to there!

Here is a video demo of the app:

The card it adds for finding/setting the patient being worked with by voice looks like this:

The contacts it adds for adding photo notes to the patient being worked with, or searching by facial recognition look like this:

Retrieved notes are a stack of everything added for that person:

Facial recognition results look like this:

The app is open source on GitHub. Facial recognition uses the Betaface web service. Uploading a new face to it looks like this:

final Mirror glass = MirrorClient.getMirror(credential);

List<Attachment> attachments = replyItem.getAttachments();

InputStream is = downloadAttachment(glass, attachments.get(0), credential);

Image trainingImage = new Image(is);

final ArrayList<Face> faces = trainingImage.getFaces();

final Face face = null != faces && faces.size() > 0 ? faces.get(0) : null;

Subject s = Dao.INSTANCE.addFace(notification.getUserToken(), face.getUID());



Checking a face against others looks like this:

Person probe = new Person();

probe.addUID(face.getUID());

List<Subject> subjects = Dao.INSTANCE.listSubjects(notification.getUserToken());

for (Subject s : subjects) {

float confidence = probe.compareWithUIDsForConfidence(s.getFaces());

s.setMatch(confidence);

}

Collections.sort(subjects);

boolean sentResult = false;

for (Subject s : subjects) {

SearchResultsCard.insert(request, credential, s);

sentResult = true;

}



Thanks for reading! In the future, on more powerful hardware and APIs, facial recognition could even be written to run all the time! What do you think the killer app is for facial recognition on Glass?