Face recognition: realtime masks development

Internal hackathon project story in Akvelon company

Akvelon is a custom software development company specializing in the development, testing, consulting and support of web, cloud, desktop and mobile solutions.

Akvelon company official logo

About hackathon

In the beginning of December 2017 one of the offices of Akvelon company in Russia, Kazan city held a hackathon event named HACKVELON. Ideas for the event were collected during 2-weeks and reviewed by office leads. Each developer had an opportunity to vote for the project he/she liked and participate in it. There were only two limitations - using Cutting-edge technologies and team size (max 5 people on one project).

I’ve started my research of interesting popular technologies, frameworks, mobile applications and found a lot of articles with reviews related to realtime masks on Snapchat and Facebook. For me it was absolutely new area and a big challenge to create something interesting using such technologies on one week.

Trending AI Articles:

Problem

People care about their look. A bad haircut is something that can disappoint everyone.

Idea

Development of the mobile application for clients of barbershops for haircut, mustache, beards selection using real-time masks which gives an ability to SEE your haircut on your face before haircutting started.

The user selects a hairstyle that he/she likes from our set and the application substitutes the mask in real time video stream using the smartphone’s front camera. We called it PocketBarber. Let me show you how it’s became a reality…

PocketBarber app logo

Face recognition

The facial recognition process normally has four interrelated phases or steps. The first step is face detection, the second is normalization, the third is feature extraction, and the final cumulative step is face recognition.

Steps in the facial recognition process

Deep Learning (using multi-layered Neural Networks), especially for face recognition, and HOGs (Histogram of Oriented Gradients) are the current state of the art for a complete facial recognition process.

Research

Google Vision API — works very slowly on facial landmark detection (determined on demo application), support offline mode, not free Azure Face API — needs Azure account, offline mode not supported, not free Augmented Reality frameworks: ARToolKit, ARCore, etc. — commercial SDK issue, no documentation on a real time masks development, possible on development but could take much more time than one week Dlib + OpenCV — free for commercial use, big community, supports iOS and Android platforms

Note: Our team decided to use open source libraries to commercialize the project in future.