Written on June 28, 2017

It’s possible to use AI to detect faces, emotions and facial expressions

Machine Learning and Deep Learning is now being used to detect emotions and facial expressions by analyzing images and videos. Here’s what you need to know.

Machine Learning and Deep Learning are a growing and diverse fields of Artificial Intelligence (AI) which studies algorithms that are capable of automatically learning from data and making predictions based on data. Machine Learning and Deep Learning are two of the most exciting technological areas of AI today. Each week there are new advancements, new technologies, new applications, and new opportunities. It’s inspiring, but also overwhelming. That’s why I created this guide to help you keep pace with all of these exciting developments.

Facial Expression Research

Convolutional Neural Networks (CNNs) are leading the Computer Vision (CV) industry in achieving state-of-the-art facial expression (emotion) recognition.

Here are my favorite facial expression academic papers, sorted by release date:

Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset [Tripathi et al. 2017]

— achieved 81.4% Valence Classification Accuracy (VCA) and 73.4% Arousal Classification Accuracy (ACA) on DEAP dataset using CNNs (Convolutional Neural Networks) DAGER: Deep Age, Gender and Emotion Recognition Using Convolutional Neural Networks [Dehghan et al. 2017]

— achieved 76.1% accuracy on a private four million image dataset using CNNs (Convolutional Neural Networks) Spatio-Temporal Facial Expression Recognition Using ConvolutionalNeural Networks and Conditional Random Fields [Hasani et al. 2017]

— achieved 93% on CKI+, 78.7% on MMI, and 66.7% accuracy on FERA datasets using a two-part network consisting of a DNN-based architecture followed by a Conditional Random Field (CRF) module Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units [Dachapally 2017]

— achieved 86.4% accuracy on JAFFEE dataset using CNNs (Convolutional Neural Networks) Facial Expression Recognition using Convolutional Neural Networks: State of the Art [Pramerdorfer et al. 2016]

— achieved 75.2% accuracy on FER-2013 dataset using CNNs (Convolutional Neural Networks) Facial Expression Recognition from World Wild Web [Mollahosseini, et al. 2016]

— achieved 82.12% accuracy on World Wild Web Dataset [dataset to be released soon] DeXpression: Deep Convolutional NeuralNetwork for Expression Recognition [Burkert et al. 2015]

— achieved 99.2% on CK+ dataset and 98.6% accuracy on MMI dataset using CNNs (Convolutional Neural Networks)

Facial Expression Datasets

Most emotion recognition research papers rely on relatively small image datasets. A larger image dataset will improve performance and accuracy of CNNs (Convolutional Neural Networks), the common algorithm used to solve this computer vision problem.

Here are my favorite facial expression datasets, sorted by release date:

Kaggle Competitions

There are a few Kaggle data science competitions that you can reference for facial key point and expression recognition.

Here are my favorite facial expression Kaggle competitions, sorted by release date:

Facial Keypoints Detection [Kaggle, 2016] Challenges in Representation Learning: Facial Expression Recognition Challenge [Kaggle, 2013]

Kaggle In-Class Competitions

Kaggle competitions are not limited to industry or private companies. Many universities and colleges now use Kaggle-style competitions to push students to new levels.

Here are my favorite facial expression Kaggle In-Class competitions, sorted by release date:

Emotion Recognition Challenges

Want to push your data science skills and deep learning skills? Consider participating in international challenges.

Here are my favorite facial expression challenges, sorted by release date:

EmotionNet Challenge [Website; Paper, 2017] EmotiW: Emotion Recognition in the Wild Challenge [Website, 2015]

Facial Expression APIs

Why build machine learning models yourself when you can leverage existing solutions? APIs allow you to implement artificial intelligence technologies quickly and easily. Software developers can now implement emotion recognition within minutes.

Here are my favorite facial expression APIs, sorted by title: