Face detection is one of the vision-focused features Firebase’s ML Kit offers (or more correctly, facilitates). It’s a feature that can be useful in many applications: tagging photos, embellishing selfies, adding emojis and effects to a camera feed, taking pictures only when everyone is smiling with their eyes open, etc. The possibilities are endless.

Despite this, implementing a face detector in an Android app still takes effort and a lot of head scratching. One would need to understand how the API works, what sort of information it provides, how to process this information and make use of it given the device’s orientation, the camera source, and which camera is in use (front or back).

Given a camera source, ideally we’d be able to write something like this and be done with it.

camera.addFrameProcessor { frame ->

faceDetector.detectFaces(frame)

}

The main components are in bold—before going through each of them, let’s assume our layout contains a camera view and an overlay view on which we’ll draw the boxes around the detected faces.

Camera

Regardless of which camera API we use, what matters is that it offers a way to process its frames. This way, we’ll be able to process each incoming frame, detect the faces in it, and identify them to the user (i.e. by drawing boxes around them on the overlay, for example).

Frame

A frame is the information given by the camera to the face detector. It should contain everything the face detector needs in order to detect faces. This needed information is defined below: