The Facial Action Coding System (FACS) refers to a set of facial muscle movements that correspond to a displayed emotion. Originally created by Carl-Herman Hjortsjö with 23 facial motion units in 1970, it was subsequently developed further by Paul Ekman, and Wallace Friesen. The FACS as we know it today was first published in 1978, but was substantially updated in 2002.

Using FACS, we are able to determine the displayed emotion of a participant. This analysis of facial expressions is one of very few techniques available for assessing emotions in real-time (fEMG is another option). Other measures, such as interviews and psychometric tests, must be completed after a stimulus has been presented. This delay ultimately adds another barrier to measuring how a participant truly feels in direct response to a stimulus.

Researchers have for a long time been limited to manually coding video recordings of participants according to the action units described by the FACS. This process is now possible to complete with automatic facial expression analysis. This saves vast amounts of time and money, as scoring no longer requires analysis of each frame by a trained researcher – the software simply does the work for you.

Below we have listed the major action units that are used to determine emotions. Roll your mouse over the image to start the movement!

Main Action Units

Action Unit Description Facial Muscle Example (Hover to Play) 1 Inner Brow Raiser Frontalis, pars medialis 2 Outer Brow Raiser (unilateral, right side) Frontalis, pars lateralis 4 Brow Lowerer Depressor Glabellae, Depressor Supercilli, Currugator 5 Upper Lid Raiser Levator palpebrae superioris 6 Cheek Raiser Orbicularis oculi, pars orbitalis 7 Lid Tightener Orbicularis oculi, pars palpebralis 9 (also shows slight AU4 and AU10) Nose Wrinkler Levator labii superioris alaquae nasi 10 (also shows slight AU25) Upper Lip Raiser Levator Labii Superioris, Caput infraorbitalis 11 Nasolabial Deepener Zygomatic Minor 12 Lip Corner Puller Zygomatic Major 13 Cheek Puffer Levator anguli oris (Caninus) 14 Dimpler Buccinator 15 Lip Corner Depressor Depressor anguli oris (Triangularis) 16 (with AU25) Lower Lip Depressor Depressor labii inferioris 17 Chin Raiser Mentalis 18 (with slight AU22 and AU25) Lip Puckerer Incisivii labii superioris and Incisivii labii inferioris 20 Lip stretcher Risorius 22 (with AU25) Lip Funneler Orbicularis oris 23 Lip Tightener Orbicularis oris 24 Lip Pressor Orbicularis oris 25 Lips part Depressor Labii, Relaxation of Mentalis (AU17), Orbicularis Oris 26 (with AU25) Jaw Drop Masetter; Temporal and Internal Pterygoid relaxed 27 Mouth Stretch Pterygoids, Digastric 28 (with AU26) Lip Suck Orbicularis oris 41 Lid droop Relaxation of Levator Palpebrae Superioris 42 Slit Orbicularis oculi 43 Eyes Closed Relaxation of Levator Palpebrae Superioris 44 Squint Orbicularis oculi, pars palpebralis 45 Blink Relaxation of Levator Palpebrae and Contraction of Orbicularis Oculi, Pars Palpebralis. 46 Wink Levator palpebrae superioris; Orbicularis oculi, pars palpebralis

Back to top

Head Movement Action Units

Action Unit Description Example (Hover to Play) 51 Head Turn Left 52 Head Turn Right 53 Head Up 54 Head Down 55 Head Tilt Left 56 Head Tilt Right 57 Head Forward 58 Head Back

Back to top

Eye Movement Action Units

Action Unit Description Example (Hover to Play) 61 Eyes Turn Left 62 Eyes Turn Right 63 Eyes Up 64 Eyes Down

Back to top

Emotions and Action Units

The Action Units described above show the different movements of facial muscles. Certain combined movements of these facial muscles pertain to a displayed emotion. Emotion recognition is completed in iMotions using Affectiva, which uses the collection of certain action units to provide information about which emotion is being displayed. For example, happiness is calculated from the combination of action unit 6 (cheek raiser) and 12 (lip corner puller). A complete list of these combinations and the emotion that they relate to is shown below. The gifs on the right are shown in the same order that the action units listed.

Emotion Action Units Description Examples (Hover to Play) Happiness / Joy 6 + 12 Cheek Raiser, Lip Corner Puller Sadness 1 + 4 + 15 Inner Brow Raiser, Brow Lowerer, Lip Corner Depressor Surprise 1 + 2 + 5 + 26 Inner Brow Raiser, Outer Brow Raiser, Upper Lid Raiser, Jaw Drop Fear 1 + 2 + 4 + 5 + 7 + 20 + 26 Inner Brow Raiser, Outer Brow Raiser, Brow Lowerer, Upper Lid Raiser, Lid Tightener, Lip Stretcher, Jaw Drop Anger 4 + 5 + 7 + 23 Brow Lowerer, Upper Lid Raiser, Lid Tightener, Lip Tightener Disgust 9 + 15 + 16 Nose Wrinkler, Lip Corner Depressor, Lower Lip Depressor Contempt 12 + 14 (on one side of the face) Lip Corner Puller, Dimpler

Back to top

Putting it all together

When measuring facial expressions within iMotions, the stimuli are paired automatically to the FACS analysis, allowing you to pinpoint the exact moment that the stimulus triggered a certain emotion. The FACS is also graded on a scale of intensity, which gives a measure of how strongly the emotion is displayed. These measurements can also be synchronized with recordings of galvanic skin response, which provides a measure of arousal. With this information combined, it’s possible to start drawing conclusions about how strongly an individual felt, and what those emotions consisted of, in response to a set stimulus.

The screenshot below shows how the facial expression data is displayed while a participant watches an advertisement.

If we zoom in, we can see the intensity of the displayed emotion. There are five emotions displayed in the image below, however iMotions provides a measure of the seven central emotions (shown in the table above), alongside, and in conjunction with measurements of action units.

I hope this explanation of action units and FACS has been helpful, and informative. If you’d like to learn even more about facial expressions, then we also have a free pocket guide that you can download for free below!

1090 x 330 ( 1895 x 574 ) 148.55 KB