Abstract
Human social communication critically relies on the accurate interpretation of nonverbal cuessuch as dynamic facial expressions, with confusions and misunderstands causing substantial negative consequences. Here, we reveal for the first time the specific dynamic face movements that drive high accuracy in emotion recognition and those which cause significant confusions. Three hundred participants (Western, 150 females, mean: 21.6, SD: 2.9 years) categorized a sub-set of 720 dynamic facial expressions models of the six classic emotions, each comprising a specific set of face movements (called Action Units – AUs; Ekman & Friesen, 1976) by emotion in a 6AFC task. To identify the specific face movements that give rise to high emotion recognition accuracy, and those that cause confusions, we measured the relationship between each AU and the participants’ recognition accuracy on each trial (i.e., correct or incorrect) using an information-theoretic analysis basedon Mutual Information (Ince et al., 2017). Results showed that, for each emotion, a specific subset of dynamic face movements drive high recognition accuracy. Specifically, recognition of fear and surprise – which are often confused – is boosted by distinct face movements: eyebrow raising (Outer/Inner Brow Raiser–AU1,2) and wide mouth opening (Jaw Drop–AU26) in surprise, and brow lowering (Brow Lowerer–AU4) and lateral mouth stretching (Lip Stretcher–AU20) in fear. A similar pattern characterized the often-confused emotions, disgust and anger – recognition of anger is boosted by mouth opening (e.g. Lip Funneler–AU22, Mouth Stretch–AU27), while disgust is boosted by nostril (Nasolabial Deepener–AU11) and eye constrictions (e.g., Cheek Raiser–AU6). Analyses of the face movements that boost recognition accuracy across emotions showed that they are highly distinct, suggesting that recognition boosters are category specific. Here, we modelled the specific face movements that drive high recognition accuracy of emotions and support optimal social communication, with implications for designing communicative digital avatars and social robots.