Abstract
Facial expression perception in-the-wild requires an ability to see emotional signals from different views. Some facial expressions (e.g., happiness) are recognized more accurately than others (e.g., fear) from faces viewed frontally. However, results on facial expression perception from non-frontal viewpoints are limited and non-convergent (Matsumoto & Hwang, 2011; Hess et. al, 2007). We investigated expression classification over viewpoint change in an experiment that incorporated human and machine perception. The goal was to test the effects of viewpoint on expression perception, and to examine the role of the visual stimulus, via machine perception, in supporting classification. We tested expression classification for human subjects (N=160) and a deep convolutional neural network (DCNN) trained for face identification (Ranjan et al., 2018). DCNNs model ventral visual stream processing and are known to retain expression and viewpoint information about face images (Colón, et al. 2018; Hill et al., 2019). The test employed the Karolinksa database (KDEF)—a controlled dataset of expressions containing 4,900 images of 70 actors posing 7 facial expressions (happy, sad, angry, surprised, fearful, disgusted, neutral) photographed from 5 viewpoints (90- and 45-degree left and right profiles, and frontal) (Lindqvist et al., 1998). For frontal faces, both humans and the DCNN replicate findings of better recognition of some expressions (e.g., happy > fear) (humans, p < .001), and equivalent classification across viewpoint. For humans, however, there was a strong advantage for detecting angry faces from the frontal viewpoint (viewpoint-expression interaction, p <.01). There was no such interaction in the DCNN, indicating that the human advantage for detecting angry faces from the front cannot be accounted for completely by visual features.
This suggests that the high accuracy humans show for detecting angry faces from the front may be due to independent facial expression processing outside of the ventral visual stream (e.g., dorsal, subcortical).