Decoding the neural representations of emotional faces in stereo- versus monoscopic viewing conditions

Abstract

Humans represent faces as three-dimensional shapes and facial expressions as changes in the spatial configuration of relevant landmarks (e.g., eyebrows) in them. Stereopsis is a key component of the visual system for perceiving spatial depth. The availability of stereoscopic depth cues might therefore influence how human observers process faces. Here, we compare the effect of stereoscopic and monoscopic presentations of human facial expressions on the neurophysiological response by combining immersive Virtual Reality (VR) technology with EEG and multivariate decoding. 34 healthy, young participants performed an emotion recognition task (720 trials) in which renderings of three computer-generated faces showed different emotional expressions (neutral, happy, angry, surprised). All stimuli were presented as frontal portraits using an HTC Vive Pro Eye VR headset. For monoscopic trials, we showed the faces as 2D planes (rendered by a single virtual camera). For stereoscopic trials, the 3D face model was displayed (rendered by one camera per eye), providing stereoscopic depth information. We trained and cross-validated time-resolved linear classifiers to predict the displayed emotional expression from the time-locked EEG signal. Participants recognized all emotions with high accuracy in both viewing conditions. The emotional expression could be decoded above chance-level starting around 150 ms after stimulus onset. Binary classification of emotion pairs (e.g., angry-vs-neutral or happy-vs-surprised) yielded distinctive decoding time-courses. Decoding accuracy did not differ significantly between mono- and stereoscopic trials. The viewing condition itself, however, could also be decoded from the EEG with a similar time-course as decoding the (task-irrelevant) identity of the displayed face. We conclude that although stereoscopically, as compared to monoscopically, presented faces elicit a distinguishable EEG response, this does not influence the decodability of neural processes related to emotion recognition. This is evidence that the way the brain decodes emotional expressions from facial geometries—viewed from a fixed perspective—is largely uninfluenced by stereoscopic information.