December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Norm-referenced neural mechanism for the recognition of facial expressions across fundamentally different face shapes
Author Affiliations & Notes
  • Michael Stettler
    Section for Computational Sensomotorics, Centre for Integrative Neuroscience & Hertie Institute for Clinical Brain Research, University Clinic Tübingen, 72076 Tübingen, Germany
    International Max Planck Research School for Intelligent Systems (IMPRS-IS), 72076 Tübingen, Germany.
  • Nick Taubert
    Section for Computational Sensomotorics, Centre for Integrative Neuroscience & Hertie Institute for Clinical Brain Research, University Clinic Tübingen, 72076 Tübingen, Germany
  • Ramona Siebert
    Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tübingen, 72076 Tübingen, Germany
  • Silvia Spadacenta
    Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tübingen, 72076 Tübingen, Germany
  • Peter Dicke
    Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tübingen, 72076 Tübingen, Germany
  • Peter Thier
    Department of Cognitive Neurology, Hertie Institute for Clinical Brain Research, University of Tübingen, 72076 Tübingen, Germany
  • Martin Giese
    Section for Computational Sensomotorics, Centre for Integrative Neuroscience & Hertie Institute for Clinical Brain Research, University Clinic Tübingen, 72076 Tübingen, Germany
  • Footnotes
    Acknowledgements  This work was supported by HFSP RGP0036/2016, ERC 2019-SyG-RELEVANCE-856495 and NVIDIA Corp. MG was also supported by BMBF FKZ 01GQ1704 and BW-Stiftung NEU007/1 KONSENS-NHE. RS, SS, PD, and PT were supported by a grant from the DFG (TH 425/12-2)
Journal of Vision December 2022, Vol.22, 3398. doi:https://doi.org/10.1167/jov.22.14.3398
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Michael Stettler, Nick Taubert, Ramona Siebert, Silvia Spadacenta, Peter Dicke, Peter Thier, Martin Giese; Norm-referenced neural mechanism for the recognition of facial expressions across fundamentally different face shapes. Journal of Vision 2022;22(14):3398. https://doi.org/10.1167/jov.22.14.3398.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Humans recognize facial expressions from highly different facial shapes, including comic figures that never have been seen before. Even though they are trained with huge numbers of pictures of faces (REF), this task is a challenge for popular deep neural network models of face recognition, while humans effortlessly accomplish such transfer. For the encoding of facial identity two different encoding mechanisms have been contrasted, norm-referenced and example-based encoding (Tsao et al., 2017, Leopold et al., 2006; Koyano et al, 2021). We demonstrate that norm-referenced encoding is suitable to account for facial expression recognition from fundamentally different head shapes without extensive training. METHODS: We propose a neural model consisting of two modules: 1) a standard deep neural network, which models the initial parts of the visual pathway and extracts a sparse set of facial landmarks that support optimally the classification of the learned facial expressions; 2) a simple neural network for norm-referenced encoding of expressions based on these landmark features. We compare this new model to a standard deep neural model with a stimulus set that presents the same 7 basic facial expressions on very different head shapes (human avatar, comic figure, etc.). RESULTS: Our model recognizes reliably the facial expressions across all tested head shapes, requiring training of the expressions only of one head shape and of the neutral expressions of all other head shapes. In contrast, a state-of-the art CNN architecture (ResNet50) trained with 400k+ facial images fail to classify expressions robustly on this data set. CONCLUSIONS: We presented a physiologically-inspired neural model for the ‘vectorized encoding’ (Beymer & Poggio, 1995) of facial expressions that accounts for transfer across very different head shapes, which is difficult to obtain with standard models.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×