September 2021
Volume 21, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2021
Facial similarity judgements are well predicted by image-computable DNNs and a statistical face distribution model
Author Affiliations & Notes
  • Kamila M. Jozwik
    University of Cambridge
  • Jonathan O'Keeffe
    Cognition and Brain Sciences Unit
  • Katherine R. Storrs
    Justus Liebig University
  • Nikolaus Kriegeskorte
    Columbia University
  • Footnotes
    Acknowledgements  This work was supported by a Sir Henry Wellcome postdoctoral fellowship (206521/Z/17/Z) awarded to KMJ; an Alexander von Humboldt Foundation postdoctoral fellowship awarded to KRS; the Wellcome Trust and the MRC Cognition and Brain Sciences Unit.
Journal of Vision September 2021, Vol.21, 1900. doi:https://doi.org/10.1167/jov.21.9.1900
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Kamila M. Jozwik, Jonathan O'Keeffe, Katherine R. Storrs, Nikolaus Kriegeskorte; Facial similarity judgements are well predicted by image-computable DNNs and a statistical face distribution model. Journal of Vision 2021;21(9):1900. doi: https://doi.org/10.1167/jov.21.9.1900.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Despite the importance of face perception in human and computer vision, no quantitative model of perceived facial similarity exists. We designed a novel behavioural task to efficiently collect similarity and same/different identity judgements, and used it to test diverse candidate models. We rendered two sets of 232 pairs of realistic faces from the Basel Face Model (BFM), a generative statistical model based on principal components analysis of 3D face scans. The two sets had identical relative geometries in BFM but different face exemplars. Participants (N=26) arranged face pairs on a large touch-screen according to how similar they appeared, relative to anchoring pairs at the top and bottom of the screen and to other adjusted pairs. Participants also placed a horizontal bar on each trial indicating the threshold between pairs that appeared to depict “the same person” vs different individuals. We compared perceived dissimilarities with diverse distance metrics derived from the BFM, image properties, or deep neural networks (DNNs) trained either on objects or faces. A face-recognition trained DNN (especially late intermediate layers) predicted human judgements best and explained unique variance over simpler image-based models. However, distances within the BFM performed almost as well, despite capturing only the distance between faces in principal-components space, with no image information. The performance was improved by taking a sigmoidal function of BFM distances. Judgements were similar for both stimulus sets (different face exemplars but same relative geometries). Faces with clearly perceptible differences were tolerated as belonging to the same identity, and the identity threshold aligned approximately with the statistically expected distance between random individuals in the BFM. Human facial similarity judgements appear tuned to the distribution of facial features, and can be well predicted by both statistical face models and image-computable DNNs.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×