Abstract
Despite the importance of face perception in human and computer vision, no quantitative model of perceived facial similarity exists. We designed a novel behavioural task to efficiently collect similarity and same/different identity judgements, and used it to test diverse candidate models. We rendered two sets of 232 pairs of realistic faces from the Basel Face Model (BFM), a generative statistical model based on principal components analysis of 3D face scans. The two sets had identical relative geometries in BFM but different face exemplars. Participants (N=26) arranged face pairs on a large touch-screen according to how similar they appeared, relative to anchoring pairs at the top and bottom of the screen and to other adjusted pairs. Participants also placed a horizontal bar on each trial indicating the threshold between pairs that appeared to depict “the same person” vs different individuals. We compared perceived dissimilarities with diverse distance metrics derived from the BFM, image properties, or deep neural networks (DNNs) trained either on objects or faces. A face-recognition trained DNN (especially late intermediate layers) predicted human judgements best and explained unique variance over simpler image-based models. However, distances within the BFM performed almost as well, despite capturing only the distance between faces in principal-components space, with no image information. The performance was improved by taking a sigmoidal function of BFM distances. Judgements were similar for both stimulus sets (different face exemplars but same relative geometries). Faces with clearly perceptible differences were tolerated as belonging to the same identity, and the identity threshold aligned approximately with the statistically expected distance between random individuals in the BFM. Human facial similarity judgements appear tuned to the distribution of facial features, and can be well predicted by both statistical face models and image-computable DNNs.