Abstract
Humans identify faces by matching real-world visual information with memorized identity information. However, there is no direct access to the information contents of human memory. Here, we address this challenge by reverse correlating the memory contents of familiar face identities using a novel generative model of face identity information. We applied a General Linear Model (GLM) on 355 3D faces, independently for shape and texture, to explain away the variance of gender, ethnicity and their interactions, isolating the multivariate identity information represented as residual deviation from the norm, independently for 3D shape and 2D texture. Next, with Principal Components Analysis (PCA) we parameterized the identity residuals as 355-dimensional PCA weights. Our experiment probed the memory contents of 4 face identities familiar to 14 observers. We generated "identity noise" for each familiar face. To illustrate, when the target was "Mary" we added randomly assigned PCA weights (the identity noise, S1-A) to the local norm. On each trial, participants viewed 6 random identities, chose the most similar to "Mary" and rated this similarity (S1-A). From the 1,800 trials pairings < face identity information; similarity score> we linearly regressed per participant the shape and texture information that modulates similarity choices. The resulting classification faces are shown in S1-B1 and S1-B2. We further analysed the fit between the objective ground truth identity information and memory contents. Results show that participants memorized "Mary" using a consistent set of features. Faithful features are the most objectively distant from the norm in the ground truth. Caricatured and imagined features occur for vertices objectively nearer the norm in ground truth. Our results are compatible with a norm-based account of identity coding in memory (S1-B), but the objectively distant features are not necessarily memorized (S1-C), suggesting semantic biases in feature selection.
Meeting abstract presented at VSS 2017