Abstract
A popular conception of face representation is that faces are represented in a multidimensional face space with the “mean face” at the origin [Valentine, Q. J. Exp. Psych. 43(2), 1991]. Several studies have hypothesized that appearance-based features may account for many of the dimensions of this face space. We examined the relationship between appearance-based dimensions and judgments made by human observers in two experiments. In the first experiment, a match-to-sample paradigm was used where observers saw a sample face followed by two alternative faces which varied from the sample in expression or illumination. Observers indicated which of the alternatives shared the same identity as the sample. The second experiment employed a sequential matching paradigm in which the standard face was partially occluded by a checkerboard grid of small black squares. The comparison face was occluded by the same checkerboard as the standard, a reversed checkerboard, a checkerboard shifted in phase by 90 degrees, or no checkerboard. Observers indicated if the two faces were of the same or different identities. Presentations in both experiments were masked by a full screen pattern.
Several computational appearance-based algorithms were investigated. Both the pixel intensities and the outputs of Gabor filters were used as input. Euclidean distances were calculated between the inputs for the pairs of images in the experiment. In addition, both intensities and Gabor filter outputs were subjected to several variations of Principle Components Analysis. The results revealed that though these techniques produced reasonably accurate performance in many of the conditions tested, the correspondence between the appearance-based algorithms and the observers was poor. This was especially true in the checkerboard experiment, and remained true even after a filling-in algorithm was applied to remove the occlusions. These results demonstrate that appearance-based dimensions alone cannot adequately parameterize the face space used by human observers.
Supported in part by a grant from NIH.