Abstract
We compared the accuracy of seven state-of-the-art face recognition algorithms with human performance on the same task. Humans and algorithms determined whether two face images, taken under different illumination conditions, were pictures of the same person or of different people. The algorithms tested were participants in the Face Recognition Grand Challenge (FRGC) test organized by the National Institutes of Standards and Technology. In that competition, algorithms matched identities in 128 million pairs of face images. For the human experiments, we sampled 120 “easy” and 120 “difficult” face pairs from the FRGC dataset, using similarity scores derived from a control algorithm based on a principal components analysis of the aligned and scaled face images. In three experiments, which varied only in exposure time (Exp. 1 - unlimited; Exp2. - 2s, Exp. 3 - 500ms), humans rated face pairs according to the likelihood that the two people were the same. ROC curves from the humans and the algorithms were compared. Three algorithms outperformed humans at matching face pairs prescreened to be “difficult” (cf., Liu, in press; Xie et al., 2005) and all but one algorithm surpassed humans on the “easy” face pairs. Although illumination variation continues to challenge face recognition algorithms, several current algorithms compete favorably with humans— even if they appear to perform poorly in absolute terms.
Work supported by TSWG funding.