Abstract
In natural environments, face identification must operate robustly over changes in illumination. We examined identification under uncontrolled illumination conditions using a constant set of identities that varied in photometric difficulty across multiple images. Photometric difficulty was estimated using a face recognition algorithm created by fusing three top-performing algorithms from a recent international competition. The algorithm computed similarity scores for same-identity and different-identity pairs from multiple images taken under a variety of uncontrolled illumination conditions (both indoors and outdoors). For each identity pair, algorithm-generated similarity scores were ranked and divided into three machine performance groups: good, moderate, and poor. Algorithm performance across these constant identity pairings varied widely. In three experiments, humans matched identity in image pairs from the good, moderate, and poor groups. In Experiment 1, participants matched 240 pairs of faces (120 same-identity) from all three conditions, rating the likelihood that the images were of the same person (1: sure same – 5: sure different). Humans performed best for the good pairs, and equivalently for the moderate and poor performance pairs. Algorithm accuracy surpassed humans, although the algorithm advantage decreased as the challenge level increased (from good to poor). Experiments 2 and 3 replicated these results using a larger number of face pairs from only the moderate and poor performance conditions. In a fourth experiment, humans matched identity on image pairs that yielded systematically incorrect performance for the algorithm (i.e., an infinitely negative ROC curve). Specifically, the same-identity pairs had algorithm-generated similarity scores that always were lower than the similarity scores for different-identity pairs, (i.e., same- and different-identity distributions reversed). Human performance was well above chance (d′ = 1.5). In summary, as the level of photometric challenge increased, the performance advantage for algorithms over humans diminished, ultimately reversing to a human advantage for the most difficult cases.