Purchase this article with an account.
Eilidh Noyes, Y. Ivette Colon, Matthew Hill, Connor Parde, Carlos Castillo, Swami Sankaranarayanan, Alice O'Toole; Face Familiarity in Deep Convolutional Neural Networks. Journal of Vision 2018;18(10):1093. doi: https://doi.org/10.1167/18.10.1093.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Human face recognition is robust in difficult image conditions for people we know, but not for unfamiliar people. This familiarity advantage holds even for disguised faces (Noyes & Jenkins, 2016). Previously, we tested a deep convolutional neural network (DCNN) developed for face recognition (Sankaranarayanan et al. 2016) on its identity-matching performance on disguised and non-disguised faces. The DCNN performed well for non-disguised and impersonation image pairs, but not for evasion disguises (i.e., disguised to look unlike oneself) (Noyes et al. 2017). Here, we asked whether this same DCNN could overcome disguise when it becomes "familiar" with an identity. Face representations were extracted from the top layer of the network. The DCNN's task was to decide if image pairs were of the same person. We compared two familiarization methods. In the feature averaging method, same/different identity decisions for each image pair were made by comparing the averaged DCNN features of the "familiarized identity" to the DCNN features in a comparison image. This simulated an average representation of a known face. DCNN feature averaging improved matching accuracy on evasion disguise faces from 50% to 69%, however it impaired performance on different-person disguise and non-disguise trials. Next, we tested an identity contrast method in which the DCNN learned each identity, in turn, from ~100 in-the-wild images that were contrasted against images of all other identities from the database with a Support Vector Machine. The DCNN face representations of the matching task images were then compared for similarity with the DCNN representations of each of the learned identities. The identity contrast method resulted in a 16% improvement in accuracy for evasion disguise faces, and importantly also maintained high performance for different person trials. In conclusion, the DCNN benefited from familiarization and exhibited between- and within-person learning that is similar to humans (Noyes, 2016).
Meeting abstract presented at VSS 2018
This PDF is available to Subscribers Only