August 2023
Volume 23, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2023
Designing an “Other Race Effect” test for forensic facial identification experts using the performance of deep networks and untrained humans.
Author Affiliations & Notes
  • Kate Marquis
    The University of Texas at Dallas
  • Selin Yavuzcan
    The University of Texas at Dallas
  • Géraldine Jeckeln
    The University of Texas at Dallas
  • Amy Yates
    National Institute of Standards and Technology
  • P. Jonathon Phillips
    National Institute of Standards and Technology
  • Alice O'Toole
    The University of Texas at Dallas
  • Footnotes
    Acknowledgements  National Institute of Standards and Technology, Grant 70NANB21H109
Journal of Vision August 2023, Vol.23, 5725. doi:https://doi.org/10.1167/jov.23.9.5725
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Kate Marquis, Selin Yavuzcan, Géraldine Jeckeln, Amy Yates, P. Jonathon Phillips, Alice O'Toole; Designing an “Other Race Effect” test for forensic facial identification experts using the performance of deep networks and untrained humans.. Journal of Vision 2023;23(9):5725. https://doi.org/10.1167/jov.23.9.5725.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Forensic face examiners outperform untrained participants in face identity matching (Phillips et al., 2018), though it is unclear whether this superiority generalizes to other-race faces. We developed a challenging test that can be performed with the limited time available to professional examiners. To select the most difficult image pairs from a set of Black (n= 3,102) and White (n= 122,728) faces (self-identified race when images were collected), we employed a deep convolutional neural network (DCNN) (Deng et al., 2019) and an experiment with untrained participants. Image pairs (n= 36 per race) were assembled using a DCNN “perceptual” similarity measure. Same-identity (different-identity) image pairs with the lowest (highest) similarity scores were selected from all possible pairs. Untrained participants (White: n= 26, Black: n= 11) judged whether the images showed the same identity or different identities. Ranking by perceptual difficulty, we created a set of 10 Black and 10 White face pairs (half same-identity pairs). “Difficulty” was measured by tallying the number of participants who incorrectly indicated same-identity pair as different identities, and vice versa. Next, we benchmarked the test by computing participants’ accuracy (area under the ROC curve) on the subset of pairs. The test proved challenging for untrained participants [Black participants: (faces: Black= 0.66, White= 0.53); White participants: (faces: Black= 0.56, White= 0.49)]. Participant race, face race, and the interaction did not affect accuracy (p > 0.05). Notably, additional DCNNs performed more accurately on the White face pairs than Black face pairs (Szegedy et al., 2017: Black= 0.72 , White= 1.0; Ranjan et al., 2017: Black= 0.5, White= 0.92). Given that the pattern of performance across race differed for humans and the DCNNs, we conclude that untrained human benchmarks are critical in building a challenging and balanced cross-race test for experts.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×