October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
Measuring race bias in face recognition algorithms: a multi-factor assessment
Author Affiliations & Notes
  • Jacqueline G. Cavazos
    The University of Texas at Dallas
  • P. Jonathon Phillips
    National Institute of Standards and Technology
  • Carlos D. Castillo
    University of Maryland Institute for Advance Computer Studies
  • Alice J. O'Toole
    The University of Texas at Dallas
  • Footnotes
    Acknowledgements  National Eye Institute Grant 1R01EY029692-01 to A. O’T; CDC was supported by JANUS program Intelligence Advanced Research Projects Activity, R&D Contract, No. 2014-14071600012
Journal of Vision October 2020, Vol.20, 553. doi:https://doi.org/10.1167/jov.20.11.553
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jacqueline G. Cavazos, P. Jonathon Phillips, Carlos D. Castillo, Alice J. O'Toole; Measuring race bias in face recognition algorithms: a multi-factor assessment. Journal of Vision 2020;20(11):553. https://doi.org/10.1167/jov.20.11.553.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Previous generations of face recognition algorithms show differences in accuracy for faces of different races (race bias) (O’Toole et al., 1991; Furl et al., 2002; Givens et al., 2004; Phillips et al., 2011; Klare et al., 2012). Whether newer deep convolutional neural networks (DCNNs) are also race biased is less well studied (El Khiyari et al., 2016; Krishnapriya et al., 2019). Here we present methodological considerations for measuring underlying race bias. We consider two key factors: data-driven and scenario modeling. Data-driven factors are driven by the data itself (e.g., the architecture of the algorithm, image quality, image population statistics). Scenario modeling considers the role of the “user” of the algorithm (e.g., threshold decisions and demographic constraints). To illustrate these issues in practice, we tested four face recognition algorithms: one pre-DCNN (A2011; Phillips et al., 2011) and three DCNNs (A2015; Parkhi et al., 2015), (A2017b; Ranjan et al., 2017), (A2019; Ranjan et al., 2019) on East Asian and Caucasian faces. First, for all four algorithms, the degree of race bias varied as a function of the identification decision threshold. Second, for all algorithms, to achieve equal false accept rates (FARs), Asian faces required higher identification thresholds than Caucasian faces. Third, dataset difficulty affected both overall recognition accuracy and race bias. Fourth, demographic constraints on the formulation of the distributions used in the test, impacted estimates of algorithm accuracy. We conclude with a recommended checklist for measuring race bias in face recognition algorithms.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.