October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
Building calibrated face-identification ability test with Item Response Theory
Author Affiliations
  • Géraldine Jeckeln
    University of Texas at Dallas
  • Ying Hu
    University of Texas at Dallas
  • Jacqueline G. Cavazos
    University of Texas at Dallas
  • Amy N. Yates
    National Institute of Standards and Technology
  • Carina A. Hahn
    National Institute of Standards and Technology
  • Larry Tang
    University of Central Florida
  • Alice J. O’Toole
    University of Texas at Dallas
  • P. Jonathon Phillips
    National Institute of Standards and Technology
Journal of Vision October 2020, Vol.20, 579. doi:https://doi.org/10.1167/jov.20.11.579
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Géraldine Jeckeln, Ying Hu, Jacqueline G. Cavazos, Amy N. Yates, Carina A. Hahn, Larry Tang, Alice J. O’Toole, P. Jonathon Phillips; Building calibrated face-identification ability test with Item Response Theory. Journal of Vision 2020;20(11):579. https://doi.org/10.1167/jov.20.11.579.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Accurate estimates of face-identification ability are crucial in applied forensic settings. Current face-identification datasets are often large and uncalibrated, making them sub-optimal for pre- and post-training evaluations. To optimize efficient and accurate performance assessments, small sets of well-labelled test items are needed. However, item-wise measures cannot be applied to the common forensic task of identity matching in image pairs, because items are either “matched” or “non-matched” identities. Therefore, in this case, an item response confounds item accuracy and response bias. Here, our goal was to construct flexible, well-calibrated subsets of face-identification items using Item Response Theory (IRT) applied to image triads. These triads were composed of two images of one identity and one image of a different identity; the task was to select the “different” identity. Participants (n=77) were tested on the full item pool of 224 face-image triads. Responses were analyzed using the IRT one-parameter model (Rasch model; Rasch, 1960). This approach provides measurements of subject ability and item difficulty on the same scale. Results of the model demonstrate the probability of endorsing a correct response given an item’s difficulty and a subject’s ability. Using these results, we constructed subsets of items that varied in item difficulty. To test the quality of these item subsets, we used responses to these subsets to estimate participants’ ability and predict accuracy for larger subsets of novel items. Leave-one-out cross validation results showed that we can predict both people’s accuracy on novel items and their individual responses. These calibrated face-identification tests can be used to develop face-identification tests with better flexibility, reliability, and time-efficiency.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.