Purchase this article with an account.
Géraldine Jeckeln, Ying Hu, Jacqueline G. Cavazos, Amy N. Yates, Carina A. Hahn, Larry Tang, Alice J. O’Toole, P. Jonathon Phillips; Building calibrated face-identification ability test with Item Response Theory. Journal of Vision 2020;20(11):579. doi: https://doi.org/10.1167/jov.20.11.579.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Accurate estimates of face-identification ability are crucial in applied forensic settings. Current face-identification datasets are often large and uncalibrated, making them sub-optimal for pre- and post-training evaluations. To optimize efficient and accurate performance assessments, small sets of well-labelled test items are needed. However, item-wise measures cannot be applied to the common forensic task of identity matching in image pairs, because items are either “matched” or “non-matched” identities. Therefore, in this case, an item response confounds item accuracy and response bias. Here, our goal was to construct flexible, well-calibrated subsets of face-identification items using Item Response Theory (IRT) applied to image triads. These triads were composed of two images of one identity and one image of a different identity; the task was to select the “different” identity. Participants (n=77) were tested on the full item pool of 224 face-image triads. Responses were analyzed using the IRT one-parameter model (Rasch model; Rasch, 1960). This approach provides measurements of subject ability and item difficulty on the same scale. Results of the model demonstrate the probability of endorsing a correct response given an item’s difficulty and a subject’s ability. Using these results, we constructed subsets of items that varied in item difficulty. To test the quality of these item subsets, we used responses to these subsets to estimate participants’ ability and predict accuracy for larger subsets of novel items. Leave-one-out cross validation results showed that we can predict both people’s accuracy on novel items and their individual responses. These calibrated face-identification tests can be used to develop face-identification tests with better flexibility, reliability, and time-efficiency.
This PDF is available to Subscribers Only