December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Adaptive methods to quickly estimate psychometric functions: the case of Psi-marg-grid and the effect of non-monotony
Author Affiliations & Notes
  • Adrien Chopin
    Sorbonne Université, INSERM, CNRS
  • Footnotes
    Acknowledgements  This research was supported by the Chair SILVERSIGHT ANR-18-CHIN-0002, by the IHU FOReSIGHT ANR-18-IAHU-01, and by the LabEx LIFESENSES (ANR-10-LABX-65).
Journal of Vision December 2022, Vol.22, 3302. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Adrien Chopin; Adaptive methods to quickly estimate psychometric functions: the case of Psi-marg-grid and the effect of non-monotony. Journal of Vision 2022;22(14):3302.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Introduction: Estimating psychometric thresholds can be challenging, especially stereoacuities (Chopin et al., 2019). Adaptive methods used for estimation generally assume monotonically increasing psychometric functions (Levitt, 1971), which can be incorrect, for example when testing stereoacuities (Kane et al., 2014). I wondered how adaptive methods are affected by non-monotony in psychometric functions. Methods: I performed Monte-Carlo simulations of 90-trial threshold estimations following six different methods: (1) the method of constant stimuli (MOCS), ZEST (King-Smith et al., 1991), Psi (Kontsevich & Tyler, 1999), Psi-marginal (Prins, 2013), Psi-grid (Doire, Brookes, & Naylor, 2017) and a new algorithm Psi-marg-grid, mixing Psi-Marg’s marginalization on nuisance parameters and Psi-grid’s adaptive search grid. I tested four conditions under which the simulated ideal-observer’s function and/or the algorithm’s assumed function were monotonic or non-monotonic. Parameters other than the threshold were random. The bias was measured by the median difference to the simulated threshold, the accuracy by the mean absolute error and the test-retest by the limit of agreement between two simulations of a given threshold (log units). The chance level was calculated as the probability to obtain a threshold better than a critical value while simulating a random observer. Results: When assuming monotony, no algorithms could issue meaningful estimates with non-monotonic observers (bias>11822%). When assuming non-monotony, Psi-marg-grid showed the best results (bias 0.6%, accuracy 8.92%, test-retest 0.159, chance level 1%). Only Psi-marg had a better test-retest (0.154) but it lacked accuracy with non-monotonic observers (error 29.4%). MOCS had the worst test-retest (0.6) and chance level (53.2%). Psi’s and Psi-grid’s chance levels were high (18.9% and 17.8% respectively). ZEST lacked accuracy with non-monotonic observers (error 122%). Conclusion: it is important to use methods assuming non-monotony, because none of the tested methods were otherwise robust to non-monotony in the observer’s psychometric functions. When assuming non-monotony, Psi-marg-grid was the best tested method.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.