September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Mixing different contrasts inflates estimated metacog-nitive ability in perceptual decision making
Author Affiliations
  • Dobromir Rahnev
    School of Psychology, Georgia Institute of Technology
  • Stephen M Fleming
    Wellcome Centre for Human Neuroimaging, University College London
    Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London
Journal of Vision September 2019, Vol.19, 143d. doi:https://doi.org/10.1167/19.10.143d
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Dobromir Rahnev, Stephen M Fleming; Mixing different contrasts inflates estimated metacog-nitive ability in perceptual decision making. Journal of Vision 2019;19(10):143d. doi: https://doi.org/10.1167/19.10.143d.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

It is becoming widely appreciated that higher stimulus sensitivity trivially increases estimates of metacognitive sensitivity (the correspondence between confidence and performance). Therefore, meaningful comparisons of metacognitive ability across conditions and observers necessitates equating stimulus sensitivity. To achieve this, one common approach is to use a continuous staircase that runs throughout the duration of the experiment under the assumption that this procedure has no influence on the estimated metacognitive ability. Here we critically examine this assumption. Using previously published data, we find that, compared to using a single level of stimulus contrast, staircase techniques lead to inflated estimates of metacognitive ability across a wide variety of measures including area under the type 2 ROC curve (F(2,30) = 8.66, p = .0005), the confidence-accuracy correlation phi (F(2,30) = 8.38, p = .0006), meta-d’ (F(2,30) = 9.68, p = .0002), Mratio (F(2,30) = 8.45, p = .0006), and Mdiff (F(2,30) = 9.61, p = .0002). Further, this metacognitive inflation correlates with the degree of stimulus variability experienced by each subject (this correlation was significant for each of the five measures above). Finally, we show that the degree of stimulus variability in a staircase procedure may itself be driven by individual differences in metacognitive ability. These results suggest that studies using a staircase approach are likely to report inflated estimates of metacognitive ability. Further, we argue that similar inflation likely occurs in the presence of variability in task difficulty caused by other factors such as fluctuations in alertness or gradual improvement on the task. We offer practical solutions to these issues, both in the design and analysis of metacognition experiments.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×