September 2021
Volume 21, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2021
Excess success in studies of object-based attention
Author Affiliations
  • Evelina Thunell
    Karolinska Institutet
    Purdue University
  • Gregory Francis
    Purdue University
Journal of Vision September 2021, Vol.21, 2271. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Evelina Thunell, Gregory Francis; Excess success in studies of object-based attention. Journal of Vision 2021;21(9):2271.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Many subareas of psychology face a replication crisis; perhaps because past empirical research used inappropriate sampling, analysis, or reporting methods. These questionable research practices impeded scientific progress by misleading researchers about effects and hindering theory development. To date, vision science was rarely mentioned in this context, but we show that many investigations of object-based attention seem to have similar problems. We applied the Test for Excess Success (TES) to each of 37 identified articles related to object-based attention with four or more experiments. Using the reported statistics and sample sizes, the TES estimates the probability that a replication of the studies in an article would produce the same degree of success as the original set of studies. For 19 of the articles (51%) this replication probability fell below 0.1, meaning that if the effects are real and of a similar magnitude as reported, it is unlikely that similar studies, with the same sample sizes, would reproduce the original results. Non-replicable results should not be trusted, so many of the articles do not provide adequate support for their conclusions. New studies will be required to properly examine the effects, and such studies will require much larger sample sizes or better designs. To investigate object-based attention with the classic two-rectangles paradigm, we ran an online study (n=274, estimated power of 0.97) and found a significant 14 millisecond reaction time advantage when a cue and target appear in a common rectangle compared to when they appear in different rectangles. Based on this finding, we suggest that studies of object-based attention require sample sizes nearly ten times larger than what are commonly used.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.