August 2014
Volume 14, Issue 10
Vision Sciences Society Annual Meeting Abstract  |   August 2014
The uninformativeness of summary statistics for comparing working memory models
Author Affiliations
  • Wei Ji Ma
    Center for Neural Science and Department of Psychology, New York University
  • Ronald Van den Berg
    Department of Electrical Engineering, University of Cambridge
Journal of Vision August 2014, Vol.14, 158. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Wei Ji Ma, Ronald Van den Berg; The uninformativeness of summary statistics for comparing working memory models. Journal of Vision 2014;14(10):158. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Performance on visual working memory tasks decreases as more items need to be remembered. Over the past decade, a debate has unfolded between proponents of "slot models" and "slotless models" of this phenomenon. Zhang and Luck (2008) and Anderson et al. (2011) notice that as more items need to be remembered, "memory noise" seems to first increase and then reach a "stable plateau". They argue that three summary statistics characterizing this plateau are consistent with slot models, but not with slotless models. Here we assess the validity of their methods. We generated synthetic data both from a leading slot model and from a recent slotless model, and quantified model evidence using log Bayes factors. We found that the summary statistics provided at most 0.15% of the expected model evidence in the raw data. In a model recovery analysis, a total of more than a million trials were required to achieve 99% correct recovery when comparing models based on summary statistics, whereas fewer than 1000 trials were sufficient when using raw data. These results show that using plateau-related summary statistics for model comparison is highly inefficient, and unreliable for realistic numbers of trials. Applying the same analyses to subject data from Anderson et al. (2011), we found that the evidence in the summary statistics was at most 0.12% of the evidence in the raw data and far too weak to warrant any conclusions. These findings call into question claims about working memory that are based on summary statistics.

Meeting abstract presented at VSS 2014


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.