Abstract
Performance on visual working memory tasks decreases as more items need to be remembered. Over the past decade, a debate has unfolded between proponents of "slot models" and "slotless models" of this phenomenon. Zhang and Luck (2008) and Anderson et al. (2011) notice that as more items need to be remembered, "memory noise" seems to first increase and then reach a "stable plateau". They argue that three summary statistics characterizing this plateau are consistent with slot models, but not with slotless models. Here we assess the validity of their methods. We generated synthetic data both from a leading slot model and from a recent slotless model, and quantified model evidence using log Bayes factors. We found that the summary statistics provided at most 0.15% of the expected model evidence in the raw data. In a model recovery analysis, a total of more than a million trials were required to achieve 99% correct recovery when comparing models based on summary statistics, whereas fewer than 1000 trials were sufficient when using raw data. These results show that using plateau-related summary statistics for model comparison is highly inefficient, and unreliable for realistic numbers of trials. Applying the same analyses to subject data from Anderson et al. (2011), we found that the evidence in the summary statistics was at most 0.12% of the evidence in the raw data and far too weak to warrant any conclusions. These findings call into question claims about working memory that are based on summary statistics.
Meeting abstract presented at VSS 2014