Abstract
Immediately after visual stimuli disappear from sight, representations of the vanished objects are thought to briefly persist in the sensory system with perceptual-level fidelity. This lingering trace of the most recent perceptual experience has been termed sensory, or iconic, memory and described as a virtually unlimited capacity store that deteriorates over a fraction of a second. Beyond this, recall relies on visual working memory (VWM) which is considered a strongly limited resource, but one which is stabilized against interference and temporal decay. Here, we systematically investigated the temporal dynamics of resource distribution and the deterioration of memory fidelity over very short intervals (0 – 1000ms). Participants viewed a set of randomly oriented stimuli and reported one of them based on a non-masking cue presented at variable delays relative to the offset of the stimuli. The target orientation was reproduced with a finger swipe on a touchpad, minimizing the opportunity for memory deterioration after the cue and providing a precise measure of response time. Contrary to previous assumptions, a robust set-size effect was observed from the moment of stimulus offset. This was followed by a rapid and then more gradual decrease of retrieval precision as the delay between stimulus and cue increased. These results were quantitatively captured by a model in which VWM fidelity is limited before the cue due to normalization of neural signal over multiple stimulus representations, but supplemented following the cue by residual activity from the sensory system. This leads to enhanced recall at the briefest delays while still being contingent on the number of stored objects. The more gradual deterioration over longer time-scales was accounted for by accumulation of random error in encoded feature values, as in previous work. These results extend a successful neurocomputational account of VWM to capture the first moments after a stimulus disappears.