Abstract
All natural visual events exhibit a spatiotemporal structure, but our understanding of how temporal aspects are represented lags far behind our understanding of the role of space. Here, we present data indicating that a combination of both the spatial and the temporal structure of visual events provides frames of reference for storage in working memory. In a series of experiments, participants performed a change detection task, for which they memorized colours presented sequentially at different locations and with different stimulus onset asynchronies. Each item could be uniquely identified by its spatial or temporal coordinates, but neither spatial nor temporal information was task-relevant. The key manipulation was that of retrieval context: The test array was identical to the memory array in terms of either its entire spatiotemporal structure, or only its spatial or temporal components. Removing either spatial or temporal information at retrieval impaired performance, indicating that memory relied, at least to some degree, on the spatiotemporal structure in which items were initially perceived. Overall, more weight appeared to be assigned to the spatial structure, but there were pronounced individual differences in the relative weighting of spatial and temporal information, which were fairly consistent throughout and across sessions. Conceivably, these are indicative of stable individual preferences for coding in the spatial or temporal domain. We observed no substantial differences between a randomly interleaved and a blocked variation of retrieval context, suggesting that encoding in a spatiotemporal frame of reference was not strategically adjusted to task demands. However, the inter-item spacing of spatial and temporal structures influenced the relative weighting of spatial and temporal information. Favouring the frame of reference that is more widely spaced presumably facilitates item individuation and thereby access to representations. We propose that time may serve a similar function as space in the architecture of visual working memory.
Acknowledgement: This work was supported by the Deutsche Forschungs-gemeinschaft (DFG Research grant HE 8207/1-1 and RO 3579/11-1).