Abstract
Low-level visual saliency is widely thought to control the allocation of overt attention within natural scenes. However, recent research has shown that the presence of meaningful information at a given location may trump saliency. Here we used representational similarity analysis (RSA) of ERP responses to natural scenes to examine the time course of the neural representation of saliency (assessed using the Graph-Based Visual Saliency model) and high-level meaning-based representations. Participants (N = 32) viewed a series of 50 different scenes, continually maintaining the most recent scene in memory. We computed the correlation between the spatial distribution of low-level salience, the correlation between the spatial distribution of higher-level meaningfulness (“meaning maps”), and the correlation between the ERP scalp distributions, for all the scenes. As we had 50 different scenes, we were able to compute a 50x50 “representational similarity matrix” (correlation matrix) for saliency, for meaning, and for each time point in the ERP waveform. We then analyzed how the relationship between the scene-related similarity matrices and the ERP-related similarity matrix evolved over time. We found that a link between the saliency-based representational space and ERP representational space emerged first (ca. 80 ms), but that a link to the meaning-based representational space emerged soon afterward (ca. 100 ms). These findings are in line with biological models of saliency and low-level visual feature processing, suggesting that meaning-related computations arise after saliency-based computations, but early enough to suppress saliency in controlling overt attention.