Abstract
Recently we have demonstrated that synchronized auditory signals can greatly increase the saliency of visual target events in cluttered, continuously changing displays [Van der Burg, E, Olivers, CNL, Bronkhorst, AW, and Theeuwes, J. (2008). Journal of Experimental Psychology: Human Perception and Performance]. The sound makes the visual target pop out. This “ip and pop” effect occurs even though the sound carries no information on the location or identity of the target, as long as it is synchronized with the visual event. Here we report evidence from EEG data that the pip and pop effect results from an early, pre-attentive integration of auditory and visual signals. This audiovisual integration boosts the visual signal and causes it to capture attention.
Participants performed a visual search task with displays consisting of a multitude of oblique bars that continuously flipped between different orientations. The target was a bar that changed to horizontal or vertical, and the task was to indicate its orientation with an unspeeded response. Behavioral data showed the pip and pop effect: Accuracy was better when the target change was accompanied by a sound, compared to when no sound was present, or when the sound was synchronized with a distractor instead. EEG analysis revealed an early modulation of the event-related potential (ERP) around 50 ms from target onset, when a tone was synchronized with the target, compared to the summed activity of those conditions in which only the auditory or only the visual signal was present. Around 200 ms, a lateralized ERP component (the N2pc) to the target emerged, reflecting the capture of spatial attention to the location of the target. This was followed by an increased P3 component reflecting target identification. Thus early audiovisual integration causes visual events to gain priority over competing visual events.