Abstract
The visual world paradigm (VWP) has been used to evaluate how a variety of cognitive and contextual factors influence the deployment of visual spatial attention. Previous VWP studies have employed linguistic cues, where a spoken or written target word immediately precedes a search array. This method of cuing is effective and easily controlled but lacks ecological validity. A more ecologically valid way to cue a visual event would be to use environmental sounds as cues. While this answer to ecological validity seems straightforward, matching audio and visual cues when those cues are meant to represent real objects is methodologically challenging. The present study attempts to meet this challenge by using subjective ratings of stimulus attributes as our matching variables. This study contains 150 stimuli (50 images, 100 sounds) that were rated by 10 participants for pleasantness and familiarity. For each stimulus participants were also asked for an open-ended identification response. Following the subjective ratings phase of the experiment participants completed a 4AFC task where each of the 50 images was presented with two matched (target) sounds and two mismatched (lures) sounds. There were no significant differences between sounds and images for pleasantness or familiarity. There was a significant difference in identification accuracy between sounds and images, with images being slightly more accurately identified than sounds. In the 4AFC task participants were highly accurate at selecting one of the two matched sounds, and false alarms (responses to lures) were generally low. When a matched auditory stimulus was selected the participant was significantly more likely to select the more pleasant or familiar of the two sounds. These results will inform stimulus selection and matching in a future VWP study. Using subjective factors for cross-modal matching is one possible approach to overcoming difficulties in norming real-world audio-visual events.
Meeting abstract presented at VSS 2017