September 2017
Volume 17, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2017
Cross-modal Matching as a Means of Stimulus Norming for the Visual World Paradigm
Author Affiliations
  • Kelly Dickerson
    Human Research and Engineering Directorate, US Army Research Laboratory
  • Brandon Perelman
    Human Research and Engineering Directorate, US Army Research Laboratory
  • Peter Gerhardstein
    Department of Psychology, Binghamton University
Journal of Vision August 2017, Vol.17, 200. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Kelly Dickerson, Brandon Perelman, Peter Gerhardstein; Cross-modal Matching as a Means of Stimulus Norming for the Visual World Paradigm. Journal of Vision 2017;17(10):200.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The visual world paradigm (VWP) has been used to evaluate how a variety of cognitive and contextual factors influence the deployment of visual spatial attention. Previous VWP studies have employed linguistic cues, where a spoken or written target word immediately precedes a search array. This method of cuing is effective and easily controlled but lacks ecological validity. A more ecologically valid way to cue a visual event would be to use environmental sounds as cues. While this answer to ecological validity seems straightforward, matching audio and visual cues when those cues are meant to represent real objects is methodologically challenging. The present study attempts to meet this challenge by using subjective ratings of stimulus attributes as our matching variables. This study contains 150 stimuli (50 images, 100 sounds) that were rated by 10 participants for pleasantness and familiarity. For each stimulus participants were also asked for an open-ended identification response. Following the subjective ratings phase of the experiment participants completed a 4AFC task where each of the 50 images was presented with two matched (target) sounds and two mismatched (lures) sounds. There were no significant differences between sounds and images for pleasantness or familiarity. There was a significant difference in identification accuracy between sounds and images, with images being slightly more accurately identified than sounds. In the 4AFC task participants were highly accurate at selecting one of the two matched sounds, and false alarms (responses to lures) were generally low. When a matched auditory stimulus was selected the participant was significantly more likely to select the more pleasant or familiar of the two sounds. These results will inform stimulus selection and matching in a future VWP study. Using subjective factors for cross-modal matching is one possible approach to overcoming difficulties in norming real-world audio-visual events.

Meeting abstract presented at VSS 2017


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.