September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Auditory information facilitates sensory evidence accumulation during visual object recognition
Author Affiliations & Notes
  • Jamal R Williams
    Department of Psychology, University of California, San Diego
  • Viola S Störmer
    Department of Psychology, University of California, San Diego
Journal of Vision September 2019, Vol.19, 20c. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jamal R Williams, Viola S Störmer; Auditory information facilitates sensory evidence accumulation during visual object recognition. Journal of Vision 2019;19(10):20c. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Research has shown that early visual processing is enhanced when preceded by spatially co-localized sounds (Störmer et al 2009). While this work has shown that visual perception for low-level stimuli (Gabor patches) is cross-modally influenced by equally low-level auditory stimuli (noise bursts), it is not known whether such cross-modal influences exist at higher processing stages. Here, we asked whether and how complex, real-world sounds facilitate visual processing of real-world objects. We designed a novel cross-modal paradigm that tracks the amount of sensory evidence needed to accurately detect real-world objects. We then compared object recognition performance when these visual objects were preceded by congruent or incongruent auditory information. Each trial started with the presentation of a 2-s sound (e.g., a train) which was immediately followed by the presentation of a continuous visual stimulus. This stimulus started out as an incoherent, grayscale noise patch that slowly and continuously decreased its noise level to eventually form a clearly visible object. On half of the trials the object was congruent (e.g., a train), and on the remaining trials it was incongruent (e.g., an elephant) with the sound. Participants were instructed to press a key as soon as they recognized the object, which also immediately ceased the stimulus presentation. Subsequently participants indicated which object they saw from two, within-category, highly similar objects positioned on either side of fixation. Participants (n=26) showed shorter reaction times when initially recognizing the object on congruent trials relative to incongruent trials (p=0.005). Overall, our results show that visual object recognition can be facilitated when preceded by congruent vs. incongruent auditory information. Broadly, this indicates that naturalistic sounds enhance the perceptual processing of visual objects.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.