December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Audiovisual integration across space and time
Author Affiliations & Notes
  • Fangfang Hong
    Department of Psychology, New York university
  • Jiaming Xu
    Department of Psychology, New York university
  • Megha Kalia
    Department of Psychology, New York university
  • Stephanie Badde
    Department of Psychology, Tufts University
  • Michael Landy
    Department of Psychology, New York university
    Center for Neural Science, New York University
  • Footnotes
    Acknowledgements  NIH EY08266 (MSL)
Journal of Vision December 2022, Vol.22, 4438. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Fangfang Hong, Jiaming Xu, Megha Kalia, Stephanie Badde, Michael Landy; Audiovisual integration across space and time. Journal of Vision 2022;22(14):4438.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

In daily life, we are bombarded with an abundance of sensory information from several modalities. Due to internal noise in the brain and external noise in the environment, two sensory cues originating from the same object will not perfectly agree with each other in terms of space and time. Therefore, to maintain a coherent percept of the world, the brain integrates cues that are likely to come from the same source and segregates those that are not. The goal of the current study is to understand how temporal discrepancies impact the influence of spatially discrepant visual stimuli on auditory localization as well as perceived unity of the spatially and temporally discrepant cues. To this aim, we presented participants with a visual (a Gaussian blob) and an auditory stimulus (a brief burst of white noise) with various spatial and temporal discrepancies between them. Participants first localized the auditory stimulus and then reported whether they perceived the two stimuli as originating from the same source. We computed the ventriloquism effect, the size of auditory shifts towards the accompanying visual stimulus by comparing the auditory localization responses to those when the auditory stimulus was presented alone in a control experiment. As expected, the bias, the ventriloquism effect relative to the spatial discrepancy, first increased and then decreased as the two stimuli were further apart in space. Importantly, the bias also declined as the two stimuli were more temporally discrepant. Consistent results were observed in the unity judgments: the probability of reporting a common cause fell off as a function of both spatial and temporal discrepancy. Our results provide robust evidence that temporal alignment is taken into account even in spatial tasks and thus spatial and temporal discrepancy are driving factors that determine audiovisual integration and inference of a common cause.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.