Abstract
In daily life, we are bombarded with an abundance of sensory information from several modalities. Due to internal noise in the brain and external noise in the environment, two sensory cues originating from the same object will not perfectly agree with each other in terms of space and time. Therefore, to maintain a coherent percept of the world, the brain integrates cues that are likely to come from the same source and segregates those that are not. The goal of the current study is to understand how temporal discrepancies impact the influence of spatially discrepant visual stimuli on auditory localization as well as perceived unity of the spatially and temporally discrepant cues. To this aim, we presented participants with a visual (a Gaussian blob) and an auditory stimulus (a brief burst of white noise) with various spatial and temporal discrepancies between them. Participants first localized the auditory stimulus and then reported whether they perceived the two stimuli as originating from the same source. We computed the ventriloquism effect, the size of auditory shifts towards the accompanying visual stimulus by comparing the auditory localization responses to those when the auditory stimulus was presented alone in a control experiment. As expected, the bias, the ventriloquism effect relative to the spatial discrepancy, first increased and then decreased as the two stimuli were further apart in space. Importantly, the bias also declined as the two stimuli were more temporally discrepant. Consistent results were observed in the unity judgments: the probability of reporting a common cause fell off as a function of both spatial and temporal discrepancy. Our results provide robust evidence that temporal alignment is taken into account even in spatial tasks and thus spatial and temporal discrepancy are driving factors that determine audiovisual integration and inference of a common cause.