Purchase this article with an account.
Adam J. Ecker, Laurie M. Heller; Audio-Visual Cue Combination in Depth Perception. Journal of Vision 2004;4(8):699. doi: https://doi.org/10.1167/4.8.699.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
The perceived path of a ball moving in 3D-space is altered solely by the addition of an auditory cue. All visual stimuli consist of a ball moving in a 3-dimensional box. Critically, each video is paired with one of three sound conditions: No sound, Roll sound, or Jump Sound. Dependent on the sound cue, Subjects perceive the ball as either 1) jumping in the frontal plane, or 2) rolling back in depth on the box's floor. The perceptual effect of the audio cues is large, and robust to changes of path shape. Follow-up experiments determine that the reported shift in path perception truly reflects a perceptual experience, rather than a post-perceptual reporting bias or a deliberate decision process. In a secondary experiment, rather than directly judge the path of the ball, Subjects judge a different aspect of the stimulus: the speed of the ball. The floor of the box was extended in depth by a factor of three so that if the ball were rolling it would be traveling much faster than if the ball were jumping (even though the 2D on-screen speed is the same). As predicted, Subjects select a ball paired with a rolling sound as faster than a ball paired with a jumping sound, even when the videos of two balls are identical. From these results we can infer that the Subject indeed sees a difference in the paths of the two otherwise identical videos, and this difference is due to the addition of a sound cue. The experiment opposes visual cues (path shape, size change, shadow) against auditory path cues and then measures the combined perceptual effect using a single metric. The results demonstrate that, in much the same way that a shadow cues can induce apparent motion in depth, so too can an appropriate audio cue alter depth perception. Should vision researchers consider adding “Audio” to their menagerie of ocular and pictorial depth cues?
This PDF is available to Subscribers Only