Abstract
Establishing object correspondence over time ("Which object went where?") is of central importance for a meaningful interpretation of the surrounding environment. Here, we study auditory contributions to this process using the bouncing/streaming paradigm wherein two discs move toward each other, superimpose, and then move apart. Critically, this event is ambiguous with regard to object correspondence as it is consistent with the interpretation of two discs streaming past each other as well as two discs bouncing off each other. When presented in silence, human observers tend to perceive streaming discs; however, a brief beep that coincides with the moment of visual overlap biases this impression toward bouncing. In four experiments, we tested the hypothesis that this crossmodal interaction is primarily mediated by low-level magnitude-based rather than high-level semantic-based processing. To do so, we orthogonally manipulated the number and semantic category of auditory transients. Specifically, different combinations of onsets and offsets generate qualitatively different events with distinct meanings; a single auditory transient can be a tone onset or a tone offset, and a pair of transients can be a brief tone (onset+offset) or a brief gap (offset+onset). The proportion of seeing bouncing increased with an increasing number of auditory transients (0 vs. 1 vs. 2) regardless of the sound's semantic category. For example, a tone onset and a tone offset were equally effective (relative to no transients), and a brief tone (onset+offset) and a brief gap (offset+onset) were equivalently more effective. We identified a critical window of ± 200 ms around the visual overlap; a longer tone whose offset occurred outside the window was only as effective as a single onset. These results suggest that a simple additive integration of auditory transients within the critical time window primarily mediates the auditory biasing of visual bouncing percepts.
Meeting abstract presented at VSS 2018