Abstract
Goals: To study how perception of a stimulus in one modality (vision or audition) is influenced by the simultaneous presence of a stimulus in the other modality. To investigate whether such influences have perceptual or cognitive origins, we tested for aftereffects in one modality after adaptation in the other modality. Methods: We used three configurations: I. Approach/Recede (A/R): Visual stimuli were two low-contrast concentric counter-phase flickering sinusoidal (CPFS) gratings moving in opposite directions; their relative contrasts were varied to produce an expanding or shrinking stimulus. The loudness of auditory stimuli increased or decreased over time. II. Left-right (L/R): CPFS gratings moved right or left. Auditory stimuli moved right or left, by changing the relative loudness of two lateral speakers. III. Up/Down (U/D): CPFS gratings moved up or down. The pitch of auditory stimuli glided up or down. For each configuration, we studied both directions: visual stimuli influencing auditory stimuli and vice versa. For each combination of configuration and direction, we used simultaneous and cross-modal adaptation/test stimuli. Results: A/R and L/R: Visual stimuli influenced auditory perception in both simultaneous and adapt/test cases; this influence was larger in the A/R than the L/R case. This indicates perceptual origins for these effects. Some influence of auditory stimuli on visual perception was present only in the simultaneous case, indicating cognitive origins. U/D: The only strong crossmodal influence was in the auditory-to-visual direction for the simultaneous case [Maeda, Kanai, Shimojo 2004]. Very weak effects for the simultaneous case in the visual-to-auditory direction were observed. Conclusions: Cross-modal interactions between vision and audition are complex and do not follow a uniform rule. Most such interactions occur for simultaneous stimulation, and do not produce significant aftereffects, with the exception of auditory aftereffects following A/R visual adaptation [Kitagawa, Ichihara 2002], and weaker such aftereffects in the L/R case.