Learning associations between simple visual and auditory features

Abstract

It has recently been shown that exposure to new visual features can lead to a strengthening of the representation of those features in adult humans (Falconbridge et al. VSS '07). Is this plasticity confined to features within visual modality or does it extend to crossmodal features? The aim of this study was to discover if passive exposure to novel auditory-visual feature correlations can lead to the learning of crossmodal features in human adult sensory system. METHODS: We performed two experiments. In each experiment, we exposed subjects to a visual feature that was coupled with an auditory feature during an exposure phase, and we tested the detection/discrimination of the visual feature in absence or presence of the auditory feature before and after the exposure. In both experiments, one visual feature (V₁) was coupled with a specific tone (A₁) and another visual feature (V₂) was presented in silence. In Experiment 1, V₁ and V₂ corresponded to two sinusoidal gratings of orthogonal orientation and the task during test sessions was a 2IFC contrast detection task. In Experiment 2, the V₁ and V₂ corresponded to two orthogonal motion directions, and the task during test sessions was 2AFC motion discrimination. RESULTS: Exposure caused a relative improvement in visual performance for the exposed pairing V₁A₁ compared to V₁, V₂, and V₂A₁ conditions. Comparing d' and criterion for auditory-coupled conditions vs. silent conditions before and after exposure suggests that this learning is due to a change in sensitivity rather than a change in bias. CONCLUSIONS: These results suggest that auditory and visual stimuli are integrated at an early stage of visual processing, and that low-level AV features can be acquired even in adulthood. Exposure to a coupled auditory stimulus can facilitate detectability and discriminability of a visual stimulus, specific to the exposed feature pairing.