Abstract
Sensory substitution is a process by which information from one sensory modality is extracted and applied to another sensory modality. Previous fMRI studies have suggested that sensory substitution causes increased functional connectivity between auditory and visual cortices. The timing of such information transfer between sensory modalities, however, is currently unknown. The present study investigated the effects of auditory-visual sensory substitution in intact individuals by recording event related potentials (ERPs). ERPs to auditory 'soundscapes' and visual shapes were recorded before and after sensory substitution training in an experimental group using a simple sound-to-shape matching task. These participants were trained on auditory-visual pairs created by the Meijer sound-to-image conversion algorithm. ERPs to the same stimuli were recorded in a control group, before and after training, using the same matching task, however, control participants were trained to memorize random sound-shape pairs. First, to confirm that sensory substitution training was successful both groups of participants completed a transfer test on completely novel sound-shape pairs. Participants in the experimental group identified novel visual shapes based on the translated auditory information with above chance accuracy (72%) while control subjects performed at chance levels (50%). ERPs elicited by sounds and shapes were compared before versus after training within each group and the resulting ERP differences were then compared between groups. Two specific effects of sensory substitution were isolated. The auditory P2 component was enhanced post-training versus pre-training in both groups, but this difference was significantly larger in the experimental group compared to the control group. Similarly, ERPs to visual stimuli revealed increased visual N1 amplitudes as a function of training in both groups, but this difference was more pronounced in the experimental group. Taken together, the latency of these ERP modulations suggest that sensory substitution may be mediated by early interactions between auditory and visual cortical areas.
Meeting abstract presented at VSS 2014