August 2014
Volume 14, Issue 10
Vision Sciences Society Annual Meeting Abstract  |   August 2014
Electrophysiological Dynamics of Auditory-Visual Sensory Substitution
Author Affiliations
  • Christian Graulty
    Psychology, Reed College
  • Orestis Papaioannou
    Psychology, Reed College
  • Phoebe Bauer
    Psychology, Reed College
  • Michael Pitts
    Psychology, Reed College
  • Enriqueta Canseco-Gonzalez
    Psychology, Reed College
Journal of Vision August 2014, Vol.14, 438. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Christian Graulty, Orestis Papaioannou, Phoebe Bauer, Michael Pitts, Enriqueta Canseco-Gonzalez; Electrophysiological Dynamics of Auditory-Visual Sensory Substitution. Journal of Vision 2014;14(10):438. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Sensory substitution is a process by which information from one sensory modality is extracted and applied to another sensory modality. Previous fMRI studies have suggested that sensory substitution causes increased functional connectivity between auditory and visual cortices. The timing of such information transfer between sensory modalities, however, is currently unknown. The present study investigated the effects of auditory-visual sensory substitution in intact individuals by recording event related potentials (ERPs). ERPs to auditory 'soundscapes' and visual shapes were recorded before and after sensory substitution training in an experimental group using a simple sound-to-shape matching task. These participants were trained on auditory-visual pairs created by the Meijer sound-to-image conversion algorithm. ERPs to the same stimuli were recorded in a control group, before and after training, using the same matching task, however, control participants were trained to memorize random sound-shape pairs. First, to confirm that sensory substitution training was successful both groups of participants completed a transfer test on completely novel sound-shape pairs. Participants in the experimental group identified novel visual shapes based on the translated auditory information with above chance accuracy (72%) while control subjects performed at chance levels (50%). ERPs elicited by sounds and shapes were compared before versus after training within each group and the resulting ERP differences were then compared between groups. Two specific effects of sensory substitution were isolated. The auditory P2 component was enhanced post-training versus pre-training in both groups, but this difference was significantly larger in the experimental group compared to the control group. Similarly, ERPs to visual stimuli revealed increased visual N1 amplitudes as a function of training in both groups, but this difference was more pronounced in the experimental group. Taken together, the latency of these ERP modulations suggest that sensory substitution may be mediated by early interactions between auditory and visual cortical areas.

Meeting abstract presented at VSS 2014


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.