August 2016
Volume 16, Issue 12
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2016
ON and OFF channels in auditory-visual integration
Author Affiliations
  • Cesare Parise
    University of Bielefeld
  • Martin Banks
    University of California, Berkeley
  • Marc Ernst
    University of Bielefeld
Journal of Vision September 2016, Vol.16, 861. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Cesare Parise, Martin Banks, Marc Ernst; ON and OFF channels in auditory-visual integration. Journal of Vision 2016;16(12):861. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Unisensory information is of course processed before integration with signals from other senses. Sensory input in the visual and auditory systems are segregated into ON and OFF pathways that respond differentially to changes in input intensity. Specifically, an increase in intensity leads to an increment in neural response in the ON pathway, and decrement in the OFF pathway. The opposite occurs with an intensity decrease. We investigated how humans combine auditory-visual stimuli that consist of step changes in intensity. Subjects made temporal-order judgments. Stimuli consisted either of increments (ON stimuli) or decrements (OFF stimuli) in intensity over time. ON and OFF stimuli were paired in all possible combinations. In some pairings, the stimuli both increased or decreased in intensity. In other pairings, one increased while the other decreased. Subjects reported the modality that seemed to change first. Recently, we developed a model in which multisensory integration relies on elementary units analogous to the motion-energy units observed in insect visual systems. In its original form, the model consisted only of ON channels, so an increment in intensity in one modality paired with a decrement in the other would produce an inversion of perceived temporal order reminiscent of the "reverse phi" effect in visual motion perception. The results from the temporal-order judgment task showed that subjects could accurately identify the temporal order of the two signals even when the intensity in the two modalities changed in opposite directions. This suggests that unisensory information is separately processed in ON and OFF channels, yielding an estimate of the time of change without regard to sign, before the information is integrated. A simple model in which unisensory information is first segregated into ON and OFF channels, low-pass filtered, and then recombined before feeding into an energy unit yields behavior consistent with our experimental observations.

Meeting abstract presented at VSS 2016


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.