September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Modality switch effects and the impact of predictability of the sensory environment.
Author Affiliations & Notes
  • Maria Bianca Amadeo
    Unit for Visually Impaired People, Italian Institute of Technology, ITALY
    Department of Informatics, Bioengineering, Robotics and Systems Engineering, Università degli Studi di Genova, ITALY
  • Michael C. Crosse
    The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Montefiore Medical Center, NY
    The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY
  • Monica Gori
    Unit for Visually Impaired People, Italian Institute of Technology, ITALY
  • Claudio Campus
    Unit for Visually Impaired People, Italian Institute of Technology, ITALY
  • John J. Foxe
    The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Montefiore Medical Center, NY
    The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY
    Department of Neuroscience, The Ernest J. Del Monte Institute for Neuroscience, University of Rochester School of Medicine and Dentistry
  • Sophie Molholm
    The Sheryl and Daniel R. Tishman Cognitive Neurophysiology Laboratory, Department of Pediatrics, Albert Einstein College of Medicine, Montefiore Medical Center, NY
    The Dominick P. Purpura Department of Neuroscience, Rose F. Kennedy Intellectual and Developmental Disabilities Research Center, Albert Einstein College of Medicine, Bronx, NY
Journal of Vision September 2019, Vol.19, 20b. doi:https://doi.org/10.1167/19.10.20b
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Maria Bianca Amadeo, Michael C. Crosse, Monica Gori, Claudio Campus, John J. Foxe, Sophie Molholm; Modality switch effects and the impact of predictability of the sensory environment.. Journal of Vision 2019;19(10):20b. doi: https://doi.org/10.1167/19.10.20b.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The modality switch effect (MSE) refers to the increase in reaction time that occurs when switching between different sensory modalities (Spence, Nicholls, & Driver, 2001). Our perceptual system is thought to continually make predictions about upcoming sensory information and this may contribute to our reluctance to switchmodality when the probability of a repeated stimulus is high. The MSE has traditionally been tested in the context of bimodal detection tasks. Here, we tested MSEs in a more complex, trimodal environment where the probability of a repeated stimulus is lower. We hypothesized two possible outcomes: 1) increasing the complexity by adding a third sensory modality could lead to greater MSEs or 2) MSEs could decrease due to the increased uncertainty that leads us to make fewer predictions. In 12 healthy adults, we measured RTs to bisensory (audio and/or visual) and trisensory (audio, visual and/or tactile) signals presented on the left index finger in random order. MSEs for the bisensory and the trisensory condition were calculated as the area between the cumulative distribution functions of the RT distributions for repeat (same modality) and switch (different modalities) trials. Comparing switch costs between the bisensory and the trisensory context, we found lower MSEs in the trisensory context for audio and visual stimuli, whereas the opposite pattern was seen for the audio-visual stimuli. Overall, results show that participants can more easily switch between modalities when the complexity of the sensory environment increases and a stimulus repetition becomes less likely. These findings suggest that expectancy-driven effects affect our sensory processing but the uncertainty of a complex environment likely forces us to make fewer predictions. We become less “sticky” to the previous sensory modality and engage in a more sensory-driven bottom-up mode of processing.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×