Modality switch effects and the impact of predictability of the sensory environment.

Abstract

The modality switch effect (MSE) refers to the increase in reaction time that occurs when switching between different sensory modalities (Spence, Nicholls, & Driver, 2001). Our perceptual system is thought to continually make predictions about upcoming sensory information and this may contribute to our reluctance to switchmodality when the probability of a repeated stimulus is high. The MSE has traditionally been tested in the context of bimodal detection tasks. Here, we tested MSEs in a more complex, trimodal environment where the probability of a repeated stimulus is lower. We hypothesized two possible outcomes: 1) increasing the complexity by adding a third sensory modality could lead to greater MSEs or 2) MSEs could decrease due to the increased uncertainty that leads us to make fewer predictions. In 12 healthy adults, we measured RTs to bisensory (audio and/or visual) and trisensory (audio, visual and/or tactile) signals presented on the left index finger in random order. MSEs for the bisensory and the trisensory condition were calculated as the area between the cumulative distribution functions of the RT distributions for repeat (same modality) and switch (different modalities) trials. Comparing switch costs between the bisensory and the trisensory context, we found lower MSEs in the trisensory context for audio and visual stimuli, whereas the opposite pattern was seen for the audio-visual stimuli. Overall, results show that participants can more easily switch between modalities when the complexity of the sensory environment increases and a stimulus repetition becomes less likely. These findings suggest that expectancy-driven effects affect our sensory processing but the uncertainty of a complex environment likely forces us to make fewer predictions. We become less “sticky” to the previous sensory modality and engage in a more sensory-driven bottom-up mode of processing.