September 2005
Volume 5, Issue 8
Vision Sciences Society Annual Meeting Abstract  |   September 2005
From independence to fusion: A comprehensive model for multisensory integration
Author Affiliations
  • Marc O. Ernst
    Max Planck Institute for Biological Cybernetics, Tübingen, Germany
Journal of Vision September 2005, Vol.5, 650. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Marc O. Ernst; From independence to fusion: A comprehensive model for multisensory integration. Journal of Vision 2005;5(8):650.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Recently we demonstrated that humans integrate visual and haptic information in a statistically optimal way (Ernst & Banks, 2002). I.e., subjects make optimal use of the information provided in order to reach the decision necessary for the task. As shown by Hillis et al. (2002), however, this does not necessarily imply that the sensory signals are completely fused into a unified percept. If subjects would completely fuse the signals, by definition, they would not at all retain access to the incoming sources of information. In contrast, Hillis et al. found some weaker form of interaction between the sensory signals. The degree of interaction between the sensory signals can be taken as definition for the strength of coupling between the signals: there is no coupling if the signals are independent; there is maximal coupling if the signals are fused. Using Bayesian decision theory I here propose a comprehensive model that can account for both results. The prior used in this model represents the probability of the physical relationship (mapping) between the signals derived by the sensory systems. This probability has a narrow tuning if the mapping between the physical signals is relatively constant (such as e.g., the mapping between texture and disparity signals). If the mapping changes easily (such as e.g. the mapping between visual and haptic signals), the distribution of possible mappings reflected in such a prior is wider. This is called the “coupling prior” because the tuning of the prior will determine the level of interaction, i.e., the strength of coupling. I will further present data from a visual-haptic discrimination experiment that will support these theoretical considerations. Taken together, I propose that such a Bayesian model that uses a “Coupling Prior” for describing sensory interactions is a convenient theoretical framework for understanding multisensory integration as a continuous process between independence and complete fusion.

Ernst, M. O. (2005). From independence to fusion: A comprehensive model for multisensory integration [Abstract]. Journal of Vision, 5(8):650, 650a,, doi:10.1167/5.8.650. [CrossRef] [PubMed]
 This work was supported by the 5th Framework IST European Program (Touch-Hapsys, IST-2001-38040)

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.