September 2005
Volume 5, Issue 8
Vision Sciences Society Annual Meeting Abstract  |   September 2005
Looking in the mirror does not prevent multimodal integration
Author Affiliations
  • Hannah B. Helbig
    Max Planck Institute for Biological Cybernetics, Tübingen
  • Marc O. Ernst
    Max Planck Institute for Biological Cybernetics, Tübingen
Journal of Vision September 2005, Vol.5, 750. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Hannah B. Helbig, Marc O. Ernst; Looking in the mirror does not prevent multimodal integration. Journal of Vision 2005;5(8):750.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Ernst & Banks (2002) showed that humans integrate visual and haptic signals in a statistically optimal way if they are derived from the same spatial location. Integration seems to be broken if there is a spatial discrepancy between the signals (Gepshtein et al., VSS 04).

Can cognitive factors facilitate integration even when the signals are presented at two spatial locations? We conducted two experiments, one in which visual and haptic information was presented at the same location. In the second experiment, subject looked at the object through a mirror while touching it. This way there was a spatial offset between the two information sources. If cognitive factors are sufficient for integration to occur, i.e. knowledge that the object seen in the mirror is the same as the one touched, we expect no difference between the two experimental results. If integration breaks due to the spatial discrepancy we expect subjects' percept to be less biased by multimodal information.

To study integration participants looked at an object through a distortion lens. This way, for both the “mirrored” and “direct vision” conditions there was a slight shape conflict between the visual and haptic modalities. After looking at and feeling the object simultaneously participants reported the perceived shape by either visually or haptically matching it to a reference object.

Both experiments revealed that the shape percept was in-between the haptically and visually specified shapes. Importantly, there was no significant difference between the two experimental results regardless of whether subjects matched the shape visually or haptically. However, we found a significant difference between matching by touch and matching by vision. Haptic judgments are biased towards the haptic input and vice versa.

In conclusion, multimodal signals seem to be combined if observers have high-level cognitive knowledge about the signals belonging to the same object, even when there is a spatial discrepancy.

Helbig, H. B. Ernst, M. O. (2005). Looking in the mirror does not prevent multimodal integration [Abstract]. Journal of Vision, 5(8):750, 750a,, doi:10.1167/5.8.750. [CrossRef]
 Supported by the Max Planck Society and by the EU-Project TOUCH-HapSys (IST-2001-38040)

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.