August 2010
Volume 10, Issue 7
Vision Sciences Society Annual Meeting Abstract  |   August 2010
Integration and the perceptual unity of audio-visual utterances
Author Affiliations
  • Shoko Kanaya
    The University of Tokyo
  • Kazuhiko Yokosawa
    The University of Tokyo
Journal of Vision August 2010, Vol.10, 891. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Shoko Kanaya, Kazuhiko Yokosawa; Integration and the perceptual unity of audio-visual utterances. Journal of Vision 2010;10(7):891.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

When multisensory stimuli are unified as an event from one common source, the integration of these stimuli is considered to be facilitated. Some studies estimated the degree of perceptual unification by using participants' subjective reports concerning whether they felt the stimuli were coming together, although this conversely reflects the occurrence of integration. We investigated whether integrated speech perception affects the unification of audio-visual utterances. The visual stimulus was a movie of two bilateral faces uttering /pa/ and /ka/ respectively, and the auditory stimulus was one channel of voiced utterance of /pa/ or /ka/ from one of two speakers. When the auditory utterance is /pa/, audio-visual integration might elicit speech perception altered like /ta/, based on the McGurk illusion in Japanese participants. In addition, whether the auditory and visual stimuli of the corresponding spatial locations were consistent was manipulated to make the unification distinct or not. In our task, participants reported both the face perceived as the talker and the syllable perceived to be uttered by the talker. In this situation, participants would report the unified information source as the talker. If the outcome of integrated speech perception affects the unification, the judgment about the talker should be related to the auditory utterance as the presence or absence marker of the illusory perception. The results showed that the perceived talker was not affected by the kind of auditory utterance, although the participants reported illusory hearing only for the auditory /pa/. When the audio-visual stimuli were presented repeatedly for more distinct unification, the judgment about the talker was influenced only by the frequency of the presentation, and not by the auditory utterance. Moreover, these results were independent of the spatially manipulated ambiguity. These findings indicate that the perception of unity of audio-visual information is the cause and not the result of integration.

Kanaya, S. Yokosawa, K. (2010). Integration and the perceptual unity of audio-visual utterances [Abstract]. Journal of Vision, 10(7):891, 891a,, doi:10.1167/10.7.891. [CrossRef]

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.