Abstract
When multisensory stimuli are unified as an event from one common source, the integration of these stimuli is considered to be facilitated. Some studies estimated the degree of perceptual unification by using participants' subjective reports concerning whether they felt the stimuli were coming together, although this conversely reflects the occurrence of integration. We investigated whether integrated speech perception affects the unification of audio-visual utterances. The visual stimulus was a movie of two bilateral faces uttering /pa/ and /ka/ respectively, and the auditory stimulus was one channel of voiced utterance of /pa/ or /ka/ from one of two speakers. When the auditory utterance is /pa/, audio-visual integration might elicit speech perception altered like /ta/, based on the McGurk illusion in Japanese participants. In addition, whether the auditory and visual stimuli of the corresponding spatial locations were consistent was manipulated to make the unification distinct or not. In our task, participants reported both the face perceived as the talker and the syllable perceived to be uttered by the talker. In this situation, participants would report the unified information source as the talker. If the outcome of integrated speech perception affects the unification, the judgment about the talker should be related to the auditory utterance as the presence or absence marker of the illusory perception. The results showed that the perceived talker was not affected by the kind of auditory utterance, although the participants reported illusory hearing only for the auditory /pa/. When the audio-visual stimuli were presented repeatedly for more distinct unification, the judgment about the talker was influenced only by the frequency of the presentation, and not by the auditory utterance. Moreover, these results were independent of the spatially manipulated ambiguity. These findings indicate that the perception of unity of audio-visual information is the cause and not the result of integration.