Effects of concurrent auditory stimulation on human visual cortex

Abstract

There is increasing evidence that visual processing can be modified by sensory stimulation in another modality. Traditionally it has been assumed that such multisensory integration occurs after sensory signals have been extensively processed in unisensory cortical regions. However, recent studies have shown multisensory convergence at early stages of visual processing. An important, but unresolved issue is how such neural interactions might be reflected in conscious visual perception. Here, we investigate the association of V1 activity with conscious perception during a crossmodal illusion. The auditory induced flash illusion shows that conscious visual perception can be modified by concurrent irrelevant auditory stimulation. A single visual flash accompanied by two bleeps is incorrectly perceived as two flashes (‘fission’). Similarly, two flashes can be incorrectly perceived as one flash when accompanied by one bleep (‘fusion’). Signal detection analysis suggests that this is a perceptual effect rather than response bias. Activity in human V1 is increased in association with the ‘fission’ illusion suggesting that V1 activity is modulated by sound. However, it is not clear whether such modulation represents a correlate of altered visual perception or a response to mismatch between the two sensory modalities. Here, we used functional MRI in conjunction with retinotopic mapping to examine the cortical response to both the ‘fission’ and ‘fusion’ illusions. We present data from both early visual areas and across the whole brain suggesting that V1 does indeed follow conscious perception rather than the physical stimulus and providing further evidence for the neural mechanisms underlying this crossmodal illusion.