Abstract
The temporal relationship between signals from different modalities is a key factor for multisensory integration. In a series of recent studies we demonstrated that the similarity in temporal fine-structure between visual and auditory stimuli plays a leading role in solving the multisensory correspondence problem. Here we investigate the role of cross-correlation in multisensory integration by combining standard psychophysical techniques with reverse correlation analyses. We presented a series of auditory and visual signals that were either correlated in time or not, and we experimentally manipulated the delay between the two signals. The signals had a complex stochastic temporal structure and consisted of trains of impulses (clicks or flashes) presented over a 1.5s interval. In two separate experiments, participants reported the relative order of presentation (temporal order judgment) or whether the stimuli appeared to share a common cause or not (causality judgment). Standard psychophysical analyses revealed higher sensitivity to temporal delays for correlated signals in both causality and temporal order judgment. Notably, sensitivity to temporal delays was virtually identical in the two tasks. Additionally, in line with previous findings, temporally correlated signals were more likely reported to have a common source. Next, we used reverse correlation techniques to calculate the classification images for both tasks using the cross-correlation between visual and auditory signals. Results demonstrate that cross-correlation is indeed the factor underlying participants judgments: The sign of the lag of the cross-correlation peak correlates with temporal order judgments, whereas the amount of cross-correlation at short lags correlates with causality judgments. Overall, the present results demonstrate the primary role of cross-correlation cues in multisensory temporal processing. These findings will be discussed in the light of recent models of sensory cue integration.
Meeting abstract presented at VSS 2014