Abstract
Multisensory information is suggested to be integrated based on reliability-based model. Recently, it was suggested that perception and adaptation may be underlying different processes. Multisensory adaptation may follow a different integration model. The current study, therefore, investigates the possible integration models that explain audiovisual integration in multisensory emotion adaptation. Eighteen participants were tested in a 2-alternative forced choice adaptation paradigm. On each trial, the adaptor was shown for 3 seconds, followed by an emotion judgment task on a face morphed between happy and angry expressions. The noise information in visual adaptors was manipulated: the facial emotion recognition rate under noise was 60% - 80% as determined in a pre-experiment block. There were six conditions in the main experiment: 3 Unisensory conditions (with Auditory/Clear Visual/Noisy Visual adaptor), 2 Multisensory conditions (with Auditory with Clear Visual/Noisy Visual adaptor), and the baseline (no adaptor). All adaptors depicted anger emotion. Except for the auditory adaptor, all the other 4 adaptors (including the Noisy Visual) generated significant facial emotion aftereffects. By comparing the Bayesian Information Criteria of the three multisensory models obtained from within-subject multiple linear regressions (BIC visual-dominant = -130.299; BIC fixed-ratio = -136.125; BIC reliability-based = -130.740), our results suggested that multisensory emotion adaptation is best explained by the fixed-ratio model, with visual input contributing 62.8% regardless of the presence of the visual noise, Est./S.E. = 5.081, p < .001. Our findings indicated that in emotion adaptation, the multisensory percept is a weighted sum of the visual input and the auditory input, but the values of the weights are not determined by the reliability of the source of information. This is in contrast with multisensory emotion perception. We discuss the possibility that, on top of the perception of emotion, neural areas that are responsible for higher level executive functions are also adapted during multisensory adaptation.
Meeting abstract presented at VSS 2017