Abstract
Visual recognition of complex shapes might be accomplished by the integration of information from simpler components (e.g. fragments, or geons). Equivalent components might exist also for the recognition of complex body movements. This is suggested by results in motor control showing that complex motor behavior is organized in terms of simpler components (synergies) that depend only of subsets of the available degrees of freedom (joints). Our experiment tested how the visual system integrates information from components of emotional body movements that were either congruent or incongruent with components recovered from motor behavior.
METHOD: Based on motion capture data from lay actors performing actions with different emotional affects, we extracted spatio-temporal components that are specific for the expression of individual emotions (applying an algorithm that combines ICA and sparse feature learning). We generated point light stimuli specifying variable amounts of information about emotional style for different joint combinations. These components did or did not match the components extracted from the motor behavior. By comparing perceptual performance with predictions from a Bayesian cue fusion model, we assessed if subjects' perception integrated the information components in a statistically optimal way.
RESULTS: Performance was very close to an ideal Bayesian integration of information components, while compatibility of the components with motor behavior seems not to be critical. This suggests that Bayesian cue integration might be relevant for the visual recognition of emotion. However, the relevant information components might be different from the components that are relevant in motor execution.
Human Frontier Science Program, DFG, Volkswagen Foundation