Abstract
Humans are highly adept at multisensory processing of object shape in both vision and touch - furthermore, unfamiliar objects with complex shape properties are learned easily in both modalities. Here, we used psychophysics and fMRI to investigate neural correlates of visual and haptic processing of novel, three-dimensional stimuli. The experiment used 9 objects that varied parametrically in shape, forming a cross in object-parameter space. First, two groups of participants judged object similarities either visually or haptically (90 trials, each n>30). Similarity ratings were averaged across participants to obtain one group similarity matrix in both modalities. The reconstructed behavioral space conformed well to the underlying object-parameter space in both modalities. After two days, we invited 11 participants from the visual group to perform an fMRI-experiment, consisting of two blocks of a 1-back recognition task. The first block was run visually in the familiar modality, whereas the second block was run haptically, thus in a new modality. Objects were presented successively in the scanner for 6sec (with an ISI of 16sec including response period). We selected anatomically-defined Brodmann areas BA17, BA18 as well as the localizer-defined lateral-occipital-complex (LOC) as regions of interest for a correlation analysis of behavioral similarity ratings with neural responses. Neural activations in each area were averaged across participants to obtain group visual and haptic neural similarity matrices. In the visual block, both BA18 (r=.6585 p<.000) and early visual cortex, BA17 (r=.6875 p<.000) showed high correlations with behavior. Interestingly, this correlation was found also for the haptic block for both BA18 (r=.6135 p<.000) and BA17 (r=.463 p<.004). Furthermore, neural responses in LOC correlated only for visual data, but not for haptic data. These results show clear involvement of early and higher-order visual areas in both visual and haptic tasks, indicating a complex network of brain areas involved in multisensory processing.
Meeting abstract presented at VSS 2014