Relating neural object representations to perceptual judgments with representational similarity analysis

Abstract

Human inferior temporal cortex (hIT) has been shown to be involved in the representation of visual objects. Recent studies have begun to investigate the relationship between perceived object similarity and similarity of response patterns in hIT. These studies often used a small set of (novel) stimuli from a few a priori defined categories. Here, we use a stimulus set consisting of 96 object images from a wide range of object categories including faces, body parts, animals, places, and artificial objects. We compare the neural and perceptual similarity structure of these 96 object images using representational similarity analysis.

We performed BOLD fMRI measurements at high resolution (voxel size 1.95x1.95x2 mm³). Activity in response to 96 different object photos was measured in four subjects. hIT was defined at a range of sizes by selecting the most visually responsive voxels, based on independent data. The neural similarity structure of hIT response patterns was constructed by computing the dissimilarity (1-correlation distance) between each pair of object activity patterns. Subjects were asked to arrange the 96 object images in 2D to report perceptual similarity.

The neural and perceptual similarity structures were significantly correlated (r =0.46, p[[lt]]0.0001). This indicates that objects that are perceived as similar tend to elicit similar response patterns in hIT. In addition, both structures showed a categorical organization, with the main clusters being animate and inanimate objects. These findings suggest that, for a wide range of real-world objects, similarity of neural object representations in hIT reflects perceived object similarity.

Footnotes

 This research was supported by the Intramural Research Program of the NIH, NIMH.