Abstract
We present an account for how the structure of the representation of living things emerges in the object vision pathway, investigating three regions: medial occipital (MO), inferior occipital (IO), and ventral temporal cortex (VT). We investigated the similarity structure for patterns defined by responses to a variety of animate categories using functional magnetic resonance imaging (fMRI). Participants (N=12) viewed photographs of six animal species–two species each of insects, birds, and primates. Pair-wise dissimilarities between condition patterns were used to construct similarity spaces for each region within each subject. The similarity structures revealed how categorical representations emerge along the visual pathway. Patterns in early visual cortex (MO), as compared to those in IO and VT, are less differentiated and do not have a clear category structure. IO reveals differentiation between vertebrates and insects, while in VT each category becomes clearly defined. Individual differences multi-dimensional scaling (INDSCAL) showed how similarity structures transform from one region to the next. Thirty-six similarity structures from three brain regions–MO, IO, and VT–in each of 12 subjects were used to find a common multidimensional scaling solution where weights on dimensions varied between similarity structures. Differences in dimension weights reveal a reliable translation from MO through VT in similarity spaces organized according to low-level visual features in MO to semantic categories in VT. Similarity structures were highly stable and replicable both within and between subjects–especially in VT with an average between-subject correlation of r=.91. The consistency of similarity structures in IO and MO was also high, albeit not as strong as in VT (IO, r=.75; MO, r=.65). Similarity-based pattern analysis reveals a categorical structure in VT that mirrors our knowledge about animal species, providing a window into the structure of neural representations that form the basis of our categorical knowledge of the living world.