Abstract
Recent work has shown that the identity and category of a stimulus can be “read out” from the spiking activity of populations of neurons in monkey inferior temporal cortex (Hung, Kreiman, et al., 2005). Here we extend these decoding techniques to create a neural similarity score for visual objects. In particular, we train a linear support vector machine to discriminate between pairs of objects based on feature vectors that consist of the firing rates of 256 monkey ITC cells, and we define the similarity score to be the average classification accuracy over 100 bootstrap samples of the neurons. We compare these neural similarity scores to human judgments of similarity from a simple delayed match to sample discrimination task. Results show a high level of agreement between human similarity judgments and neural readout similarity, even though the neural readout results are only using a small fraction of the cells in ITC.