September 2011
Volume 11, Issue 11
Free
Vision Sciences Society Annual Meeting Abstract  |   September 2011
Analysis of similarity matrices and its application to the study of semantic and visual information processing in the inferior temporal cortex
Author Affiliations
  • Imri Sofer
    The Interdisciplinary Center for Neural Computation, and The School of Computer Science and Engineering, The Hebrew University of Jerusalem
  • Daphna Weinshall
    The Interdisciplinary Center for Neural Computation, and The School of Computer Science and Engineering, The Hebrew University of Jerusalem
  • Thomas Serre
    Department of Cognitive and Linguistic Sciences, Brown University
Journal of Vision September 2011, Vol.11, 872. doi:10.1167/11.11.872
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Imri Sofer, Daphna Weinshall, Thomas Serre; Analysis of similarity matrices and its application to the study of semantic and visual information processing in the inferior temporal cortex. Journal of Vision 2011;11(11):872. doi: 10.1167/11.11.872.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Representational similarity analysis has become a popular tool to study the nature of visual object representations (Kriegeskorte et al, 2008). Such analysis aims at characterizing patterns of brain activity via a so-called similarity matrix, which contains a similarity measure between all pairs of activity patterns elicited by a given stimulus set. This approach is particularly suited for comparing patterns of neural activity measured with different experimental techniques. Here we describe a computational approach based on distance-learning techniques and a statistical framework to uncover which stimulus dimensions are emphasized in the underlying representation. Our test is based on a restricted permutation test (Good, 2000) that we generalized to create a new “two-layer” test. The new test is specifically designed for hierarchically structured data such as natural object categories, to assess what level of a hierarchy a similarity matrix shares with the visual representation.

As a validation of the approach, we applied our method to the representational similarity analysis from Kriegeskorte et al. (2008) based on patterns of fMRI activity response in human and monkey inferior temporal (IT) cortex to natural images of objects. We show that an image representation based on relatively low-level color and shape features is able to account for the underlying similarity matrix. These results cast doubts on the original interpretation of the data suggesting a high-level semantic visual representation at the level of IT. Using the proposed method we are able to relate different aspects of the similarity matrix to different types of low level features. Our findings suggest that careful consideration should be taken when conducting experiments with natural object categories.

This study was supported by the European Union under DIRAC integrated project IST-027787, and by a DARPA grant to TS (DARPA-BAA-09-31). 
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×