October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
Tuned representational similarity analysis: Improving the fit between computational models of vision and brain data
Author Affiliations & Notes
  • Philipp Kaniuth
    Max Planck Institute for Human Cognitive and Brain Sciences
  • Martin N Hebart
    Max Planck Institute for Human Cognitive and Brain Sciences
  • Footnotes
    Acknowledgements  This work was funded by a Max Planck Research Group grant to MNH.
Journal of Vision October 2020, Vol.20, 1076. doi:https://doi.org/10.1167/jov.20.11.1076
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Philipp Kaniuth, Martin N Hebart; Tuned representational similarity analysis: Improving the fit between computational models of vision and brain data. Journal of Vision 2020;20(11):1076. https://doi.org/10.1167/jov.20.11.1076.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

In cognitive neuroscience, representational similarity analysis (RSA) is a common method to assess the correspondence between representations in computational models, brain and behavior. RSA is based on comparing representational dissimilarity matrices (RDMs), which are composed of pairwise dissimilarities between responses to all pairs of stimuli or conditions (e.g. voxel activity, model features). However, this approach of computing dissimilarities treats each feature of a computational model as equally important. As a consequence, classical RSA may underestimate the information represented in these models and may lead to biases in model selection. To address these issues, we propose “tuned RSA”, a refinement of RSA based on additive clustering (Shepard & Arabie, 1979; Peterson et al., 2018). Instead of directly relating two RDMs, this approach linearly reweights each model feature to maximize the correspondence between both RDMs. Therefore, dissimilarity of activity patterns in the brain is predicted by a linear combination of the dissimilarity in each model feature. We validated tuned RSA on object-selective cortex activity in two publicly available fMRI datasets of real-world object images (92 and 118 conditions, respectively) and tested the correspondence with deep neural networks, semantic embeddings, and MEG data for MEG-fMRI fusion. Across all models and both datasets, tuned RSA led to strong increases in the amount of variance shared between both RDMs, never reducing and in some cases even doubling the explained variance. Sanity checks in control regions of interest and on noise data confirmed that this was not due to overfitting. RSA promises important insights into which models capture best how the brain represents relations between stimuli or conditions. Tuned RSA thus has the potential to become a general purpose method for measuring the information content shared between representations in computational models, brain, and behavior, and may improve our ability as scientists to adjudicate between competing models.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.