Purchase this article with an account.
Philipp Kaniuth, Martin N Hebart; Tuned representational similarity analysis: Improving the fit between computational models of vision and brain data. Journal of Vision 2020;20(11):1076. doi: https://doi.org/10.1167/jov.20.11.1076.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
In cognitive neuroscience, representational similarity analysis (RSA) is a common method to assess the correspondence between representations in computational models, brain and behavior. RSA is based on comparing representational dissimilarity matrices (RDMs), which are composed of pairwise dissimilarities between responses to all pairs of stimuli or conditions (e.g. voxel activity, model features). However, this approach of computing dissimilarities treats each feature of a computational model as equally important. As a consequence, classical RSA may underestimate the information represented in these models and may lead to biases in model selection.
To address these issues, we propose “tuned RSA”, a refinement of RSA based on additive clustering (Shepard & Arabie, 1979; Peterson et al., 2018). Instead of directly relating two RDMs, this approach linearly reweights each model feature to maximize the correspondence between both RDMs. Therefore, dissimilarity of activity patterns in the brain is predicted by a linear combination of the dissimilarity in each model feature.
We validated tuned RSA on object-selective cortex activity in two publicly available fMRI datasets of real-world object images (92 and 118 conditions, respectively) and tested the correspondence with deep neural networks, semantic embeddings, and MEG data for MEG-fMRI fusion. Across all models and both datasets, tuned RSA led to strong increases in the amount of variance shared between both RDMs, never reducing and in some cases even doubling the explained variance. Sanity checks in control regions of interest and on noise data confirmed that this was not due to overfitting.
RSA promises important insights into which models capture best how the brain represents relations between stimuli or conditions. Tuned RSA thus has the potential to become a general purpose method for measuring the information content shared between representations in computational models, brain, and behavior, and may improve our ability as scientists to adjudicate between competing models.
This PDF is available to Subscribers Only