Abstract
Representational similarity analysis (RSA) is increasingly part of the standard analytic toolkit in neuroimaging studies investigating the organization of visual cortical regions. Core to RSA is the measuring of neural dissimilarity between the response patterns for different conditions to construct neural representational dissimilarity matrices (RDMs) for comparison with those constructed from behavioral data or computational models of visual processing. It has been proposed that noise normalizing these patterns, and using cross-validated distances as a dissimilarity measure, is superior for characterizing the structure of neural RDMs for visual and not-visual brain regions (Walther et al. 2014). This assessment has been motivated by results suggesting improvement in within-subject neural dissimilarity after noise normalization. However, between-subject reliability is more directly related to determining the amount of explainable variance, and the evaluation of observed effect sizes when they are correlated with behavioral or model RDMs.
To further evaluate the impact of noise normalization we re-analyzed three data sets that included activity patterns from multiple ventral visual pathway regions, and also non-visual regions. Across the three datasets, using multiple measures of dissimilarity (correlation distance, classifier accuracy, cross-validated Euclidean distance, and cross-validated Mahalanobis distance) we did not find that noise normalization consistently boosts within-subject reliability, between-subject reliability, or correlations with behavioral or model RDMs. In fact, in some cases, it made things worse (Charest, Kriegeskorte, and Kay, 2018). Overall, our results provide equivocal support for the utility of noise normalization to RSA, the impact of which may depend heavily on the stimulus, visual region of interest, and dissimilarity measure used.