Abstract
A key challenge in human neuroscience is to gain information about neural population codes using indirect measures of neural activity. Recently, a multivariate pattern analysis method termed Leave-One-Person-Out (LOPO) has been used to support inferences regarding neural coding. Two key caveats, however, have been overlooked by users of this method: (1) its sensitivity to low-level feature imbalances across conditions (e.g. image contrast, luminance, and their variability over instances of each stimulus class), and (2) implausibility of the methods’ assumption that spatial normalization meaningfully aligns neural patterns of activation across subjects. Here, we show with simulations instantiating multilayered randomly connected feedforward networks that LOPO leads to erroneous conclusions when the methods’ assumptions are not met. In particular, low-level properties of the images chosen by an experimenter are shown to largely determine the observed pattern of results, and not the covariance structure of the underlying spatial patterns of activation. We further show how deceptively complex representational structures emerge due to specifics of the LOPO analysis scheme. Interestingly, simulations initially intended to study the general behavior of LOPO led to concrete predictions regarding the relative norms, means and variances of images from an image database recently used in multiple recent neuroimaging studies. Low-level feature imbalances of the precise form predicted by our model were confirmed by ensuig direct analyses of these images. We argue that under plausible assumptions regarding the fMRI measurement process, the observed confounds may account for recent putative evidence of mirror-symmetric tuning of neural populations in a human face-selective area. In sum, we conclude that (1) LOPO is particularly sensitive to low-level image feature imbalances across conditions, and (2) the method’s sound interpretation hinges on implausible assumptions. Our observations cast doubts on the validity of LOPO analyses to support inferences that require the detection of spatially structured brain patterns.