Abstract
Many visual tasks, including object recognition and target search, require an “untangling” computation, in which inaccessible task-relevant information contained in an initial population representation is transformed into a more accessible linearly separable format. How are untangling computations implemented in the brain? Here we propose that a linear-nonlinear-linear (LNL) model similar to the Energy model for V1 complex cells can account for the untangling of target match information as it flows from neural populations in IT to those of Perirhinal cortex, recorded as monkeys performed a delayed-match-to-sample task. Our model rests on two intuitive principles. First, that a population representation is linearly separable if different classes of conditions evoke significant differences in mean response. Second, that a population in which different classes evoke responses with large variance differences may be transformed into a linearly separable format by applying a squaring nonlinearity. Consequently, our model consists of a first linear transformation that maximizes variance differences between the responses to different classes, followed by a squaring nonlinearity and a final orthogonal linear transformation. When applied to a population of IT responses, our model replicated multiple aspects of recorded Perirhinal responses, most notably the linear separability of classification information, as measured by linear decoder performance. Finally we demonstrate that, under an assumption of Gaussian response variability, our model is optimal within the class of quadratic classifiers. These results provide evidence that within the family of LNL models, a generalization of the Energy model is sufficient to explain the untangling of visual target match information.