Abstract
The computations carried out in the primate ventral visual pathway allow for accurate categorization of objects, even for images that have undergone transforms and in spite of variance of features within an object category. In order to achieve this type of representation, neurons in the ventral stream become simultaneously more selective (or sparse) to image features and more invariant to certain transforms of those features. Here we argue that the geometry of neural responses allows for a unified view of selectivity and invariance, and provides a method for quantifying these opposing aspects of neural representation. To probe selectivity, we investigated the iso-response contours in high-dimensional image space of a population of neurons in a sparse coding network. For linear neurons, the isocontours are necessarily straight and orthogonal to the receptive field vector. With an overcomplete code, the lack of orthogonality between encoding vectors can result in significant redundancy. Curvature of the isocontours toward the encoding vector can remove much of this redundancy. We demonstrate that this negative sectional (hyperbolic) curvature also produces a family of nonlinearities found in V1 neurons. We measured curvature of the isocontours of second-layer neurons in a two-layer variance components network, and found both negative as well as positive sectional (spherical) curvature for different subspaces of the responses of individual neurons. We argue that this geometric approach could lead to a unified understanding of selectivity and invariance in the representations of both neural network models and visual cortical neurons.