Abstract
Traditionally, in biological and artificial neural networks, a neuron's response is represented as a two-dimensional feature map (or receptive field). This receptive field typically represents the stimulus for which the neuron is most responsive. For sparse codes applied to natural scenes, the receptive fields often resemble those of V1 simple cells. Highly overcomplete sparse codes (e.g., 10 times overcomplete) often reveal a wider variety of receptive fields. This variety is usually the most noted aspect of these overcomplete codes. However, the response properties of the neurons in these codes are often quite non-linear. Here, we provide a method of visualizing these non-linearities in terms of the curvature in the iso-response surfaces. We measured the iso-response surfaces of artificial neurons in a sparse coding network in response to natural scenes. We find that as the code becomes more overcomplete, the magnitude of the non-linearity increases (the iso-response surfaces become more curved). This curvature in iso-response contours also describes the nonlinearities observed in V1 (such as end-stopping, cross-orientation inhibition, contrast gain control, etc.). Although it has been argued the precise form of the cost function does not have a large effect on the family of receptive fields produced, we show that this does have a significant effect on the non-linear responses (the precise form of curvature). We discuss these results in terms of the magnitude of non-linearities in V1 neurons and the theorized degree of overcompleteness in the primate visual system. We also show that both selectivity and invariance can be described in terms of this curvature.
Meeting abstract presented at VSS 2016