**In this article, we explore two forms of selectivity in sensory neurons. The first we call classic selectivity, referring to the stimulus that optimally stimulates a neuron. If a neuron is linear, then this stimulus can be determined by measuring the response to an orthonormal basis set (the receptive field). The second type of selectivity we call hyperselectivity; it is either implicitly or explicitly a component of several models including sparse coding, gain control, and some linear-nonlinear models. Hyperselectivity is unrelated to the stimulus that maximizes the response. Rather, it refers to the drop-off in response around that optimal stimulus. We contrast various models that produce hyperselectivity by comparing the way each model curves the iso-response surfaces of each neuron. We demonstrate that traditional sparse coding produces such curvature and increases with increasing degrees of overcompleteness. We demonstrate that this curvature produces a systematic misestimation of the optimal stimulus when the neuron's receptive field is measured with spots or gratings. We also show that this curvature allows for two apparently paradoxical results. First, it allows a neuron to be very narrowly tuned (hyperselective) to a broadband stimulus. Second, it allows neurons to break the Gabor–Heisenberg limit in their localization in space and frequency. Finally, we argue that although gain-control models, some linear-nonlinear models, and sparse coding have much in common, we believe that this approach to hyperselectivity provides a deeper understanding of why these nonlinearities are present in the early visual system.**

*R*(

*S*) will simply be the dot product of the receptive field

*S*:

*Classic selectivity*describes the stimulus that produces the maximum response for a given stimulus magnitude (

*S*

_{max}). If a neuron is linear, then classic selectivity can be determined by measuring the response to a basis set (e.g., spots or gratings). This receptive field describes how the neuron responds as a function of position (and time, color, etc.). Our second form of selectivity,

*hyperselectivity*(Golden et al., 2016), is a measure of how narrowly tuned the neuron is to that optimal stimulus.

*θ*

_{sp}(see Equation 2) is not equivalent to the orientation difference between the neurons. Figure 1 shows examples of the angle for four pairs of neurons. The state-space angle represents the overlap between two neurons and is a function of the dot product of the receptive fields

*X*× Δ

*F*). Gabor's work was derived from the Heisenberg uncertainty principle (Heisenberg, 1927) and is often considered to be a fundamental limit to localization.

*S*

_{max}). In Figure 3a we show a simple two-dimensional example where the iso-response surface is represented by a contour, and different response magnitudes are represented by the equally spaced contours. If the neuron is linear past some threshold, then the response contours will be like that shown in Figure 3b.

*S*

_{max}as the stimulus that optimally stimulates the neuron—the stimulus that produces the highest response for all possible stimuli

*s*of the same total energy

*c*.

*ψ*is any orthonormal basis set and

*R*(

*ψ*) is the linear response to stimulus

_{i}*ψ*. That is, for a linear neuron, the receptive field will be the same whether one measures the responses with gratings or pixels.

_{i}*R*(

*S*

_{max}+

*S*

_{2}) is not equal to the response

*R*(

*S*

_{2}).

*S*

_{max}. For any given dimension orthogonal to

*S*

_{max}, the neuron shows hyperselectivity if the response to

*S*

_{max}plus an orthogonal stimulus is less than the response to

*S*

_{max}:

*I*is the input image patches,

*ϕ*is basis matrix with feedforward weights of all the neurons in the network,

*λ*is tradeoff parameter between the sparsity of the network and the reconstruction error,

*a*is the response activity of

_{i}*i*th neuron, and

*S*(

*a*) is the cost function. Sparse coding attempts to both minimize the reconstruction error (preserve information) and minimize the cost of response activities

_{i}*a*. With sparse coding, we have found that the curvature is a function of the angle between neighboring neurons. When the angles between neighbors are 90° (orthogonal), there is little or no curvature. As the angle between neighboring neurons decreases, the curvature increases accordingly. We have found (Golden et al., 2016) that the equation of a folding fan is a good approximation to the curvature produced by sparse coding when applied in two-dimensional space—that is, if orthogonal lines are drawn on an open fan and the fan is closed. The equation of a folding fan is

_{i}*c*is the distance of a stimulus from the origin (i.e., the stimulus contrast), is the angle between a stimulus and the neuron,

*a*is the response magnitude of a neuron

_{i}*i*,

*n*determines the curvature and

*f*and

_{i}*f*are the vectors representing the two neurons. n is a function of the angle between neighboring neurons. When

_{j}*n*= 1 the iso-response contours are flat (e.g., linear); when

*n*> 1 the neuron has iso-response contours with exo-origin curvature and when

*n*< 1 the neuron has iso-response contours with endo-origin curvature.

- For both sparse coding and the fan equation, the curvature is mostly near the vector representing the optimal stimulus
*S*_{max}of a neuron. Away from the vector, the iso-response surfaces flatten out and become parallel to the vectors representing the neighboring neurons. - The curvature is determined by the angle between neighboring neurons (Golden et al., 2016). If two neighboring neurons are orthogonal, then there is no curvature in the subspace between them. A smaller angle produces higher curvature, as shown in Figures 4c and 4f. Using a high-dimensional overcomplete sparse coding network (e.g., 256 pixels), we found in our previous work that when the network is applied to natural scenes, the curvature does predict the angle between neurons, although the resulting curvature was less than predicted by the fan equation (Equation 8). We are currently investigating whether the reduced curvature is due to a limitation in the sparse coding network or is an efficient component of the network.
- For any given stimulus, the ratio of the responses
Display Formula \(({\textstyle{{a1} \over {a2}}})\) of the different neurons to a particular stimulus is independent of the contrast of the stimulus. That is, despite the nonlinearities, the relative activities of the neurons responding to a stimulus do not change as the contrast changes (from*c*1 to*c*2):

*r*

_{1}and

*r*

_{2}are the squared linear responses of two neurons.

*r*

_{1}and

*r*

_{2}are linear responses of two neurons and

*a*,

*b*,

*c*,

*d*,

*e*, and

*f*are free parameters of the quadratic model. This will generate a wide family of curves that are described by hyperbolas and ellipses in lower dimensional subspaces. A more restricted family of quadratic curves is represented by

*r*

_{1},

*r*

_{2}, etc.). This is an interesting family of curves due to their algebraic simplicity. Figure 4 shows an example of the hyperselective curve that is produced by

*S*

_{max}) a neuron (what a neuron is selective to) and how selective the neuron is to that stimulus. We argue that without this distinction we end up with an apparent paradox: that a neuron can be very narrowly tuned to a broadband stimulus. Second, we show that with this curvature, the optimal stimulus will not be the same as the receptive field. That is, the receptive field depends on the basis set used to measure it. Finally, we show that if we compare the localization of the neuron in the spatial-frequency domain and the space domain, neurons with hyperselectivity can violate the Gabor localization limit.

*ax*

^{2}+

*bx*+

*c*), and the magnitude of the curvature was the magnitude of the coefficient

*a*. Figure 5e shows the average curvature of the five closest bases to each base (i.e., the mean curvature of the most curved regions of each neuron's response surface). We also plot the average angle of these five closest bases (angle plotted on the right axis). These results show that with an increase in the degree of overcompleteness, the angle between neighbors decreases and the curvature (hyperselectivity) increases.

*S*

_{max}—would be a stimulus that matched the receptive field).

*S*

_{max}) are reduced (i.e., they require more stimulus contrast). The curvature is, in a sense, a model of a variable threshold, with the lowest thresholds near the optimal stimulus and increased thresholds away from that optimal stimulus. Therefore, if the neuron is probed with a stimulus that is near the optimal response for the neuron, then the neuron will show a lower stimulus threshold.

*π*

^{2}(referred to as the Gabor limit).

*π*

^{2}= 0.025).

*S*

_{max}) for a neuron. The analysis we describe here probes the hyperselectivity in a simple unsupervised network (sparse coding). We are confident that this geometric approach to hyperselectivity will prove useful for analyzing multilayer hierarchical (deep) networks as well. Currently (Golden, Vilankar, & Field, 2017), we are investigating networks which learn both invariance and selectivity (e.g., Karklin & Lewicki, 2005) and are planning to explore the units of supervised deep networks. Visualization of the hidden layers of a deep network is a major topic of research, and it is well known that a single receptive field is insufficient to describe any given unit's response. It is not yet clear what sort of curvature is produced by the nonlinearities typical of these networks (e.g., max pooling). However, the selectivity and tolerance shown by these networks imply to us that hyperselective curvature (exo-origin) and invariant curvature (endo-origin) play an important contribution to their success.

*x*

^{2}+

*y*

^{2}) and exo-origin (

*x*

^{2}− y

^{2}) curvatures. It can also produce a combination (e.g.,

*x*

^{2}+

*y*

^{2}−

*z*

^{2}). We feel that this approach as currently conceived has a number of limitations (e.g., the forms of curvature and the tiling that are produced). However, modifications of this approach may well provide a means of developing efficient hierarchical networks.

*S*

_{max}). The network also allows hyperselectivity (exo-origin curvature) as well as tolerance (endo-origin curvature). Since the goal of this network is to classify, it is not clear what sets of curvatures will be optimized by the network. It is also not clear what forms of curvature would be optimal (e.g., gain control versus the fan equation). At some point, we hope that physiology will produce a clear insight into the curvature produced in the mammalian visual system. As we mentioned earlier, the methods with the most promise are those that use the spike triggered covariance technique (e.g., Rust et al., 2004, 2005; Schwartz et al., 2002; Schwartz et al., 2006; Vintch et al., 2015). These techniques can reveal the relevant subspace where the nonlinear interactions between neighboring neurons are occurring. By recording from multiple neurons, it may be possible to determine if the nonlinearities can be predicted by the angles between neighboring neurons. The difficulty with this approach is that an accurate description of the subspace requires a very large number of stimuli. Vintch et al. (2015) have provided some of the most interesting recent results. However, we think that to get an accurate account of curvature, the inhibitory subspaces must be probed with a denser array of stimuli. Ideally, once the interesting subspace has been determined with spike-triggered covariance, a new set of stimuli can be created that focuses on that subspace.

*The Journal of Physiology*, 119 (1), 69–88.

*Journal of the Optical Society of America A*, 15 (7), 1723–1732.

*Nature Reviews Neuroscience*, 13 (1), 51–62.

*Proceedings of the National Academy of Sciences, USA*, 104 (48), 19120–19125.

*Journal of the Optical Society of America A*, 2 (7), 1160–1169.

*Computational Neuroscience*, 2, 403–423.

*Network: Computation in Neural Systems*, 16 (2–3), 239–260.

*IEEE Transactions on Biomedical Engineering*,

*BME-15*(3), 169–179.

*Frontiers in visual science*(pp. 544–556). Berlin, Heidelberg: Springer.

*Vision Research*, 22 (5), 545–559.

*Journal of the Optical Society of America A*, 2 (7), 1115–1123.

*Neural Computation*, 6 (4), 559–601.

*Proceedings of the Royal Society of London*.

*Series B: Biological Sciences*, 228 (1253), 379–400.

*Journal of Vision*, 4 (8): 283, doi:10.1167/4.8.283. [Abstract]

*Journal of the Institution of Electrical Engineers—Part III: Radio and Communication Engineering*, 93 (26), 429–441.

*Vision Research*, 32 (8), 1409–1410.

*Curvature of neural response surfaces in a multi-layer network*. Manuscript in preparation.

*Vision Research*, 120, 74–92.

*The Journal of General Physiology*, 39 (5), 651–673.

*Visual Neuroscience*, 9, 181–197.

*Zeitschrift für Physik*, 43 (3–4), 172–198.

*The Journal of Physiology*, 160, 106–154.

*Journal of Neurophysiology*, 58 (6), 1233–1258.

*Neural Computation*, 17 (2), 397–423.

*Advances in neural information processing systems*(pp. 1097–1105). Red Hook, NY: Curran Associates.

*Journal of Neurophysiology*, 16, 37–68.

*Anatomy and Embryology*, 190 (4), 351–366.

*Nature Neuroscience*, 8 (12), 1690–1697.

*Journal of the Optical Society of America*, 70 (11), 1297–1300.

*Computational and cognitive neuroscience of vision*(pp. 59–84). Singapore: Springer.

*Computational and cognitive neuroscience of vision*(pp. 59–84). Singapore: Springer.

*Journal of Vision*, 11 (5): 2, 1–25, doi:10.1167/11.5.2. [PubMed] [Article]

*Vision Research*, 37 (11), 1535–1543.

*Proceedings of SPIE*, 8651, 86510S.

*Nature*, 381 (6583), 607–609.

*Vision Research*, 37, 3311–3325.

*Neural Computation*, 17 (8), 1665–1699.

*Physical Review Letters*, 110 (4), 044301.

*Neural Computation*, 28, 2291–2319.

*Vision science: From photons to phenomenology*. Cambridge, MA: MIT Press.

*Neural Networks*, 17 (5), 663–679.

*Neuron*, 70 (6), 1178–1191.

*Cognitive Science*, 28 (2), 147–166.

*The Journal of Neuroscience*, 30 (39), 12978–12995.

*Neurocomputing*, 58, 793–799.

*Neuron*, 46 (6), 945–956.

*Journal of the Optical Society of America*, 61, 1176–1186.

*Advances in neural information processing systems*(pp. 269–276). Cambridge, MA: MIT Press.

*Journal of Vision*, 6 (4): 13, 484–507, doi:10.1167/6.4.13. [PubMed] [Article]

*Nature Neuroscience*, 4, 819–825.

*Annual Review of Neuroscience*, 36, 103–120.

*Visual Neuroscience*, 3 (5), 445–454.

*Visual Neuroscience*, 14 (2), 293–309.

*The Journal of Neuroscience*, 35 (44), 14829–14841.

*Advances in neural information processing systems*(pp. 3104–3112). Cambridge, MA: MIT Press.

*Proceedings of the National Academy of Sciences, USA*, 111 (23), 8619–8624.

*Vision Research*, 30 (7), 1111–1117.

*Journal of the Optical Society of America A*, 16 (7), 1554–1565.

*Network: Computation in Neural Systems*, 16 (2–3), 191–221.

*Network: Computation in Neural Systems*, 12 (3), 331–350.

*PLoS Computational Biology*, 9 (8), e1003191.