Neural bandwidth of veridical perception across the visual field

Abstract

Neural undersampling of the retinal image limits the range of spatial frequencies that can be represented veridically by the array of retinal ganglion cells conveying visual information from eye to brain. Our goal was to demarcate the neural bandwidth and local anisotropy of veridical perception, unencumbered by optical imperfections of the eye, and to test competing hypotheses that might account for the results. Using monochromatic interference fringes to stimulate the retina with high-contrast sinusoidal gratings, we measured sampling-limited visual resolution along eight meridians from 0° to 50° of eccentricity. The resulting isoacuity contour maps revealed all of the expected features of the human array of retinal ganglion cells. Contours in the radial fringe maps are elongated horizontally, revealing the functional equivalent of the anatomical visual streak, and are extended into nasal retina and superior retina, indicating higher resolution along those meridians. Contours are larger in diameter for radial gratings compared to tangential or oblique gratings, indicating local anisotropy with highest bandwidth for radially oriented gratings. Comparison of these results to anatomical predictions indicates acuity is proportional to the sampling density of retinal ganglion cells everywhere in the retina. These results support the long-standing hypothesis that “pixel density” of the discrete neural image carried by the human optic nerve limits the spatial bandwidth of veridical perception at all retinal locations.

Introduction

Neural sampling of the continuous retinal image imposes fundamental constraints on form and motion perception. Aliasing, defined as the misperception of scenes caused by insufficient density of sampling elements, is more likely for peripheral than for central vision because the density of retinal neurons declines with eccentricity (Curcio & Allen, 1990; Curcio, Sloan, Kalina, & Hendrickson, 1990). According to the sampling theory of visual resolution, when other limiting factors are avoided (e.g., optical or neural filtering, stimulus or biological noise), resolution acuity for sinusoidal gratings is set by the spatial density of neural sampling elements (Bergmann, 1857; Geisler & Hamilton, 1986; Helmholtz, 1911; Hughes, 1981; Merchant, 1965; Ten Doesschate, 1946; Thibos, 1998; Thibos, Cheney, & Walsh, 1987; Weber, 1846; Weymouth, 1958; Williams & Coletta, 1987; Yellott, 1990). In this sampling-limited domain, resolution acuity is equal to the highest spatial frequency that can be represented veridically by the retinal mosaic of neurons, the so-called Nyquist frequency. Theory predicts that, unless removed by filtering, retinal image components with spatial frequencies higher than the Nyquist limit will still be signaled by the array, and therefore accessible to perception, but will be misrepresented as aliases of the physical stimulus. Thus, to understand the limitations imposed by neural undersampling for all subsequent stages of vision, including perception, it is important to delineate the range of spatial frequencies that will be represented veridically by the neural image transmitted to the brain by the optic nerve. 

Image sampling by cone photoreceptors is frequently cited as the neural limit to resolution acuity for central vision (Green, 1970; Williams, 1985a), whereas ganglion cells have been cited as the limiting array in peripheral vision (Anderson, 1996; Anderson, Drasdo, & Thompson, 1995; Anderson, Evans, & Thibos, 1996; Anderson & Hess, 1990; Anderson, Mullen, & Hess, 1991; Anderson & Thibos, 1999b; Anderson, Wilkinson, & Thibos, 1992; Artal, Derrington, & Colombo, 1995; Beirne, Zlatkova, & Anderson, 2005; Cheney, Thibos, & Bradley, 2015; Coletta & Williams, 1987; Rossi & Roorda, 2010; Smith & Cass, 1987; Thibos, Cheney, et al., 1987; Thibos, Still, & Bradley, 1996; Thibos, Walsh, & Cheney, 1987; Wang, Bradley, & Thibos, 1997a, 1997b). Although the eye's optical system normally serves as an effective antialias filter in the foveal region of the retina, thereby preventing the attainment of sampling-limited performance for central vision, aliasing has been reported when this optical limitation has been circumvented by stimulating the normal retina with interference fringes (Coletta & Williams, 1987; He & MacLeod, 1996; Williams, 1985a, 1985b; Williams & Coletta, 1987; Williams & Collier, 1983). In peripheral vision, however, aliasing can occur for natural stimuli (Smith & Cass, 1987; Thibos et al., 1996) because the relatively high optical bandwidth of retinal images typically exceeds the Nyquist frequencies of retinal ganglion cells (Williams, Artal, Navarro, McMahon, & Brainard, 1996) even when moderately defocused (Millodot, Johnson, Lamont, & Leibowitz, 1975; Wang, Thibos, & Bradley, 1997). Thus resolution acuity, defined as the transition spatial frequency that separates the domain of veridical perception (supported by well-sampled retinal images) from the domain of nonveridical perception (supported by undersampled retinal images), has become a noninvasive tool for measuring the functional density of retinal neurons in normal eyes as well as pathological retinas (Anderson & O'Brien, 1997; Chui, Thibos, Bradley, & Burns, 2009; Chui, Yap, Chan, & Thibos, 2005). 

Although the theoretical possibility of visual resolution being sampling-limited was postulated in the mid-19th century (Helmholtz, 1911; Weber, 1846), Wertheim's (1891) classic study of grating acuity across the entire visual field reported no evidence of the telltale signs of aliasing that herald the end of veridical perception. Wertheim's study was the first, and apparently the only, published investigation of grating acuity throughout the two-dimensional visual field. His method was to slowly bring the target, consisting of five cycles of a square-wave grid of variable orientation, closer to the eye until the direction of the grid became recognizable. Careful correction of refractive error for such a target is not necessary for observing aliasing (Wang, Thibos, et al., 1997), yet Wertheim made no comment about the visibility or false appearance of the stimulus (which Bergmann [1857] and Helmholtz [1911] had described previously) for viewing distances beyond the resolution limit (D'Zmura, 1996; Thibos, Walsh, et al., 1987; Wertheim, 1980). This is a puzzling omission since Wertheim was a highly experienced observer who served with considerable insight as his only trusted subject. It is also a crucially important point for establishing the mechanism that limits resolution because perceptual aliasing is the proof that resolution is sampling-limited. Nevertheless, it seems likely that neural undersampling was the mechanism that limited resolution in Wertheim's experiment (and the prior work he cites) since his isoacuity contours were horizontal ovals elongated by increasing amounts into the temporal visual field and, to a lesser extent, into the inferior field, with increasing eccentricity, which agrees with descriptions of isodensity contours of retinal neurons from classic studies of retinal anatomy in the 20th century (Curcio & Allen, 1990; Curcio et al., 1990). 

Given the paucity of two-dimensional maps of grating acuity in the literature, and the uncertainty of whether classic measurements of resolution acuity were sampling-limited, we undertook new experiments to determine the topography of visual resolution using methodology that ensures sampling-limited performance. Double lines, double dots, geometrical figures, and letters are traditional stimuli for measuring acuity across the visual field (Aubert & Förster, 1857; Genter, Kandel, & Bedell, 1981; Weymouth, 1958) but resolution of these stimuli is not necessarily a sampling-limited task. Letters are also the standard targets for clinical measurements, but theoretical predictions of letter acuity based on sampling theory depend on many factors that complicate the interpretation of empirical results and their comparison with Nyquist frequencies of neural sampling arrays (Anderson & Thibos, 1999a, 1999b; Anderson & Thibos, 2004; Demirel, Anderson, Dakin, & Thibos, 2012). Instead, we used sinusoidal gratings because they provide the simplest, most direct link to the sampling theory of visual resolution for the purpose of demarcating the neural bandwidth and local anisotropy of veridical perception, unencumbered by optical imperfections of the eye. Our results point to the retinal ganglion cell array as the anatomical substrate limiting veridical perception everywhere in the visual field. 

Methods

A circular patch of sinusoidal grating was created on the observer's retina as interference fringes produced by a commercial instrument (Haag Streit, Berne, Switzerland) called the Lotmar Visometer (Haag-Streit Diagnostics, Bern, Switzerland; Bradley, Thibos, & Still, 1990; Lotmar, 1972, 1980). This instrument is an achromatic moiré interferometer that uses a tungsten light source to generate high contrast, white interference-fringes of any orientation seen in Maxwellian view. To avoid orientation-dependent attenuation of fringe contrast by ocular chromatic aberration (Cheney et al., 2015; Thibos, 1990), a 505 nm interference filter was inserted in the light path to produce quasi-monochromatic fringes without the bothersome speckle characteristic of lasers. Retinal illuminance was 540 photopic Trolands and stimulus diameter was either 1.5° (eccentricity ≤10°), 2.5° (eccentricity = 20°) or 3.5° (eccentricity ≥30°) as shown in Figure 1A. These sizes were selected as a compromise to ensure the patch was large enough to contain at least six cycles of the interference fringes at the acuity limit (Anderson et al., 1996) and yet small enough to keep resolution approximately uniform over the retinal patch being stimulated. These stimulus requirements limited our experiment to eccentricities in the range 0°–50°. 

Figure 1

 

Methodological details. (A) Retinal test locations (blue rings, 10° steps in eccentricity) with symbols indicating stimulus size (1.5° black circles, 2.5° green circles, or 3.5° red circles). (B) Vector analysis of orientation bias treats acuity as a vector with magnitude = resolution spatial frequency and direction = 2× orientation of test grating. Bias is sum of vectors divided by the sum of vector lengths. (C) Nyquist frequency for an anisotropic array is the geometric mean of the Nyquist frequencies for a pair of orthogonal stimulus orientations. For the illustrated example, the x-direction is also the radial direction of stretching. (D) Convention for specifying retinal location and stimulus orientation for the right eye uses Roman typeface for terms relating to retinal location as seen by an experimenter viewing an observer's fundus. Italic typeface signifies terms relating to stimulus orientation. Meridian is measured counterclockwise from the 0° meridian (horizontal nasal retina). Absolute fringe orientation is measured by the counterclockwise angle of the bars in the fringe relative to the horizontal. Relative fringe orientation is measured from the meridian line instead of the horizontal. Radial (i.e., meridional) fringes are parallel to the meridian line and therefore have 0° relative orientation. Tangential fringes are perpendicular to the meridian line and therefore have 90° relative orientation.

The Visometer instrument (Haag-Streit Diagnostics) was mounted on a gimbal that enabled the experimenter to place fringes on any meridian of the observer's right eye up to 50° of eccentricity. We tested eccentricities 0°, 2.5°, 5°, and 10°–50° in 10° steps. To maintain a constant state of light adaptation, the fringe patch was surrounded by a white, uniform field with the same mean luminance as the stimulus. The chromatic difference between stimulus and background provided a visual cue that enabled the observer to focus attention in the appropriate location of the visual field. A fixation target seen through a viewing port kept gaze fixed in the primary position and a bite-bar stabilized the observer's head. The bite bar was attached to an XYZ linear translator that allowed the experimenter to position the Maxwellian view stimulus in the pupil center, which is essential for avoiding vignetting by the iris. Training sessions taught observers the benefit of steady fixation and the futility of attempting to fixate a peripheral stimulus that disappeared when ocular rotations toward the stimulus diverted the eye's pupil away from the instrument's optical axis and obscured the target. Neither cycloplegia nor spectacle correction was required since the contrast of interference fringes is not affected by defocus or astigmatism (Halliday & Ross, 1983; Le Grand, 1937). The authors (age range 26–41 years), all of whom were emmetropic and highly practiced in peripheral vision psychophysics, served as observers. Experimental procedures complied with the Declaration of Helsinki. 

Sampling-limited measures of visual resolution acuity were obtained by a one-sided method of adjustment. The experimenter set fringe frequency well above the resolution limit and randomly set orientation to one of four possible settings (0° = horizontal, 90° = vertical, 45° = right oblique, 135° = left oblique). The subject then reduced fringe frequency of the continuously displayed stimulus until the orientation of the fringe could be identified with confidence and signalled by hand. Subjects were instructed to blink between trials to minimize Troxler fading of stimuli. Perceptual fading was rarely reported by subjects, possibly because of small fixational eye movements and the dynamic nature of aliased percepts of fringes with constantly changing spatial frequency. This paradigm gives highly repeatable results because when fringe frequency is above the resolution limit, the stimulus appears as an unstable perceptual alias with random variations in spatial frequency, orientation, and structure (Thibos & Bradley, 1993; Thibos, Walsh, et al., 1987). When spatial frequency transitions from this nonveridical, aliasing zone of the spectrum into the veridical zone, perceptual stability is achieved and stimulus orientation can be identified with few errors. This criterion of temporal stability also avoids supra-Nyquist performance that can occur for forced-choice orientation identification for irregular sampling arrays (Evans, Wang, Haggerty, & Thibos, 2010b). A preliminary experiment in peripheral retina for two of our subjects indicated that the spatial frequency yielding 75% correct responses on a two-alternative forced-choice paradigm was approximately 10% greater than the resolution value obtained by method of adjustment. Twenty resolution settings (five each for four orientations) were obtained for foveal viewing and for each of seven eccentricities along eight meridians for a total of 1,140 settings per subject. Observers were allowed unlimited viewing time since slow adjustment of spatial frequency was encouraged to reduce intertrial variability. Bracketing adjustments, often used in method of adjustment, were not allowed since this would negate orientation identification as a perceptual veridicality task. 

To compute isoacuity contour maps of the retina, it was convenient to convert resolution measurements in cycles/deg to minimum angle of resolution (MAR = 0.5/cutoff spatial frequency), which is known to vary approximately linearly with eccentricity (Genter et al., 1981; Thibos, Cheney, et al., 1987; Weymouth, 1958). Using bilinear interpolation in polar coordinates, MAR measurements were represented on a finer spatial scale in order to locate the retinal loci for which MAR is constant. When these interpolated contour maps are displayed using a logarithmic spatial frequency axis, the contours are approximately evenly spaced and the statistical reliability of the map is nearly uniform over the retina because the ratio of standard deviation to mean of repeated measures was largely independent of test location in the visual field. Our sign convention for Cartesian coordinates of retinal maps uses positive x for nasal retina and positive y for superior retina. 

Results

Resolution acuity is defined in this report as the highest spatial frequency supporting veridical perception of sinusoidal gratings. Although resolution acuity varied slightly with grating orientation in our experiments, we averaged across target orientations initially to provide a representative value of acuity at each retinal location. The resulting depiction of mean acuity declining with stimulus eccentricity in Figure 2 was remarkably similar for all three subjects. Since each subject made five acuity settings at each of four target orientations, we had 20 measurements available for computing acuity statistics at each stimulus location. Standard deviation of the 20 settings was typically 5%–10% of the mean, independent of retinal location. The standard errors of the mean are thus smaller than the diameter of the symbols used in Figure 2, which demonstrates the high level of precision achieved by our practiced observers. This precision, in turn, revealed small but systematic differences between semimeridians. For all three subjects, acuity in the nasal retina was greater than the corresponding temporal retina (panel A) and acuity along the superior vertical midline was slightly greater than along the inferior retinal meridian (panel C). For example, at 40°–50° of eccentricity, horizontal nasal resolution was about double that for the horizontal temporal retina as reported previously (Anderson et al., 1992). The average decline in acuity with eccentricity for our three subjects is illustrated in Figure 3 as MAR curves to demonstrate their nearly linear form. These results, which are averaged across stimulus orientation, are consistent with Wertheim's finding (Wertheim, 1980) that acuity declines fastest along the retinal inferior vertical meridian (i.e., superior visual field), whereas the slowest rate of decline is for the retinal nasal horizontal meridian (i.e., temporal visual field). 

Figure 2

 

Comparison of mean acuity (averaged across stimulus orientations) as a function of retinal eccentricity for all three subjects. Symbols show mean acuity for individual subjects, and the solid curve shows the population average. Standard deviations of 20 settings were typically about 10% of the mean, which suggested a logarithmic scale for the spatial frequency axis so that confidence intervals for each point would be about the same size anywhere on the graph. Confidence intervals (±2 SEM) for individual means are smaller than symbol diameter.

Figure 3

 

Variation of minimum angle of resolution with eccentricity along eight retinal meridians. Symbols show the population average (three subjects) of the geometric mean acuity computed for the radial/tangential orientations and for the 45/135 oblique orientations. The solid curve is the mean of the eight meridional curves.

At any given retinal location, acuity varied with stimulus orientation and tended to be greatest when the grating was aligned parallel to the radial line connecting the stimulus center to the fovea, lowest for the orthogonal (i.e., tangential) orientation, and intermediate for gratings oblique to the meridional line. To quantify this tendency, we computed orientation bias at each test location in the visual field using the formula given in Figure 1B. Bias is a normalized, unitless vector with length indicating the magnitude of bias on a scale of 0 (no bias) to 1 (total bias) and direction indicating the preferred fringe orientation for maximum acuity, with 0° indicating radially oriented gratings (Figure 1D; Cheney et al., 2015). Positive and negative preferred orientations are rotated counterclockwise and clockwise, respectively, from the visual meridian. For example, a mean preferred orientation of −40 degrees for stimuli on the horizontal nasal meridian indicates that to maximize acuity the grating would need absolute orientation 140°. We tested the predictions of the radial bias model separately for all eight visual meridians and the results are shown in Figure 4. On average, the magnitude of bias increased with eccentricity, from 0.05 at 2.5° to 0.08 at 30° eccentricity, declining slightly beyond 30°, but the rate of increase varied with meridian. As predicted by the radial bias model, the preferred relative orientation was 0° on average but deviated significantly from the prediction along the horizontal nasal meridian (closed circles in Figure 4B). 

Figure 4

 

Orientation bias of resolution acuity as a function of retinal eccentricity averaged across subjects. (A) Magnitude of orientation bias. (B) Preferred stimulus orientation relative to the radial orientation. Symbols show the population mean bias for individual meridians, and the solid curve shows the average bias across meridians. Symbol key in (A) applies also to (B).

Variation of acuity across the retina for all subjects is displayed in Figure 5 in the form of topographic contour maps. In view of the radial bias described above, contour maps were computed for relative stimulus orientations rather than absolute stimulus orientations. Accordingly, we present acuity maps separately for radially oriented gratings, for gratings oriented oblique to the meridional line, and for the geometric mean of acuity for orthogonally oriented gratings. Geometric mean acuity (square root of the product, see Figure 1C) was computed for the radial/tangential pair of orthogonal orientations and again for the orthogonal pair of oblique orientations. Both of these geometric mean acuity maps were similar and therefore were averaged arithmetically for display in Figure 5. Acuity maps for tangentially oriented gratings are not shown, but were similar in appearance to the radial maps with closer isoacuity contours because acuity for tangential gratings was always less than for radial gratings. Tabulated acuity data used to produce the contour maps of Figure 5 are provided in a supplementary file. 

Figure 5

 

Retinal contour maps of log-acuity for all three subjects. Left column of maps is for radially oriented fringes, middle column is for oblique fringes, and the right column is for the arithmetic average of the geometric means for the radial/tangential orientations and oblique orientations. Contours are spaced at 0.1 log spatial frequency intervals. For clarity, only the contours for log(1 c/°) and log(3.16 c/°) are labeled. Black circles show location and size of the blind spot caused by the optic nerve head as measured by manual perimetry for each subject. Horizontal and vertical retinal coordinates are in degrees of visual angle. Positive x-values indicate nasal retina and positive y-values indicate superior retina. Tabulated acuity data used to produce the contour maps are provided in a supplementary file.

For all three subjects, contours in the radial fringe maps are elongated horizontally, revealing the functional equivalent of the anatomical visual streak (Curcio & Allen, 1990). For subject LNT, for example, the horizontal extent of the contour for 0.8 log c/°, is 30% greater than the vertical extent of the same contour. The contours are extended into nasal retina, especially for contours corresponding to lower spatial frequencies, and also extended slightly into the superior retina. Except for a difference in scale, the contour maps for radial and tangential fringes had similar form. However, for fringes oriented obliquely to the radial line connecting stimulus location and the fovea (middle column in Figure 5), the acuity contours were more circular compared to the cruciform contours in the radial maps. The most striking difference in appearance is along the vertical midline for subject AB. Instead of the cruciform shape of the radial maps, the oblique maps have a puckered appearance, indicating that acuity for fringes oriented obliquely to the radial direction is slightly lower on the vertical midline compared to the oblique meridians. 

Discussion

This study measured the highest spatial frequency of a sinusoidal grating stimulus that is perceived veridically at selected locations in the visual field. For gratings just beyond this resolution limit, the stimulus always remained visible but was misperceived as an alias that we attribute to undersampling by the retinal mosaic of neurons. We found that isoacuity contour maps for our sampling-limited task are similar in shape to those from classic studies that used a variety of stimuli and criteria that were not explicitly sampling-limited (Aubert & Förster, 1857; Wertheim, 1891). For a fixed peripheral eccentricity, acuity is greatest in nasal retina (temporal field) and lowest in the inferior retina (superior field) as reported in the classic 19th century literature. 

Our results summarized in Figure 4 confirm and extend previous evidence that resolution acuity is greater for radially oriented gratings than for other orientations, the so-called meridional effect (Anderson et al., 1992; Beirne et al., 2005; Berkley, Kitterle, & Watkins, 1975; Rovamo, Virsu, Laurinen, & Hyvarinen, 1982; Temme, Malcus, & Noell, 1985). This radial bias is not a large effect over most of the visual field but appears especially strong on the horizontal meridian, where a 2:1 ratio of performance for radial and tangential gratings is sometimes found. Acuity for gratings oriented oblique to the radial line connecting the stimulus location to the fovea was typically less than for radially oriented gratings, but greater than for tangentially oriented gratings. Our results showed no evidence of an absolute oblique effect based on absolute stimulus orientation because radially oriented oblique gratings yielded superior acuity when located on oblique meridians of the peripheral field. Since our grating stimuli were high-contrast, monochromatic interference fringes that are unaffected by optical imperfections of the eye, we infer that radial bias is due to neural factors. Furthermore, since resolution was sampling-limited in our experiments, our results support a model of resolution anisotropy due to local anisotropy in the spacing of the sampling elements (but not necessarily the size of their receptive fields), with sampling elements more widely spaced radially than tangentially as illustrated schematically in Figure 1C. 

Developmental models of the eye cite nonuniform retinal growth (Mastronarde, Thibeault, & Dubin, 1984; Robinson, Dreher, & McCall, 1989) and redistribution of neurons (Packer, Hendrickson, & Curcio, 1990) to account for the radial gradient in cell density of retinal photoreceptors and ganglion cells, as well as the radial elongation of anatomical dendritic fields (Leventhal & Schall, 1983; Rodieck, Binmoeller, & Dineen, 1985; Schall, Perry, & Leventhal, 1986) and functional receptive fields (Levick & Thibos, 1980, 1982; Soodak, Shapley, & Kaplan, 1987; Thibos & Levick, 1985). Although size and spacing of anatomical sampling elements may be correlated in such models, radial elongation of receptive fields would not be expected to limit visual acuity for a sampling-limited task. Receptive field size would be expected to limit the spatial bandwidth for contrast detection because of neural spatial summation, but spatial summation cannot explain aliasing. The fact that our subjects always perceived aliasing for grating frequencies just beyond the resolution limit proves that neural bandwidth for detection is always greater than neural bandwidth for resolution throughout the peripheral retina. Thus, it is the spacing, not the size, of neural receptive fields that limits the bandwidth of veridical perception and its variation with stimulus location and orientation. 

To draw a quantitative comparison between the variation of neural bandwidth for veridical perception across the visual field with the sampling limits imposed by retinal neurons, we employed a recently published formula for receptive field density of midget retinal ganglion cells as a function of visual field location (Watson, 2014) based on the anatomical study of human retina by Curcio and Allen (1990) and a model of displacement of foveal ganglion cell bodies from their dendritic and receptive fields (Drasdo, Millican, Katholi, & Curcio, 2007). In the final sentence of his paper, Watson says “Since the midget retinal ganglion cells provide the primary limit on human visual spatial resolution across the visual field, this formula may be useful in the modeling of human spatial vision.” (p. 12). That envisaged utility applies directly to our study of sampling limited resolution acuity so we expect a close match between psychophysical iso-log-acuity maps and isodensity maps of midget retinal ganglion cells as specified by Watson's mathematical model. Taking into account several anatomical constraints, Watson's model predicts the Nyquist limit of the combined array of ON and OFF midget ganglion cells in the human retina as a function of retinal location. Appendix 1 of Watson's paper provides an interactive tool that calculates Nyquist frequency (assuming hexagonal packing) as 0.537√D, where D is the anatomical sampling density. We relaxed the assumption of hexagonal packing by converting Watson's values to “nominal” Nyquist frequency computed as 0.5√D (Evans et al., 2010b). Using Watson's calculator to predict acuity at the same sample locations used in our empirical study, we interpolated those predictions by the same method used to make retinal maps for our psychophysical data. The result is shown in Figure 6A. Since Watson did not take local anisotropy into account, this prediction of resolution acuity is most relevant to our geometrical mean maps in Figure 5. To facilitate a direct comparison, maps of average geometrical mean acuity for our three subjects in Figure 5 were averaged and the resulting contour map for this hypothetical average subject is shown in Figure 6B. 

Figure 6

 

(A) Retinal contour maps of predicted log-acuity computed from Watson's (2014) mathematical model of ON+OFF midget ganglion cell density. (B) Average across subjects of the geometrical mean acuity maps from Figure 5. Plotting conventions are the same as in Figure 5. Positive x-values indicate nasal retina and positive y-values indicate superior retina.

The striking similarity between anatomical (Figure 6A) and functional (Figure 6B) contour maps provides strong support for the hypothesis that undersampling by midget retinal ganglion cells is the fundamental limit to spatial resolution throughout the visual field. All of the major features of the anatomical map described by Curcio and Allen (1990; see their figure 5A) and captured by Watson's formula are present also in the psychophysical maps of mean acuity for the hypothetical average subject. The isodensity contours are elongated along the horizontal meridian, and extend into nasal and superior retina by an amount that increases with eccentricity. The visual streak is clearly evident along the horizontal meridian and the asymmetries between nasal/temporal and superior/inferior meridians in retinal ganglion cell density are also apparent psychophysically. Although the contours for visual acuity are systematically smaller in diameter than the corresponding contours for ganglion cell Nyquist frequency, the degree of elongation is similar. The ratio of horizontal to vertical extent of the contour for 0.8 log c/° was 1.3 for our subjects, which agrees closely with Curcio and Allen's (1990) reported ratio of 1.28. 

Although the pattern of predicted and measured acuity maps are similar (Figure 6), there is a quantitative discrepancy examined more closely in Figure 7 by a scatter-plot of predicted and measured acuity. For any given eccentricity, each subject contributed eight points to this graph corresponding to the eight visual meridians tested. The black dashed 100% line in the figure is the theoretical prediction assuming the limiting array consists of the combined populations of ON and OFF midget ganglion cells, with each population independently sampling the cone array at interspersed locations as shown schematically by the inset of Figure 7. An alternative model suggested previously (Coletta & Watson, 2006; Lennie & Fairchild, 1994; Merigan & Katz, 1990), indicated by the black dotted 50% line, assumes the task employs only half of the midget ganglion cells. This could happen, for example, if ON and OFF ganglion cells sample exactly the same retinal locations (again shown by the inset of Figure 7), thus producing redundant sampling arrays with only half the density of the total population. Alternatively, the 50% model would apply if either the ON cells or the OFF cells (but not both) support resolution, or if neighboring pairs of ON and OFF cells function as a single sampling unit. The effective Nyquist frequency for the 50% model is smaller by the factor √2 compared to the 100% model, so both prediction lines have unity slope when plotted on logarithmic axes. Before assessing how well either model fits our data we need to review the anatomical and physiological evidence suggesting the 100% model for peripheral retina but the 50% model for central retina. 

Figure 7

 

Comparison of measured resolution acuity (symbols) with anatomical Nyquist frequency predicted by Watson's (2014) mathematical model of midget ganglion cell density. Dashed line is the prediction for sampling by 100% of midget ganglion cells, which we argue is appropriate for independent arrays of ON and OFF retinal ganglion cells that sample at maximally different locations. Dotted black line is the prediction for sampling by 50% of midget ganglion cells, which is appropriate for identical arrays of ON and OFF cells that sample the same retinal locations. Insets provide schematic diagrams of the two models. Red lines represent adjusted models for which the two black reference lines are shifted downward together so the 50% model fits the foveal data.

A 100% model is conceivable in peripheral retina because the relatively coarse array of ganglion cells undersamples the cone array (Dacey, 1993; Schein, 1988) and therefore the opportunity arises for ON and OFF cells to sample different retinal locations. However, the 50% model is more appropriate for foveal and parafoveal (0°–6°) retina where each cone connects (via bipolars) almost exclusively with one ON and one OFF midget ganglion cell (Kolb & Dekorver, 1991). For this anatomical arrangement, the array of sample locations for the ON and OFF midget cells are the same because both are identical to the sample locations of cone photoreceptors. Consequently, the Nyquist frequencies of cones, ON ganglion cells, OFF ganglion cells, and the combined ON+OFF arrays are identical. Thus, it would be equally correct to say that cone sampling limits acuity (Green, 1970), that sampling by ON ganglion cells limits acuity, that sampling by OFF ganglion cells limits acuity (Drasdo et al., 2007; Rossi & Roorda, 2010), or that sampling by the combined ON+OFF array limits acuity. All four claims invoke Watson's 50% model that predicts a foveal Nyquist frequency (61 c/°), which is significantly greater than the average geometrical mean acuities for foveal vision of our three subjects (40 c/°, range 38°–43 c/°, as indicated by the far right points in Figure 7). Possible reasons for this discrepancy are discussed below, but if we assume Watson's formula predicts acuity values that are too large by the same factor (61/40) everywhere in the visual field for our three subjects, then the two black reference lines in Figure 7 need to slide downward as shown by the pair of red lines. These revised predictions for the 50% model now fit the foveal data and the predictions for the 100% model fit the peripheral data. Thus, given this adjustment factor, a single unified model emerges for foveal and peripheral results: acuity is proportional to the sampling density of combined arrays of ON+OFF ganglion cells everywhere in the retina. That combined ON+OFF array has a sampling density determined by 100% of midget ganglion cells in peripheral retina, which provides a functional advantage of increased peripheral acuity relative to either the ON or OFF arrays acting separately. However, a combined ON+OFF array does not improve acuity in central vision because the special connectivity of the human fovea creates redundant, spatially identical ON and OFF sampling arrays with sampling density determined by 50% of midget ganglion cells. 

The preceding comparison of competing models is limited by several factors affecting the accuracy of anatomical predictions of resolution acuity. Some of these factors might apply globally across the visual field, hence requiring a multiplicative scaling factor of the kind we applied in Figure 7. First, individual differences between donor eyes in anatomical studies and living eyes in psychophysical studies are to be expected. Second, the assumed values of retinal magnification factor used to convert anatomical dimensions to visual angles are unlikely to be exactly correct for our subjects or for subjects in the literature cited above. Third, our conservative psychophysical criterion was selected to eliminate aliasing artifacts (Anderson et al., 1991), but higher acuity would have been expected for forced-choice paradigm of orientation identification. Fourth, the degree of regularity in a sampling array impacts acuity values, with more irregular arrays yielding higher acuity because of increased probability of sampling elements being closely spaced (Evans et al., 2010b). Fifth, stimulus size is an important factor since at least six cycles of the grating are required to achieve maximum acuity on the grating resolution task (Anderson et al., 1996; Evans, Wang, Haggerty, & Thibos, 2010a; Evans et al., 2010b). The largest stimulus available in our experiments was 3.5° so grating frequencies less than about 1.7 c/° would not be expected to yield acuity values that achieve anatomical predictions. Although we chose not to employ the empirical factor N/(N + 1) to correct acuity measurements for the known effect of grating truncation (Anderson et al., 1996), such a correction factor would have brought acuity for the far periphery (50° eccentricity) into closer agreement with anatomical predictions in Figure 7. 

Perhaps the most important and interesting factor influencing accuracy of predictions in Figure 7 is the degree of independence of the sampling arrays for the two subpopulations of ON and OFF ganglion cells. We envision the 100% and 50% models as two extremes of a continuum along a dimension of nearest neighbor distances. One extreme occurs if the separation between each cell in one array (e.g., ON cells) and its nearest neighbor in the other array (e.g., OFF cells) is the maximum achievable, given their local densities. In that case, the two sets of sample points are maximally independent and the full Nyquist frequency of the 100% model applies. The other extreme occurs when the two sampling arrays coincide, so the nearest neighbor distance between ON and OFF cells is zero. In this case, the ON and OFF sample points are identical, their combined Nyquist frequency is the same as either array separately, and the 50% model applies. For our subjects, peripheral resolution is greater than predicted by the adjusted 50% model fit to their foveal acuity values (Figure 7, red dotted line), which implies the complementary pair of ON and OFF neural images are combined in the brain to achieve veridical perception of spatial patterns. However, that conclusion is rather uncertain given the many possible sources of inaccuracy for such predictions enumerated above. 

Innovative experiments by Rossi and Roorda (2010) addressed the problem of intersubject variability by simultaneously using adaptive optics imaging and psychophysical testing in the same subjects to measure cone spacing and resolution acuity for a tumbling-E letter over the parafoveal eccentricity range 0°–2.5°. They found that resolution acuity was accurately predicted by cone spacing only at the foveal center. Immediately outside of the center, resolution was worse than predicted for cone spacing, more closely matching the sampling limit calculated for a model of 50% of midget retinal ganglion cells based on published anatomical data. Although intersubject uncertainty persists for an argument based on ganglion cell densities, the authors concluded that resolution is well predicted by the Dradso et al. (2007) model of sampling density for 50% of the midget ganglion cell mosaic. We concur. Our MAR values for 2.5° eccentricity (AB:1.77, LNT:1.78, RSA:1.74 arcmin) are in line with measurements presented in Rossi and Roorda's (2010) figure 2A, and fall very close to predictions made by Watson's (2014) 50% formula, which is based in part on the Drasdo et al. (2007) model (black dotted line in Figure 7). In fact, all of our data outside the fovea agree with the unscaled 50% model yet we cannot exclude the possibility that this is a spurious agreement, given all of the potential sources of inaccuracy inherent in these comparisons. Our foveal MAR values are higher (AB:0.78, LNT:0.75, RSA:0.70 arcmin) than predicted by the unscaled 50% model, which prompted rescaling of the acuity predictions shown by the red lines in Figure 7. However, it is also possible that the mismatch for our foveal data is a local phenomenon confined to central vision, so a global adjustment based on the foveal data may be inappropriate. 

We argued above that ganglion cell density should, in principle, determine acuity everywhere in the visual field. Accordingly, we suggest the longstanding debate over cone sampling versus ganglion cell sampling should be redirected to the intriguing possibility suggested by Figure 7 that discrete neural images carried by ON and OFF subpopulations of peripheral ganglion cells are combined by the brain to form a unified neural image with improved sampling density. For example, one approach for future investigations might be to devise experiments that selectively silence either the ON or OFF retinal arrays, perhaps using asymmetric temporal modulation of stimuli (Kremers, Lee, Pokorny, & Smith, 1993), to see if resolution acuity suffers. The possibility of complementary pairs of ON and OFF neural images in the brain superimposing to improve sampling density would be a positive evolutionary strategy, similar to combining independent sampling arrays from the two eyes centrally to improve binocular sampling density of a cyclopean neural image (Zlatkova, Anderson, & Ennis, 2001). 

In the vertebrate visual system, parallel pathways for ON and OFF signals are initiated at the first visual synapse (between cone photoreceptors and bipolar cells in the outer plexiform layer) and remain segregated at the retinal inner plexiform layer and thalamic relay to visual cortex (Nelson & Kolb, 2004). Over almost the entire human retina, cones outnumber ganglion cells so in order to achieve complete coverage the receptive field of an individual ganglion cell must pool signals from more than one cone (Dacey, 1993). That process of creating ganglion cell receptive fields by pooling cone signals has the dual effect of filtering the neural image and splitting it into a complementary pair of discrete neural images carried by the ON and OFF pathways. Both of these effects have the potential to limit the neural bandwidth of veridical perception, but the observation of perceptual aliasing confirms that sampling, not filtering, is the limiting factor everywhere in the retina (provided optical filtering is avoided or minimized). Combining in the brain those complementary ON and OFF neural images arising from peripheral retina would have the functional advantage of increasing sampling density to compensate partially for the inherently low acuity of peripheral vision (Zlatkova et al., 2001). This specific advantage does not extend to the fovea, but central vision might still benefit if ON and OFF arrays are combined, like in an electronic push–pull amplifier, to produce linear contrast responses from rectified inputs. 

In principle, neural processing of the retinal image by an array of receptive fields with linear spatial summation over a finite area is equivalent to a two-step sequence of spatial filtering followed by point sampling with an array of centroids of the receptive fields (Thibos & Bradley, 1995). Since the receptive fields of ON and OFF ganglion cells are determined structurally by their dendritic fields, which are segregated in the IPL and are systematically smaller (making density greater) for OFF cells compared to ON cells (Dacey, 1993), it is highly unlikely that ON and OFF sample points coincide exactly (e.g., see Dacey, 1993, figure 16). Thus, over most of the visual field, ON and OFF ganglion cells will sample different retinal locations. Therefore, combining their neural images at a later stage would increase the neural bandwidth by up to a maximum possible factor √2. The fovea is an exception to this generalization because individual cones provide the primary input to pairs of ON and OFF ganglion cells, so the two subarrays are identical. A gradual transition from full-Nyquist bandwidth in the periphery to reduced neural bandwidth in the fovea would be expected to follow the trend from oversampling of cones to undersampling of cones by the midget ganglion cells. This trend is evident in the psychophysical data of Figure 7, and in the results of Rossi and Roorda (2010), which support the long-standing hypothesis that “pixel density” of the discrete neural image carried by the human optic nerve determines the spatial bandwidth of veridical perception. 

Perceptual ambiguity of aliasing is only one aspect of misperception of spatial patterns attributed to neural undersampling. A radial gradient of sampling density also influences the topological transformation of the visual field onto the surface of the visual cortex (Dougherty et al., 2003; Tootell, Silverman, Switkes, & De Valois, 1982). Distortion of this retinotopic map, often called “cortical magnification,” results mainly from the anatomical projection of the nonuniform array of retinal ganglion cells onto uniformly distributed neurons in target nuclei of the brainstem and cortex (Wassle, Grunert, Rohrenbeck, & Boycott, 1990). This distortion of the visual field has wide-ranging implications for the neural representation and perceptual coding of natural scenes (Fischer, 1973; Schwartz, 1980). Retinotopic projection can be described mathematically as a complex logarithmic conformal mapping (Schwartz, 1980) that compresses the peripheral field relative to the central field and makes the neural image invariant to size, rotation, and projection scaling (e.g., “looming” caused by forward locomotion in a stable environment). 

To demonstrate the dual effects of aliasing and retinotopic distortion of a natural scene shown in Figure 8A, we simulated the neural image for a retinal gradient in sampling density inferred from our experimental measurements of resolution acuity. For demonstration purposes, we simplified the complex logarithmic model of space-variant sampling by including only the radial component of sampling along a horizontal strip of visual field. To create the simulated neural image in Figure 8B, we assumed the retinal image is well focused and point-sampled by a rectangular array of ganglion cells whose spacing increases linearly from central visual field (the far left side of the scene) to a midperipheral location at 30° eccentricity (the far right side of the scene). The angular scale corresponds to viewing an adult zebra (2.1 m long) from a distance of 30 m. Nonuniform sampling was produced by exponentially stretching a square sampling array in the horizontal direction to achieve a linear gradient in spacing equal to that of the steepest curve in Figure 3 (30 arcmin per 40° of eccentricity). The resulting array of sampled intensity values was then packed into the uniformly spaced matrix of pixels displayed in Figure 8B. This sampled image represents the neural image carried by the optic nerve with one pixel per ganglion cell axon. Assuming no further distortion occurs in subsequent stages of the visual pathway, this simulated retinal image represents the distorted retinal image present at individual layers of the lateral geniculate nucleus or primary visual cortex. 

Figure 8

 

Simulation of the neural image carried by retinal ganglion cell axons of the optic nerve. (A, top panel) Original scene contained in a narrow strip of horizontal visual field extending from the fovea (far left) to mid periphery (30° eccentricity, far right). (B, bottom panel) The simulated neural image rendered in a uniform space containing one pixel per neuron. To create the simulation, an initially square sampling array was stretched exponentially in the horizontal direction to produce a space-variant array with horizontal spacing between adjacent cells proportional to eccentricity. Aliasing due to undersampling is evident in the misrepresentation of the zebra's stripes. Peripheral compression of the scene (i.e., “cortical magnification”) is another functionally important consequence of logarithmic space-variant sampling.

Aliasing first appears in the simulated neural image of Figure 8B at about 5° of eccentricity, where zebra stripes resemble the mottled coat of the spotted leopard. Perhaps similar misperception by predatory carnivores with coarse mosaics of retinal ganglion cells even in central retina (Hughes, 1977, 1985) provides additional camouflage and survival value for zebra stripes. Although aliasing artifacts are obvious in this particular scene, and would be even more dramatic for a dynamic movie of running zebras producing motion aliasing (Anderson & Hess, 1990; Coletta, Williams, & Tiana, 1990; Wang, Thibos, & Bradley, 1996), misperception of other scenes lacking periodic structures will depend on the spatial frequency content of the scene and its contrast spectrum relative to visual contrast thresholds. For example, Galvin and Williams (1992) reported the absence of perceptual aliasing at high-contrast edges, which was later traced to insufficient contrast in those spatial frequencies that were misrepresented by undersampling (Wang, Bradley, et al., 1997a). Similarly, optical blurring of the retinal image and spatial summation by ganglion cell receptive fields will attenuate the contrast of neural images by an amount that depends on the stimulus and visual contrast sensitivity at each retinal location. Thus, the relative importance of sampling and contrast attenuation for scene perception will be expected to depend not only on the performance characteristics of central and peripheral vision, but also on the spatial frequency spectrum of the scene. 

Acknowledgments

This work was supported by NIH/NEI grant EY05109 to LNT. We thank Dr. Andrew Watson for providing the computational tool used to estimate ganglion cell density. 

Commercial relationships: none. 

Corresponding author: Larry N. Thibos.