Eye movements facilitate stereo-slant discrimination when horizontal disparity is noisy

Abstract

Conditions in which saccadic gaze shifts within planar surfaces facilitate stereo-slant discrimination for slant about the horizontal and vertical axis were investigated. When horizontal disparity noise was added, large gaze shifts in the direction of the slant lowered stereo-slant discrimination thresholds compared to thresholds measured with steady central fixation, whereas eye movements orthogonal to the slant orientation did not lower slant-discrimination thresholds. When no horizontal noise was added, performance was the same with and without gaze shifts. These results suggest that slant is recovered from depth differences between target edges when horizontal disparity signals are variable and that foveal fixation improves the measures of disparity. Eye movements did not lower slant thresholds by providing multiple foveal samples of slant at different target locations that were averaged to reduce disparity noise levels, because eye movements only lowered the thresholds when there was a depth difference between the fixation points. To study which signals for azimuth are used when slant is recovered from the difference in depth between target edges, vertical disparity noise was added and stimulus height was reduced. Both methods elevated slant-discrimination thresholds when horizontal disparity noise was present, suggesting that vertical disparity is used as a cue for azimuth.

Introduction

Orientations of natural surfaces such as irregular or uneven ground planes are normally inspected with scanning eye movements. In this work, we investigate whether eye movements within a target facilitate slant discrimination when horizontal disparity signals are variable. There are two reasons why facilitation might occur, and both rely on the same assumption (i.e., that stereo-slant estimates are weighted more at the fovea than in the para-fovea or periphery). Stereo-acuity, which measures the sensitivity to relative disparities, is higher within the fovea than at peripheral locations (Fendick & Westheimer, 1983), and it is higher on the horopter than off the horopter (Enright, 1991; McKee, Welch, Taylor, & Bowne, 1990; Ogle, 1956; Wright, 1951). In sequential stereo, where two absolute disparities are compared between two targets separated in time, stereo acuity also decreases with target separation (increasing retinal eccentricity) (Enright, 1991; McKee et al., 1990; Ogle, 1956; Wright, 1951). Disparity noise sources associated with sequentially presented targets are independent so they cannot be cancelled by a differencing process (Westheimer, 1979). If absolute disparity noise increased in the periphery, this would contribute to the reduction of sequential stereopsis with large target separations (McKee et al., 1990). Thus, weighting foveal absolute or relative disparity signals more at the fovea than periphery could improve slant discrimination. However, it is not clear whether the visual system makes use of this potential advantage when making foveal gaze shifts to different locations on a single slanted surface. 

What sources of information for stereo-slant discrimination could be enhanced by foveal gaze shifts? Slant estimates can be based on the difference in depth between two widely separated points, recovered from the combination of absolute horizontal binocular disparity and vergence cues at a given fixation point (binocular parallax) (Foley, 1978) or from disparity differences between adjacent points (i.e., relative disparities). Foveal gaze shifts could improve the resolution of depth differences between points near target edges, and they could also provide multiple foveal samples of slant at different target locations. The slant samples could be averaged to reduce relative disparity noise levels. We will discuss this in detail below. 

Slant Interpolated From Binocular Depth Estimates at Two Locations

Estimates of slant about the vertical axis (further referred to as horizontal slant) based on binocular depth cues can be computed by interpolating between depth estimates at two horizontally separated locations. Slant estimates based on depth differences would be more accurate with widely separated locations than closely separated locations if the two targets are fixated sequentially, because the depth difference for a given slant angle increases with target separation (Bridge, Cumming, & Parker, 2000), the retinal noise at the fovea is constant, and the eye position noise is assumed to be small and/or independent of saccade amplitude. 

Binocular depth cues could be used for slant estimation by interpolating between depths from binocular parallax signals at two separate locations. Binocular parallax is the difference in the head-centric visual directions of a target in space (Foley, 1978), so it has also been called head-centric azimuth disparity (Erkelens & van Ee, 1998). It is defined by  

 

(1)

with i the interocular distance, ϑ the eccentricity from straight-ahead, and D the distance (depth). When accurately fixating a certain target location in space, the binocular parallax of that location is equal to the vergence angle. If there is a vergence error or fixation disparity, then the binocular parallax is equal to the vergence angle plus the residual horizontal disparity (assuming that there is no cyclo-torsion). Azimuth signals are needed to correct depth estimates from binocular parallax signals (Fendick & Westheimer, 1983) and to compute the separation between the two depth estimates. In the “Appendix,” we derive an expression of how slant can be computed from binocular parallax estimates at two locations (γ₂ and γ₂) and the azimuth eccentricities of the same locations (ϑ₁ and ϑ₂). 

There are both retinal and extra-retinal sources for obtaining information about azimuth, namely vertical disparity and eye-position signals (Backus, Banks, van Ee, & Crowell, 1999; Rogers & Bradshaw, 1995). Retinal disparity signals for azimuth can be obtained from horizontal gradient of vertical disparity (Backus et al., 1999; Gillam & Lawergren, 1983; Rogers & Bradshaw, 1993). Extra-retinal signals of azimuth can be obtained from the version angle plus the averaged retinal eccentricity from the two eyes’ foveas. 

Slant Estimated From Relative Disparity at the Fixation Point

Horizontal slant estimates can also be based directly on differences between adjacent horizontal disparities (relative disparity). Relative disparities between different target locations can be described as a horizontal gradient of horizontal disparity (Δ horizontal disparity / Δ horizontal separation). They can also be described as the horizontal size ratio (HSR) (Gillam & Lawergren, 1983; Rogers & Bradshaw, 1993). HSR is the size comparison at one retinal location between features in the left and right eye image. HSR can be converted into horizontal gradient of horizontal disparity (horizontal disparity gradient) by  

 

(2)

. 

The use of relative disparities near threshold is dominated by the fovea due to the poor spatial resolution in the periphery (Schor & Badcock, 1985). Furthermore, in natural scenes, the disparity gradient is not constant, so that the disparity gradient is obtained in a small area. 

Relative disparity signals do not provide sufficient information by themselves to estimate slant. Slant can be estimated by combining relative disparities with azimuth and distance information. The horizontal disparity gradient is scaled with distance and corrected for azimuth to compute slant. HSR can be mapped into head-centric coordinates by  

 

(3)

(Backus et al., 1999). Distance and azimuth can be obtained from extra-retinal cues (i.e., eye position signals) of vergence (μ) and version (γ). Retinal sources (vertical disparities) could also be used for obtaining the azimuth and distance information. 

Two Explanations of Why Eye Movements Might Lower Slant Discrimination Thresholds

The way that gaze shifts could enhance slant discrimination depends on which binocular cues are used for slant discrimination (i.e., relative disparities at a limited area or the difference in depth between two widely separated points). 

Foveal gaze shifts could improve slant estimates by allowing observers to sample slant from relative disparities at different regions of a planar surface. Multiple samples of slant at different target locations, each location containing independent noise, could be averaged to reduce the noise level. This strategy would be most beneficial when horizontal disparity signals were variable, for instance when random depth variations were added to points on the surface. Then, it would find the best-fit planar slant to represent the surface. Note that the visual system is not averaging the measured disparity or HSR at each location because HSR varies with azimuth for a horizontally slanted surface. For example, HSR for a fronto-parallel plane located at a 57-cm viewing distance is 1.0 in primary direction and is 1.09 at 20-deg azimuth. These two values of HSR correspond to the same slant angle when azimuth information is taken into account (see Equation 3). The noise from disparity variations in HSR for estimating overall slant of the surface is scaled by vergence and added to the slant estimate. Averaging slant estimates effectively averages out the independent disparity noise. 

Another possible advantage to making gaze shifts is to estimate slant by interpolating depths between widely separated points near the left and right edges of the horizontally slanted planar surface. Foveal fixations improve the resolution of depth signals at the target edges. The difference between the two-point depth estimates is large in comparison to the depth changes within a small patch used to obtain HSR at a single fixation point. If a certain amount of disparity noise is added, the depth signal for the two-point comparison is larger than the depth variations within the single fixation area, whereas the noise is the same for both signals. Therefore, the difference in depth has a larger signal-to-noise ratio than the relative disparities. 

Both averaging slant samples at different locations and interpolation of slant between two widely separated points are possible without eye movements, but eye movements can bring the different locations onto the fovea and the horopter, where relative and presumably absolute disparity measures are most sensitive. 

Studies of supra-threshold slant and curvature perception indicate that eye movements do not facilitate stereo-slant or curvature estimates of smooth surfaces. Scanning eye movements did not reduce the bias in supra-threshold stereo-slant estimates of planar surfaces (Van Ee & Erkelens, 1999). In stereo-based curvature discrimination, a small improvement occurred when making eye movements (Rogers, Bradshaw, & Glennerster, 1994). In these experiments, slant estimates were probably based on relative disparities at a limited area at the fovea. A study on the influence of eye movements on stereo-slant discrimination would be more convincing than these studies, but has not been described yet in the literature. 

We assume that the visual system estimates stereo-slant from the cues that provide the most reliable information about horizontal disparity with either single or multiple fixations. If a slant estimate based on relative disparities at a single fixation point is at least as reliable as both the averaged slant estimate from multiple locations and the slant estimate based on difference in depth at two locations, then eye movements will not improve the slant estimate. In the experiments where eye movements did not improve performance (Rogers et al., 1994; Van Ee & Erkelens, 1999), there was sufficient information about relative horizontal disparities at any location on the smooth plane that allowed accurate slant or curvature estimates. However, in natural scenes, irregular surfaces such as tree foliage or brush on a ground plane are not flat or smooth. Under these conditions, horizontal disparity signals for overall surface orientation can be variable. Then, the overall surface orientation of an object might be more visible when stereo-slant is estimated from the large difference in depth at two or more points near the horizontal edges of the surface, or when several stereo-slant estimations based on relative disparities are averaged, than when stereo-slant is estimated from small changes in horizontal disparity within a limited area. We assume that the visual system does not estimate stereo-slant by weighting the various disparity cues according to their reliability (Clark & Yuille, 1990; Landy, Maloney, Johnston, & Young, 1995). Weighting of the various disparity cues is inconsistent with the results of previous studies in which eye movements did not improve performance (Rogers et al., 1994; Van Ee & Erkelens, 1999). If weighting did occur, then the saccadic eye movements would always improve slant estimation, because the difference in absolute disparity between two separate locations is estimated more accurately when the two locations are foveated. 

In the first experiment, we investigated whether horizontal eye movements could improve horizontal slant discrimination when external noise was added to the horizontal disparity. Adding horizontal disparity noise to a smooth surface is equivalent to the effect of surface irregularity, and it increases the variance of horizontal disparity signals. Indeed, we found that horizontal stereo-slant discrimination thresholds were lower when making horizontal eye movements than when fixation was kept constant at the target center when the noise was added. 

In the second experiment, we tested the averaging explanation of why eye movements might improve slant discrimination of irregular surfaces. Both horizontal and vertical gaze shifts would allow averaging of slant from relative disparities at multiple locations. Thus, if eye movements facilitate slant discrimination by averaging, then both horizontal and vertical eye movements would lower discrimination thresholds for slant about the horizontal or vertical axis. However, if eye movements facilitated slant discrimination by bringing two separated points with different depths onto the fovea, then only horizontal gaze shifts would facilitate horizontal slant discrimination and only vertical gaze shifts would facilitate discrimination of slant about the horizontal axis (further referred to as vertical slant). This outcome would reject the averaging hypothesis. 

Correcting Slant Estimates for Target Azimuth

Azimuth information is needed to correct slant estimates from either depth estimates at two locations or relative disparity (Fendick & Westheimer, 1983). With tall smooth surfaces (Backus et al., 1999; Rogers & Bradshaw, 1995), vertical disparity is weighted much more than eye-position signals for information about azimuth and distance. It is unknown whether retinal or extra-retinal sources for azimuth are used for the two-point depth difference strategy when slanted surfaces are noisy and when vertical disparity information is available. 

Azimuth information is necessary in two ways to estimate slant from the difference in depth at two points. First, azimuth information is needed to estimate absolute depth from the absolute horizontal disparity at one location (azimuth correction). Figure 1 shows the need for azimuth correction. It is an example of two points on a slanted surface, which have the same absolute horizontal disparity, namely zero and the same vergence angle. These points are at a different absolute depth because the azimuth angle is different. This azimuth information can be obtained either from vertical disparity or eye position signals. Second, azimuth information in combination with the retinal eccentricity is needed to estimate the separation (difference in direction) between the two points. It is likely that eye position signals are used to estimate the difference in direction because it has been shown that vertical disparity does not influence perceived direction (azimuth) (Banks, Backus, & Banks, 2002; Berends, van Ee, & Erkelens, 2002). 

Figure 1

 

This is an example of two points on a slanted plane that both have zero disparity when fixated and the same vergence angle (μ), but they have different absolute depths because their azimuth is different.

In Experiments 3 and 4, we tested whether retinal cues were used for obtaining azimuth in the two-point depth difference strategy. We reduced the vertical disparity information by adding vertical disparity noise or by decreasing stimulus height and measured these affects on slant discrimination. If a reduction of retinal cues for azimuth elevated the slant-discrimination thresholds based on depth differences between target edges, then slant estimates are corrected with retinal cues for azimuth, just as they are with slant estimates that are based on relative disparity. 

Summary

The goal of this study was to find out whether scanning eye movements could lower horizontal stereo-slant discrimination thresholds for irregular surfaces, what information was used under these conditions (horizontal disparity or perceived depth), and whether retinal or extra-retinal sources were used to obtain azimuth information for recovering slant from depth differences between two widely separated points. 

General Methods

Display and Stimuli

The stimuli were displayed on a 20-in. monochrome monitor (Monoray Model M20ECD5RE, Clinton Electronics, Lafox, IL, USA) at 120-Hz noninterlaced frame rate with 1,024 by 768-pixel resolution. This monitor had a fast DP 104 phosphor that decays to 0.1% peak in 0.6 ms with a burn resistant property. The fast phosphor decay is critical for minimizing the cross talk between images presented to left and right eyes because we were using the same screen area with shutter glasses to generate the stereograms. Video images were controlled using Visual Stimulus Generators (VSG) 2/5 graphics card (Cambridge Research Systems, Kent, England) in a host Pentium II computer. The images were corrected for any screen distortions at the 30-cm test distance using a grid-loom calibration method (Backus et al., 1999). At that viewing distance, each pixel subtended 3.9 min arc. Subpixel resolution was obtained by anti-aliasing each dot under assumption of adjacent pixel linearity. 

Stimuli were viewed through 120-Hz Ferro-shutter optics (model FE-1 ferro-electric shutter goggle; Cambridge Research Systems). Each eye viewed stimuli at 60 Hz with no discernable flicker. 

The observer’s head position was restricted by means of a bite board and headrest to position the observer at the calibrated viewpoint. The stimuli were presented in complete darkness to eliminate visibility of the room, the edges of the monitor and facial features as a frame of reference. 

The stimuli were large rectangular random-dot patches (60 deg horizontal by 32 deg vertical) such that for a given horizontal slant angle, the difference in depth between the edges was large relative to the disparity gradient. The random dots were sparse (5% dot density) and irregularly spaced to minimize perspective and texture cues for surface orientation. Due to the low number of dots (68 on average), it was hard to recognize the rectangular shape. In Experiment 2, the size of the stimulus was 48 by 48 deg, which resulted in 82 dots with 5% dot density, and in Experiment 3, we varied the size of the stimulus. The size of a dot is defined by the width of the Gaussian luminance profile (σ = 2/3 pixel) and its peak luminance of 4.2 cd/m² when viewed through the Ferro-shutters. Each slant stimulus presentation was a different random-dot display to avoid changes in perceived image compression as a cue. The stimuli were presented at the center of the screen (straight-ahead). Horizontally slanted stimuli were obtained by applying a horizontal magnification of one eye’s image. At the 30-cm viewing distance, a 1% magnification of the left eye’s image (M = 1.01) corresponds to a slant angle of approximately 2.6 deg. Vertically slanted stimuli were obtained by applying a horizontal shear to one eye’s image. At the 30-cm viewing distance, a 1° shear of the left eye’s image corresponds to a vertical slant angle of approximately 4.6 deg. 

Procedure and Analysis

Before each trial, a fixation mark and two vertical nonius lines (1-deg long) were placed in the center of the display. Observers initiated a trial by pressing a mouse button. The nonius lines were replaced by the reference stimulus (a fronto-parallel base slant), followed by the slanted test stimulus presented at the same screen location. Reference and test patches were presented sequentially at the center of the monitor for 3 s each with an inter-stimulus interval (ISI) of 0.5 s. A 3-s stimulus presentation is sufficient time to make several saccades and for the stereo-percept to develop (Van Ee & Erkelens, 1996). The forced-choice task for the observer was to indicate whether the left or right side of the test stimulus was slanted farther away from the observer than the corresponding side of the reference stimulus. No feedback was provided regarding the correct response. 

Two conditions were measured, one in which subjects maintained steady fixation and the other in which subjects made either horizontal or vertical saccadic eye movements. In the steady fixation condition, subjects fixated a small dot in the center of the display during the whole trial. In the horizontal eye movement condition, subjects fixated the center dot during the presentation of the reference target. After the test target came up, subjects made free scanning eye movements, typically four large saccades during the 3-s interval that shifted gaze from the center to 20 deg to the left and right of the center of the 60- or 48-deg-wide stimulus. The eye movements were made along the horizontal meridian that passed through the center of the target. The vertical eye movement condition was similar, but subjects shifted gaze from the center to 20 deg above and below of the center of the 48-deg-tall stimulus. 

Horizontal magnification of the test stimulus was varied according to the method of constant stimuli. In each trial, the magnification was selected randomly from 1 of 9 levels, and each level was presented 6 times in a given session. The magnification or the shear range differed across observers and conditions. Data from four to six sessions were averaged and fit (maximum-likelihood fit) with a psychometric function (cumulative Gaussian) to estimate a threshold or just noticeable difference (JND). The JND is half of the difference between the values of the independent variable corresponding to 16% and 84% of correct performance (d′ = 1). We estimated the SEs of the discrimination thresholds by performing Monte-Carlo simulations on the data sets. Three observers (authors) were tested (EB, ZZ, and CS). 

Experiment 1: Horizontal Disparity Noise

In the first experiment, we investigated whether eye movements could facilitate horizontal slant discrimination of irregular surfaces. The stimuli were made irregular by adding horizontal disparity noise. 

Methods

In this experiment, we added noise to horizontal disparity. The noise reduces both the signal-to-noise ratio of the relative disparities at a limited area and the signal-to-noise ratio of the difference in stereo-depth between left and right edges of the stimulus. Adding noise reduces the signal-to-noise ratio more for the relative disparities within a limited area than for the larger difference in depth between the widely separated horizontal edges of a slanted surface. Eye movements could lower slant discrimination thresholds, either by averaging samples of slant from relative disparities at several locations or by improving resolution of difference in depth near the edges of wide surfaces when horizontal noise is added. 

We compared horizontal slant thresholds measured with and without horizontal eye movements as a function of the amount of added horizontal disparity noise. Horizontal disparity noise was defined as a random horizontal shift on the screen of each dot in the unmagnified half-image. The distribution of the noise magnitudes was Gaussian. A Gaussian distribution can describe the histogram of the noise magnitudes:  

 

(4)

. 

The mean of the distribution of the noise magnitudes (σ) was zero. The SD of the distribution of the noise magnitudes (σ) specified the magnitude of the noise. σ was constant over the stimulus:  

 

(5)

. 

The noise level (σ) is expressed in minutes of arc. 

Results and Conclusions

The results of Experiment 1 are shown in Figure 2. In both the steady fixation (open circles) and horizontal eye movement conditions (closed circles), thresholds increased as noise increased. Horizontal noise elevated slant-discrimination thresholds more in the without-eye-movements condition than in the with-eye-movements condition. The range of noise levels was varied per subject in order to reveal the facilitory effect of eye movements on slant discrimination. Subject CS showed the facilitory effect at a very high noise level (100 min arc), and he could perform the task well with this noise level, whereas subject EB was unable to perform the task at 100 min arc of noise. She showed the facilitory effect of making eye movements at a much lower noise level (36 min arc). Although the thresholds for subject EB are higher than for CS and ZZ, the trends in the results are the same. The differences in thresholds between subjects can be explained by the difference in stereo-acuity measured with the Randot stereo test (CS: 5 s arc; EB: 25 s arc; ZZ: 5 s arc). 

Figure 2

 

The results of Experiment 1. Each panel shows the relation between the amount of horizontal noise and the slant-discrimination threshold for one subject. The open circles represent the steady fixation condition and the solid circles represent the horizontal eye movement condition. Note that the scales on both the horizontal and vertical axis differ per subject.

These results support prior reports that eye movements do not facilitate slant estimation of smooth surfaces, in which the disparity gradient at a single location provides a reliable disparity signal for slant estimates (Van Ee & Erkelens, 1999). However, eye movements facilitated slant discrimination when the relative disparities near the point of fixation were variable. Therefore, this result suggests that when horizontal disparity is smooth (not noisy), slant estimates are based on a single sample of relative horizontal disparities at a limited area on the surface, but when horizontal disparity is noisy, slant is estimated from multiple samples from several target locations. 

Experiment 2: Vertical Eye Movements

In the second experiment, we tested the hypothesis that horizontal noise level was reduced by averaging several samples of slant obtained by shifting gaze to different points along the surface. If the reduction of slant discrimination thresholds with eye movements was based on the average slant sampled at multiple locations, then either vertical or horizontal eye movements would lower the slant discrimination threshold. This hypothesis was tested by having subjects make vertical eye movements along the mid-sagittal line of a surface or horizontal eye movements through the center of a surface. The surface was slanted either about the horizontal or vertical axis. 

Both horizontal and vertical eye movements make averaging slant over several locations possible. However, for surfaces slanted about the vertical axis (i.e., horizontally slanted surfaces), vertical gaze shifts are to points that have a constant depth, whereas horizontal gaze shifts are to points whose depth varies with surface slant. For surfaces slanted about the horizontal axis, the opposite is true; vertical gaze shifts are to points whose depth varies with surface slant, whereas horizontal gaze shifts are to points that have a constant depth. We assume that if vertical and horizontal eye movements facilitate slant discrimination of rough surfaces equally, then eye movements could facilitate slant discrimination by averaging samples of slant at different locations. However, if vertical eye movements do not facilitate slant discrimination of rough surfaces that are slanted about the vertical axis and horizontal eye movements do not facilitate slant discrimination of rough surfaces that are slanted about the horizontal axis, then the improvement of slant discrimination in Experiment 1 is not due to averaging samples of slant at different locations. 

Methods

To compare the conditions in which horizontal and vertical saccades were made, height and width of the large stimuli were kept the same (48 deg). The height we used was the maximum height that was possible with our experimental set-up. As in Experiment 1, the dot density was 5% (82 dots). We added noise to horizontal disparity as in Experiment 1. For each subject, we restrict the noise to one high noise level (60 min arc for EB and 100 min arc for ZZ and CS). 

Two slant orientations were studied. Slant about the vertical axis was obtained by horizontally magnifying one eye’s image, and slant about the horizontal axis was obtained by horizontally shearing one eye’s image. 

We compared the effects of making vertical and horizontal eye movements on slant discrimination thresholds measured for the two slant orientations. In the baseline condition, no eye movements were made, and subjects fixated a small dot in the center of the display. In the horizontal eye movement condition, subjects shifted their gaze from the center to 20 deg to the left and right of the screen center. The vertical eye movement condition was similar, but subjects shifted gaze from the center to 20 deg above and below of the screen center. Subjects made the same number of saccades in the horizontal and vertical eye movement condition (i.e., typically four saccades). Thus, they could take the same number of samples of slant at different locations in both conditions. 

Results and Conclusions

The results of Experiment 2 are shown in Figure 3. For slant about the vertical axis, thresholds for the vertical eye movement condition and the steady fixation condition were similar for all subjects, whereas the thresholds for the horizontal eye movement condition were lower than the other two thresholds for ZZ and EB and were the same as the other two thresholds for subject CS. Thus, vertical eye movements do not facilitate slant discrimination of surfaces slanted about the vertical axis. Subject CS probably has a wide spatial integration area for horizontal disparities or good peripheral acuity, because horizontal eye movements do not facilitate slant discrimination of 48-deg wide stimuli, whereas they do facilitate slant discrimination of 60-deg wide stimuli. The other two subjects probably have a smaller integration area or worse peripheral acuity, because horizontal eye movements facilitate slant discrimination of the narrower 48-deg wide stimuli. 

Figure 3

 

Each panel shows the results of Experiment 2 for one subject. The left side of the panel shows the slant discrimination thresholds for slant about the vertical axis, which is expressed in just noticeable difference (JND) horizontal size ratio (HSR) (left y axis). The right panel of the graph shows the slant discrimination thresholds for slant about the horizontal axis, which is expressed in JND shear (right y axis). The y axes were scaled such that a certain bar height represents the same veridical slant angle for both slant orientations. Black bars represent the horizontal eye movement condition, dark gray bars the vertical eye movement condition, and light gray bars the steady fixation condition.

For slant about the horizontal axis, thresholds for the horizontal eye movement condition and the steady fixation condition were similar, whereas the thresholds for the vertical eye movement condition were lower than the other two thresholds for ZZ and EB, and they were the same for CS. Thus, horizontal eye movements do not facilitate slant discrimination of surfaces slanted about the horizontal axis. A large integration area for horizontal disparities or good peripheral acuity can again explain that the thresholds for the three conditions are the same for CS. 

We found that horizontal eye movements did not lower slant discrimination thresholds for slant about the horizontal axis, and vertical eye movements do not lower slant discrimination thresholds for slant about the vertical axis. These results suggest that the improvement of slant discrimination with eye movements does not result from averaging at multiple locations. 

Experiment 3: Vertical Disparity Information

Azimuth information is used in two ways to estimate slant from the depth difference at two locations. First, azimuth information is needed to obtain the absolute depth from the absolute disparity at each location. This azimuth information can be obtained either from vertical disparity or eye position signals. Secondly, azimuth information is needed to estimate the separation (difference in direction) between the two points. It is likely that eye position signals are used to estimate the difference in direction because vertical disparity is not used to estimate direction. 

The purpose of this experiment was to determine if the azimuth information was based on retinal signals when horizontal slant was estimated in the presence of horizontal disparity noise, from binocular depth difference between two horizontally separated locations. 

In this experiment, the retinal cues for azimuth were made less reliable by adding noise to the vertical disparity (Experiment 3a) and by reducing the stimulus height (Backus et al., 1999; Rogers & Bradshaw, 1995) (Experiment 3b). If vertical disparity was used to obtain azimuth information, then performance would decrease as either vertical disparity noise increased or stimulus height decreased. However, if slant discrimination was unaffected by vertical disparity noise and reduced stimulus height, then it is likely that extra-retinal cues from version eye position were used for obtaining azimuth information. 

Both methods used to reduce the vertical disparity information have their own drawback. The vertical noise interferes with binocular matching. Reducing the height also reduces the stimulus area and number of dots that could influence sensitivity to the horizontal disparity gradient. The results of Experiments 3a and 3b should follow the same trends, if the matching problem, the optimal area to sense the horizontal disparity gradient, and the number of dots do not play an important role in limiting performance. 

A control experiment (Experiment 3c) was carried out to investigate the influence of the dot number on slant discrimination. 

Methods

Subjects always made horizontal eye movements that were approximately 20 deg to the left and right of the target center. We measured two conditions, namely with and without the presence of a constant amount of uniform horizontal disparity noise. Horizontal noise levels were used from the prior experiment, where performance with the uniform horizontal disparity noise was better with than without eye movements (37.5 min arc for EB, 60 min arc for ZZ, and 100 min arc for CS). In the horizontal noise condition, subjects appear to have used the difference in depth near the target edges to estimate slant. In the no-horizontal noise condition, it is likely that subjects used the relative disparities in the center of the stimulus to estimate slant. 

Experiment 3a: Vertical disparity noise

The stimulus size was 60-g wide by 32-deg high as in Experiment 1. The added vertical disparity noise was a random vertical shift on the screen of each dot in the unmagnified half-image. The noise distribution was Gaussian with a mean of zero as used for the horizontal disparity noise. The SD (σ) was not constant over the stimulus, but it was proportional to stimulus height (y) in a Cartesian coordinate system with its origin in the center of the screen.  

 

(6)

. 

The noise scale factor (= const) is expressed in minutes of arc. The constant equals the maximum vertical disparity noise level in the stimulus and is used to specify the magnitude of noise. 

It follows from Equation 6 that the vertical disparity noise was zero on the horizontal center line and large above and below that line. In this way, we disrupt only the useful vertical disparity information both for azimuth and distance, because vertical size ratios are more easily measured if target height increases (Backus et al., 1999; Rogers & Bradshaw, 1995). A disparity noise proportional to y yields a constant VSR (vertical size ratio) noise, because VSR is proportional to the vertical gradient of vertical disparity:  

 

(7)

. 

Furthermore, with this type of noise, we were able to use high vertical noise levels without disrupting binocular sensory fusion where subjects fixated. 

Experiment 3b: Short stimuli

The height of the 60-deg wide stimulus was varied. The reference and the test stimulus always had the same height. The following heights were used: 32, 16, 12, 8, 4, and 0.5 deg. A dot density of 5% was used for stimulus heights of 32, 16, and 12 deg. The stimuli for these heights contained on average 68, 34, and 25 dots, respectively. For the narrower heights, a fixed number of dots, namely 12, was used because a dot density of 5% was too sparse and too few dots were visible to perceive a surface. 

Experiment 3c: Control for number of dots

We compared thresholds of an 8-deg-high stimulus for a low dot density (12 dots, as we used in Experiment 3b) and for a higher dot density (34 dots, which we used for the 16-deg-height stimulus in Experiment 3b). This control experiment was carried out with horizontal noise, because the horizontal noise condition resulted in the steepest curve and is thus the most sensitive to changes in the number of dots. 

Results and Conclusions

Figure 4 (Experiment 3a) shows that vertical disparity noise reduced performance for both levels of horizontal noise, suggesting that vertical disparity was used to obtain azimuth information. The thresholds measured with and without horizontal noise increase with similar trends. This suggests that the same source of information for azimuth is used when slant is based on the two-point depth difference as when slant is based on the relative disparities on a limited area 

Figure 4

 

The results of Experiment 3a. Each panel shows the relation between the amount of vertical disparity noise and the slant discrimination threshold for one subject. The solid circles represent the no-horizontal-noise condition, and the open circles represent the horizontal-noise condition. Note that the scales on both the horizontal and vertical axis differ per subject.

A plateau was observed for all three subjects at high vertical disparity noise levels in the without-horizontal noise condition. EB and ZZ also showed a plateau for the with-horizontal noise condition. The plateau could be caused by a switch from retinal signals (vertical disparity) in the absence of vertical disparity noise to extra-retinal signals (eye-position signals) when the vertical disparity noise was present for obtaining the azimuth information. 

The results of Experiment 3b also suggest that vertical disparity was used to obtain azimuth information. Thresholds in both the with- (open circles) and without-(filled circles) horizontal noise condition became elevated as the height was reduced (Figure 5). The short heights affect ZZ the least, then CS and EB the most. ZZ may have used the smallest area to do the slant discrimination task and EB the largest. This is consistent with the results of Experiment 3a, where the vertical noise, which depends on height, affects ZZ the least and EB the most. The thresholds for the horizontal noise condition increase more than the thresholds for the without-noise condition. In the with-horizontal noise condition, the threshold increased as height decreased, and it never plateaued as it did in the vertical disparity noise condition shown in Figure 4. The reason for the consistent increase might be related to the optimal size of the integration area for sensing horizontal disparity. The disparity gradient probably has an optimal integration area, and when horizontal noise is added, it may require a larger area to resolve the signal than when no noise is added. The visual system could average over increasing areas until it obtains a detectable signal. Narrow height stimuli would have insufficient area to average out the horizontal noise. 

Figure 5

 

The results of Experiment 3b. Each panel shows the relation between the stimulus height and the slant-discrimination threshold for one subject. The solid circles represent the no-horizontal-noise condition, and the open circles represent the horizontal-noise condition. Note that the scales on the vertical axis differ per subject.

Figure 6 illustrates that the thresholds for the 12-dot stimuli (black bars) are not significantly different from the thresholds for the 34-dot stimuli (gray bars) in Experiment 3c. Thus, the increase of thresholds with decreasing stimulus height is not due to the number of dots, but due to the reduced area. The absence of a dot density effect has also been observed for slant settings (Backus et al., 1999; Rogers & Bradshaw, 1995). 

Figure 6

 

The results of Experiment 3c. Each panel shows the effect of the number of dots on the slant-discrimination thresholds for one subject. Thresholds were measured with two different numbers of dots, namely 12 dots (black bars) and 34 dots (gray bars). The thresholds were measured in the presence of horizontal noise (37.5 min arc for EB, 60 min arc for ZZ, and 100 min arc for CS), and the target height was 8 deg.

Experiment 4: Control for Vertical Eye Alignment

This is a control experiment to investigate whether the vertical disparity noise could interfere with horizontal disparity matches by producing vertical vergence errors. We tested whether the eyes were less aligned vertically after a 20-deg saccadic eye movement was made toward the right edge of the target when vertical disparity noise was present than when noise was absent. Binocular matching would be more difficult if the eyes were not aligned properly (vergence errors). Although, even if vertical vergence signals were accurate, the vertical noise could still make binocular matching in the periphery harder. 

We estimated the variability of vertical eye alignment with a dichoptic Vernier acuity task with horizontal nonius lines. Fahle has demonstrated that the amplitude of vertical disjunctive eye movements during steady fixation can be inferred from the thresholds for vertical dichoptic Vernier acuity (Fahle, 1991). Here we are interested in the fluctuations of vertical vergence associated with horizontal saccadic eye movements in the presence of vertical disparity noise. We measured vertical dichoptic Vernier acuity after subjects made a rightward horizontal saccade to a long vertical line (40 deg) that provided disparity feedback for horizontal vergence but no feedback for vertical vergence. 

Methods

To measure fluctuations in vertical alignment, we placed two horizontal nonius lines on either side of the binocular vertical fixation line, and we varied the vertical position of one of them according to the method of constant stimuli (Fahle, 1991). The nonius lines were separated horizontally by a 10-min arc gap. 

Figure 7 shows the time course of Experiment 4. Observers initiated a trial by pressing a mouse button when the vertical nonius lines in the center of the display appeared aligned. Then a random-dot pattern that contained a long vertical fixation line at 20 deg to the right of the screen center was presented for 567 ms. The random-dot pattern was the same as the reference target used in Experiments 1 and 3. The long vertical fixation line extended the whole length of the screen so that it did not provide vertical disparity cues for vergence alignment. Subjects were asked to make a rightward horizontal gaze shift to the fixation line along the horizontal meridian that passed through the center of the target. After 400 ms, the horizontal nonius lines (1.8-deg horizontal by 10-min arc vertical) were presented briefly (167 ms). The stimulus onset latency of the horizontal nonius lines was 400 ms such that subjects had just enough time to make a saccadic eye movement to fixate the long vertical line. The time course of normal saccade includes a 250-ms latency and a duration of 50–100 ms. The presentation time of the horizontal nonius lines was short to prevent the subject from making horizontal or vertical eye movements in response to disparities subtended by the nonius lines. After the nonius lines were presented, the screen became black and subjects indicated whether the left or the right nonius line was higher by clicking the mouse pad. 

Figure 7

 

The time course of stimulus presentation in Experiment 4. The nonius lines were presented just after the subject made the eye movement. The nonius lines were presented briefly in order to prevent the subject from making eye movements when they were present.

For each subject, we measured vertical dichoptic Vernier acuity with the same vertical disparity noise levels used in Experiment 3. The vertical dichoptic Vernier acuity is the just notable difference (JND) in vertical misalignment between the left and right eye’s horizontal nonius lines. The JND of vertical misalignment is determined by the half of the difference between the values of the independent variable corresponding to 16% and 84% of correct performance or a d′ of 1. 

Results and Conclusions

Figure 8 shows the results of Experiment 4. Without vertical disparity noise, vertical dichoptic Vernier acuity ranged from 3–8 min arc for different subjects. This threshold range is higher than reported in prior studies (Fahle, 1991; McKee & Levi, 1987). The lower thresholds reported by Fahle and McKee’s compared to ours are likely to be explained by the saccadic eye movements made in our experiment. This is precisely the condition we wanted to investigate. If there is more vertical misalignment of the eyes in the presence of vertical disparity noise, then elevation of slant-discrimination thresholds in the presence of vertical disparity noise might be caused by the vertical eye misalignment rather than vertical disparity being an unreliable cue for distance and azimuth. 

Figure 8

 

The results of Experiment 4. Each panel shows the relation between the amount of vertical disparity noise and the vertical dichoptic Vernier acuity for one subject.

The results of Experiment 4 indicate that the elevation of stereo-slant thresholds with low vertical disparity noise levels is not the result of vertical eye misalignment. The dichoptic Vernier thresholds did not increase with increasing vertical disparity noise for subjects EB and ZZ, and they increased gradually for subject CS (Figure 8). 

Changes in vertical Vernier threshold with increasing vertical disparity noise were not correlated with changes in stereo-slant thresholds due to increasing vertical disparity noise (Figure 4). There was no significant difference between Vernier thresholds for the first two values of vertical disparity noise for all subjects. Thus, small amounts of vertical disparity noise do not affect the vertical alignment. Therefore, the increase in stereo-slant thresholds for vertical disparity noise in Figure 4 is likely to be due to decreased reliability of vertical disparity signals. 

Discussion

Binocular Depth Signals for Recovering Slant

Stereo-slant of a plane can be estimated with two classes of binocular cues. Either slant can be sensed directly from relative disparities at any surface location, or slant can be computed by interpolating between the depths, estimated from absolute disparities, at two widely separated locations. Then slant is obtained from sequential stereopsis in combination with the difference in azimuth between the two locations. If slant is estimated from relative disparities, it can be sensed at one location or it can be sensed at several locations and averaged. 

The first experiment demonstrated that when horizontal noise was added to the random-dot horizontal slant stimulus, thresholds were lower with than without horizontal saccadic gaze shifts. However, when no noise was added, eye movements did not lower the thresholds. Horizontal disparity noise could have reduced the signal-to-noise ratio for small changes in relative disparity at one location more than the signal-to-noise ratio of the larger depth differences, estimated from absolute disparities, between target edges. Although the noise is the same for both types of binocular cues, the signal (difference in depth) is much smaller within single fixation area than for the 2-point comparison. Disparities could be sensed at several locations without moving the eyes; however, saccadic gaze shifts bring the locations in the fovea and on the horopter where sensitivity to both absolute and relative disparities is highest. An alternative explanation is that saccades provide an opportunity to reduce noise by averaging slant estimates from several points of fixation. The averaging of slants at different sample locations reduces the variability of the perceived slant, because the noise at each location is independent. 

The literature (Howard & Rogers, 1995; Howard & Templeton, 1964; Van Ee & Erkelens, 1999) suggests that eye movements can facilitate slant estimation, because they prevent fatigue or normalization. However, if this is true then eye movements would facilitate stereo-slant discrimination in both the with- and without-horizontal disparity noise condition. 

The second experiment showed that vertical eye movements did not improve discrimination of slant about the vertical axis when horizontal disparity was noisy, whereas horizontal eye movements did. For slant about the horizontal axis, we found the opposite. Therefore, the facilitation of slant discrimination of rough surfaces by eye movements was not a consequence of averaging multiple samples of slant. Rather, eye movements appear to improve slant discrimination by bringing locations near the target edges into the fovea and on the horopter to obtain an accurate estimate of the depth differences between the edges. Comparison of the results of Experiments 1 and 2 for CS indicate that eye movements only lower slant discrimination thresholds when the difference in depth between the two (or more) fixation points is large. 

We found that saccadic gaze shifts improved stereo-slant discrimination when slant was obtained from sequential stereopsis in combination with the difference in azimuth. In sequential stereopsis, saccadic gaze shifts improve stereo-depth discrimination (Enright, 1991; Ogle, 1956; Wright, 1951; Zhang, Berends, & Schor, 2003) by improving sensitivity to absolute disparities at the widely spaced targets (McKee et al., 1990). Furthermore, saccadic gaze shifts can bring each of the separated locations closer to the horopter where stereo-depth discrimination is optimal (Blakemore, 1970; Schor & Badcock, 1985; Stevenson, Cormack, Schor, & Tyler, 1992). 

Retinal Sources of Azimuth

Both azimuth and distance information are necessary to recover slant from relative horizontal disparity signals (Backus et al., 1999) (see Equation 3). Azimuth information is also necessary to estimate slant from the difference in depth at two points. Azimuth is used both for absolute disparity correction (Gårding, Porrill, Mayhew, & Frisby, 1995) (see Equation 1) and for estimating the separation (difference in direction) between the two points. There are both retinal and extra-retinal sources of information for distance and azimuth, namely vertical disparity and eye-position signals. Both sources are used for estimating slant and curvature (Backus et al., 1999; Rogers & Bradshaw, 1995). Both eye-position signals (version and vergence) and vertical disparity have potential noise, and the most weighted source probably depends on which cue is most reliable (Clark & Yuille, 1990; Landy et al., 1995). However, vertical disparity is not used to estimate azimuth (Banks et al., 2002; Berends et al., 2002). In this study, we investigated the potential retinal and extra-retinal sources of azimuth required to correct slant estimates from depth difference between two horizontally separated points. 

In Experiment 3, we used two methods to reduce the vertical disparity information (i.e., adding vertical disparity noise and reducing the stimulus height). Slant discrimination thresholds were elevated when small amounts of vertical disparity noise were added to vertical disparity, and the threshold plateaued with larger amounts of noise for both the with- and without-horizontal noise conditions (except for subject CS in the with-horizontal noise condition). The plateau observed in the presence of vertical disparity noise could represent a switch from vertical disparity cues to extra-retinal eye-position cues for obtaining azimuth information for estimating the absolute depths at the two locations. 

Slant-discrimination thresholds increased as height decreased for both the with- and without-horizontal disparity noise condition, but the thresholds never plateaued as they did when vertical disparity noise was added. The finding for smooth surfaces is consistent with the literature (Backus et al., 1999). Backus et al. measured the magnitude of supra-threshold stereo slant as a function of stimulus height when gaze angle and vertical disparity were in conflict. They found that when stimulus height was large, retinal information for azimuth was used, whereas for short heights, extra-retinal information was used. 

We found no plateau when stimulus height was reduced, possibly because factors other than reduced vertical disparity information might have increased the thresholds as target height was reduced. For example, there is less area in narrow height stimuli over which to process the horizontal gradient of horizontal disparity, or the horizontal positional disparities near target edges. Thresholds would continue to increase as height was reduced, independent of which cue to horizontal disparity was used for recovering slant, if the integration area were not optimal. The integration area for horizontal disparity is not constant, but depends on the disparity modulation frequency content of the stimulus. Tyler and Kontsevich (2001) found summation fields up to 8-deg wide for low frequencies (0.5 disparity cycles/deg). Thus, for planar slant stimuli such as used in our studies, with very low disparity modulation frequencies, the upper limit for the integration area is expected to be large (8 deg or up). A larger integration area might be expected when horizontal disparity noise is added in order to average out the noise. This might explain why a smaller area affects performance more when horizontal disparity is noisy than when the surface is smooth. The increase in slant discrimination thresholds observed with decreasing height might also have been related to the reduced number of dots. However, in Experiment 3c, we showed that the number of dots did not affect the slant discrimination threshold. 

For the with- and without-horizontal noise conditions, both adding vertical disparity noise and reducing stimulus height elevated slant-discrimination thresholds when saccadic foveal gaze shifts were made between target edges. Thus, similar to slant based on relative disparities at a limited area, when slant is estimated from the differences in depth between several locations, if vertical disparity is available, it can be used as a retinal source of azimuth information to estimate depth at each target location. Natural scenes are full field and do not contain extrinsic vertical noise, except for potential false matches of ambiguous textured patterns, and intrinsic vertical noise does not depend on target elevation, so that normally, the visual system is more likely to rely more on retinal than extra-retinal cues for azimuth to recover slant from binocular depth cues. 

Conclusions

Irregular depth variations in natural surfaces add horizontal disparity noise to the disparity signals for overall surface slant. We observed that when noise was added to horizontal disparity signals, local variations of horizontal disparity (relative disparities) were noisier and therefore sensitivity of stereo-slant that was based upon relative disparities was reduced. Then, slant judgments were based upon the difference in depth from disparity signals between locations near target edges. Eye movements brought the horizontal disparities onto the fovea and on the horopter where accuracy of absolute disparity is highest. When using binocular depth cues to estimate slant, azimuth information for the correction of absolute disparity may have been influenced by extra-retinal signals when vertical disparity noise was added or the stimulus was narrow. When slant is estimated from depth difference between two points and there is sufficient vertical disparity information, azimuth information for the correction of absolute disparity is obtained from retinal cues. 

Acknowledgements

We are grateful for suggestions from Marty Banks and Michael Landy. The work was supported by National Eye Institute Gant EYO 8882. E.B. was supported by a Talent Grant from the Netherlands Organization for Scientific Research. Part of the research was presented at the Vision Sciences Society meeting in 2002 (Sarasota, Florida, USA). Commercial relationships: none. 

Appendix

Slant From Binocular Parallax

Here we derive an expression to quantify slant from the difference in depth based on binocular parallax between two locations (γ₁ and γ₂) and the eccentricities of the same locations (ϑ₁ and ϑ₂). 

Slant about the vertical axis is the change in depth (z) divided by the change in horizontal direction (x):  

 

(8)

z and x depend on distance, D and the eccentricity, ϑ of a location. D can be substituted with  

 

(9)

by means of the binocular parallax equation (Foley, 1978). Then we obtain the following expressions for z and x:  

 

(10)

 

 

(11)

Substituting the expressions for x₁, x₂, z₁ and z₂ in Δz/Δx gives the following expression for slant:  

 

(12)

.