Coarse scales, fine scales, and their interactions in stereo vision

Abstract

Human stereo vision can resolve remarkably small depth differences between two stimuli, but the smallest resolvable difference is usually that between stimuli located near the plane of fixation. As distance from this plane increases, so does the smallest detectable increment in disparity. We examined this loss of resolution by comparing disparity discrimination thresholds for single-scale and multi-scale stimuli as a function of the pedestal disparity. For single-scale gratings, disparity thresholds display phase constancy; thus, their spatial thresholds vary reciprocally with grating spatial frequency. For multi-scale gratings, with components separated in frequency by two or three octaves, disparity thresholds display two types of interaction between coarse-scale and fine-scale components: facilitation when pedestal disparities are moderate and interference when they are large. The facilitation extends the disparity range that yields the low thresholds associated with fine-scale components, limiting the loss of disparity resolution for multi-scale stimuli.

Introduction

Multi-scale processing allows a division of labor in the analysis of stereoscopic depth. Disparity-sensitive mechanisms tuned to high spatial frequencies can respond selectively to small depth differences between objects near the horopter, whereas mechanisms tuned to lower-frequencies can respond to larger depth differences over a greater range. Because of the difference in the receptive-field sizes of these mechanisms, this division of labor is often referred to as the size-disparity correlation (Felton, Richards, & Smith, 1972; Marr & Poggio, 1979; Schor & Wood, 1983; Schor, Wood, & Ogawa, 1984; Smallman & MacLeod, 1994). Objects in natural images generally contain a broad range of frequencies and if an object has disparity D, then all of its frequency components have disparity D. Despite the size-disparity correlation, however, we typically see objects with large disparities as coherent; we do not usually see the object’s features smeared across depth planes in an order determined by their spatial-frequency content (Boothroyd & Blake, 1984; Farell & Li, 2004). Evidently, the response of low-frequency (coarse) mechanisms alters the response of high-frequency (fine) mechanisms to large disparities. 

In the 25 years since Marr and Poggio’s 1979 paper, the idea that different scales interact in stereopsis has been highly influential, yet has gathered sparse empirical support beyond that provided by object appearance. Here we show evidence for cross-scale interactions that extend the precision of disparity discrimination without biasing perceived depth. Countering these benefits is a greatly reduced discriminability for some stimuli at large standing disparities. 

Coarse-to-fine matching-range shifts

In proposing a solution to the correspondence problem, Marr and Poggio (1979) suggested unidirectional, coarse-scale-to-fine-scale interactions. These interactions overcome stereo matching-range limitations built into the size-disparity correlation. In Marr and Poggio’s cross-channel bootstrapping operation, mechanisms tuned to low frequencies detect a disparity and then pull the matching range of mechanisms tuned to higher frequencies to this disparity. In particular, vergence eye movements initiated in response to disparities registered by low-frequency mechanisms bring the residual disparity into the range of the higher-frequency mechanisms, whose matching range would otherwise fall short. Other versions implement a similar normalization of input to higher-frequency channels neurally instead of through eye movements (Nishihara, 1984; Quam, 1987). 

Microscopes with coarse and fine adjustments implement the Marr and Poggio strategy. The coarse adjuster shifts the range of the fine adjuster. Focusing with each in turn allows efficient and precise selection of the desired depth plane. Yet there is little to suggest that the human visual system operates similarly, despite the advantages coarse-to-fine interactions would seem to confer. A shift in the fine-scale matching range would lessen the correspondence problem by eliminating potential false matches confronting high-frequency mechanisms. However, humans can perceive stereoscopic depth, including transparency mediated by high-spatial-frequency components, in stimuli flashed too briefly for eye movements (Prazdny, 1987; Rohaly & Wilson, 1993; Farell & Li, 2004). And human vergence changes can be initiated by high-frequency stereograms whose disparities exceed the half-cycle limit (Frisby & Mayhew, 1980; Mowforth, Mayhew, & Frisby, 1981; but see Howard & Rogers, 1995). Physiological support for coarse-to-fine disparity interactions has been reported recently (Menz & Freeman, 2003), though the evidence is indirect, dependent on measures (such as latency) correlated with coarse-versus-fine tuning rather than dependent on coarse-to-fine interactions, and ambivalent in the most direct measures (strength vs. number of coarse-to-fine and fine-to-coarse connections). 

In addition to easing the correspondence problem, matching-range shifts should also increase disparity resolution off the horopter. If unshifted, mechanisms tuned to high frequencies should contribute to disparity discrimination only near the horopter. If shifted, they could contribute even at large standing disparities, where their fine resolution ought to facilitate the discrimination of depth between multi-scale stimuli. Evidence for such facilitation has been sought in disparity increment thresholds as a function of the standing, or pedestal, disparity (Rohaly & Wilson, 1993; Smallman & MacLeod, 1997). However, it is not entirely clear what the shift hypothesis predicts for these thresholds. It has usually been reasoned that the hypothesis calls for flat increment threshold functions: At all pedestals, the disparity increment threshold should be limited by the stereo resolution of the channel that is most sensitive at a pedestal of zero, that is, the range-shifted fine-scale channel (Rohaly & Wilson, 1993). The idea is that a sequence of shifts, oculomotor or neural, would cancel the disparities resolvable at all but the finest scale, reducing the effective pedestal disparity to zero. In practice, however, a coarse channel is a noisy guide for estimating disparity. A shift applied to fine channels will be limited in accuracy by the disparity resolution of coarse channels, and the coarse-channel resolution is assumed to be crude. That’s one reason for having coarse-to-fine interactions in the first place. 

Consequences of a propagation of errors from coarse-to-fine channels are sketched in Figure 1. Thresholds as a function of pedestal disparity are shown in Figure 1a for independent fine and coarse channels. The open circle marks the disparity threshold (vertical axis) for a stimulus with zero pedestal. In Figure 1b and 1c, the coarse-channel disparity estimate shifts the disparity range of the fine channel. An accurate estimate by the low-resolution coarse channel (Figure 1b) shifts the disparity range so it is centered on the stimulus disparity. The result is a threshold equal to the threshold at a pedestal of zero. However, an imprecise estimate by the coarse channel shifts the fine-channel matching range non-optimally. The result of Weber-limited resolution would be increment thresholds that increase as the pedestal grows larger (Figure 1c). Still, the increase should be no greater than that found for a coarse-scale stimulus alone. 

Figure 1

 

Coarse-to-fine matching range shifts. a. Disparity increment thresholds are typically smallest on a disparity pedestal of zero and increase with pedestal size. For near-horopter discriminations, the lowest threshold is mediated by fine-scale (high-frequency) channels (circle). b. A range-shifted fine channel, if shifted by the stimulus disparity, will yield the same threshold as is found near the horopter. The dotted line gives the resulting flat increment threshold function. c. A fine channel shifted imprecisely will yield a higher threshold than found near the horopter. The increment threshold function in this case will not be flat. If the shift error is correlated with the pedestal, as predicted by the rising thresholds for the coarse channel, then the increment threshold function will be monotonically increasing.

Tests of the coarse-to-fine hypothesis seem to render such details moot. Not only are threshold functions for coarse-plus-fine stimuli not found to be flat, but there is also little evidence that the presence of fine stimulus components confers an advantage, driving thresholds below the level obtained with coarse components alone. There is, in fact, evidence of a disadvantage. Smallman and MacLeod (1997) combined two filtered random-dot samples having center frequencies of 2 and 8 c/d. Disparity thresholds for this compound stimulus were generally higher than those for the low-frequency stereogram alone, especially at large disparity thresholds, just the opposite of what is expected from a matching-range shift. Rohaly and Wilson (1993) have also examined increment thresholds for low- and high-frequency combinations and found no evidence of the facilitation expected from coarse-to-fine interactions. Increment thresholds are also little affected by a two-octave frequency difference between parallel, non-overlapping difference-of-Gaussian bars (Siderov & Harwerth, 1993). Yet while superimposed low- and high-frequency patterns, two octaves apart, showed no interaction in the detection of disparity increments in the Rohaly and Wilson (1993) study, they did restrict the fusion range. The restriction was asymmetric, being centered on the disparity of the low-frequency grating, and was coarse-to-fine: Low frequencies restricted the range for high frequencies, not the reverse (Wilson, Blake, & Halpern, 1991; Rohaly & Wilson, 1993).1 

Figure 1 implies that coarse-to-fine interactions should occur at moderate to large disparities. At small disparities fine-scale channels alone should mediate stereo performance. Thus at small disparities the coarse-to-fine hypothesis is indistinguishable from an independent-channel model in predicting no interaction. Support for independence comes from Heckmann and Schor (1989), who found that disparity detection thresholds for compound stimuli with zero pedestal equaled the smallest of the component thresholds. This lowest-threshold component, however, could be the one with the higher spatial frequency or the one with the lower spatial frequency. 

Whence channel interactions?

We’ve seen that while object appearance suggests some sort of coarse-to-fine stereo interaction, psychophysical performance seems consistent with channel independence (and where interactions occur, with channel interference [Smallman & MacLeod, 1997]). Evidence has not favored Marr and Poggio’s shift theory or other means of facilitating stereo performance through cross-scale interactions. 

We looked for evidence of interactions in disparity discriminations using stimuli that differed in three ways from those of previous studies: The spatial-frequency bandwidth was narrower, the luminance contrast was lower, and the pedestal range included phase disparities that were smaller. The purpose of this parameter selection was to restrict the set of active channels and to examine pedestals for which no comparable data currently exist. 

We have previously measured disparity increment thresholds for single-frequency gratings similar to the component gratings used here (Farell, Li, & McKee, 2004). The threshold function did not fit the classical mold. Phase-disparity pedestals up to about 60° had only a small effect on threshold, and the effect of pedestal size was non-monotonic; the threshold function usually showed a dip, with thresholds reaching a minimum at pedestals of 20–30° on either side of the horopter. Beyond a pedestal of approximately 60°, the increment threshold function rose rapidly for these stimuli, and phase disparities beyond about 120° often produced diplopia and ambiguous depth. It is clear from these and other data (Ogle, 1952; Badcock & Schor, 1985; McKee, Levi, & Bowne, 1990) that disparity discrimination passes through several regimes as the pedestal grows from small to large. Thinking that channel interactions are more likely to be bound to some of these pedestal ranges than to others, our interest lies particularly in moderate disparities in the region between the zero-disparity pedestal at which Heckmann and Schor (1989) found independent processing of component frequencies and the relatively large pedestals at which Smallman and MacLeod (1997) found interference. 

For components separated in spatial frequency by two or three octaves, we found that increment thresholds can be substantially lower than they are for either component grating alone. These results offer partial support for coarse-to-fine stereo interactions of the sort proposed by Marr and Poggio (1979), implemented neurally (Nishihara, 1984; Quam, 1987), matching-range shifts, or related processes, not previously confirmed in data. For two-component compound gratings, this cross-scale facilitation occurs over a limited disparity range and gives way to cross-scale interference at large pedestal values. Adding components extends the disparity range over which facilitation is observed, with thresholds approaching those for random-dot patterns with flat spatial-frequency spectra. Thresholds for random dots increase exponentially with standing pedestal, but still are low compared with grating thresholds. Thus, a high-frequency stimulus produces low disparity thresholds over a small pedestal range and adding a low-frequency component extends the pedestal range over which these low thresholds are found. As a result, reasonably fine disparity discrimination is possible for multi-scale stimuli even at a considerable distance from the horopter. 

Methods

Stimuli

A bipartite Gabor patch was the stimulus. The lower half was presented at the pedestal disparity. The upper half was presented either at the pedestal-plus-increment disparity or at the pedestal-plus-decrement disparity, with increments and decrements equal in absolute value. The observer judged the upper half-Gabor as “near” or “far” relative to the lower half-Gabor. 

The Gabor patches were either simple or compound. The former had a single-frequency carrier and the latter had a two-frequency carrier with a two- or three-octave difference between components. Carrier orientation was vertical. For compound Gabor patches, sinusoidal luminance modulations with spatial frequencies of 0.5 c/d and either 2.0 or 4.0 c/d were added and their disparities yoked in space, so the phase disparities of the components were proportional to their frequencies. The absolute phase of each sinusoidal component was independently randomized from trial to trial identically for the two eyes, changing the composite luminance profile unpredictably between trials without affecting disparity. The top and bottom halves of the Gabor patch were then separated by a sharp-edged horizontal band, 18′ high, which was set to the background luminance, and the carrier in the upper and lower halves given different disparities. Thus the cyclopean images of the upper and lower half-Gabors were in phase. Figure 2 shows an example. 

Figure 2

 

A near-threshold bipartite stereo Gabor patch with compound carrier. Upper and lower half-Gabors differ in disparity, which in both cases is nonzero.

All the stimuli had a Gaussian envelope with a horizontal and vertical space constant of 2° of visual angle (σ = √2°). The Gaussian was truncated at ±4°. Thus compound grating patches had the same envelopes as simple grating patches, whatever their spatial frequencies. The spatial-frequency bandwidth of the Gabor patches, measured at half height and full width, was approximately 0.8 octave at 0.5 c/d, 0.2 octave at 2 c/d, and 0.1 octave at 4 c/d. Contrast of each carrier frequency was 0.1. 

Left and right half-stimuli were displayed on the two sides of a flat-screen, luminance-calibrated CRT. Viewing was through a mirror stereoscope at an optical distance of 93 cm. The visible screen area on each side subtended approximately 10.5° (horizontal) × 16° (vertical) in visual angle and pixels subtended 1.5′ on a side. The grating patches were centered on black fixation squares, 6′ of visual angle on a side, which were continuously visible throughout the run of trials. The Gaussian window and the fixation square had a disparity of zero; the only nonzero disparities were interocular carrier phase shifts. The screen outside the stimulus boundaries had a uniform luminance of approximately 20 cd/m², matching the mean stimulus luminance. 

Stimulus presentation was controlled by a MATLAB program incorporating the Psychophysical Toolbox extensions (Brainard, 1997; Pelli, 1997). An attenuator (Pelli & Zhang, 1991) combined the video outputs to drive the monitor’s green gun with a luminance resolution of about 12 bits; the frame rate was 75 Hz, and each frame was presented to both eyes. 

Procedure

The lower half of the Gabor patch was presented at the pedestal disparity. The upper half was presented either at the pedestal-plus-increment disparity or at the pedestal-plus-decrement disparity. All components of compound gratings had the same spatial disparity. Observers judged the upper patch as “near” or “far” relative to the lower patch. The pedestal was fixed throughout a run; an increment or a decrement was presented at random from trial to trial. 

Observers initiated a trial with a mouse click. A quarter-second later nonius lines, positioned above and below the fixation square, disappeared (but the fixation square remained). The bipartite Gabor patch appeared 125 ms later and remained on the screen for 150 ms. Responses were made by clicking labeled buttons that appeared on-screen 0.5 s after stimulus offset. A subsequent click initiated the following trial. Observers were instructed to begin a trial only after nonius alignment and to remain fixated on the central square until the response screen appeared. At the end of each trial, a tone gave the observer feedback about whether the response was correct. A run consisted of 40 trials. 

Trial-to-trial disparity increments were under the control of the QUEST algorithm (Watson & Pelli, 1983; King-Smith, Grigsby, Vingrys, Benes, & Supowit, 1994) with a threshold criterion of 82% correct. The resolution of the QUEST staircase and of the threshold measurements was equated in phase disparity for simple and compound gratings. 

Observers

There were three observers, including one of the authors. Two observers were highly experienced in stereo vision experiments, and two were naïve about the purposes of the experiments. All had normal or corrected-to-normal acuity and normal stereo vision. 

Results

Figure 3 shows disparity increment thresholds as a function of pedestal value for 0.5-c/d, 2-c/d, and 0.5+2-c/d Gabor patches for two observers. Disparities are expressed in spatial units (minutes of visual angle) and in phase units (phase angle in degrees for 0.5 c/d and compound gratings, one-quarter of the phase angle for 2-c/d gratings). Pedestals ranged up to 60 min for one observer (Figure 3a) and 40 min for the other (Figure 3b). Data for the third observer (with pedestals to 40 min) were similar. 

Figure 3

 

Disparity increment thresholds as a function of disparity pedestal for simple and compound Gabor patches with spatial frequencies of 0.5 and 2 c/d. Lower and left scales give spatial disparity pedestal and threshold in minutes of visual angle; upper and right scales give phase disparity pedestal and threshold in degrees (or in the case of 2 c/d, degrees ÷ 4). a and b. Thresholds for full range of pedestals for two observers. c and d. Thresholds for low and moderate pedestals to show detail. Error bars are ±1 SEM.

Looking first at simple-grating data, thresholds for the 0.5-c/d grating are flat or dip slightly, with little or no increase, through pedestals of 20′ (60° phase disparity). This is typical of phase disparity increment thresholds for simple grating patterns (Farell et al., 2004). As the pedestal becomes larger, thresholds increase linearly (with a slope of approximately 1 on a log-log plot). The 2-c/d grating goes through a pedestal disparity range of 2 cycles for one observer (Figure 3a) and 1.33 cycles for the other (Figure 3b). Thresholds are low at small disparities (and about equal in phase disparity to the 0.5-c/d thresholds), and then rise precipitously as the pedestal approaches 15', a disparity of one half cycle. The curve then reverses (threshold disparity grows smaller as the pedestal increases between 180° and 360°) before the pattern of thresholds repeats in the second cycle of pedestal disparity. The symmetry of the threshold of the function mirrors the periodicity of the waveform; for example, pedestals of 10′ (120°) and 20′ (240°), which give rise to similar thresholds, are the same in absolute value. However, the stimuli at these two disparities appear on opposite sides of the fixation plane. 

Thresholds for the compound grating can be broken down into three main segments. Data for the first segment, with pedestals below 20′ (60° phase angle), are shown again for more detailed viewing in Figure 3c and 3d. Here thresholds for the compound grating are well below the thresholds for the low-frequency grating (e.g., Figure 3c means: 2.1′ vs. 4.3′) and are similar to thresholds for the high-frequency grating at the smallest pedestals (Figure 3c high-frequency mean for pedestals <10′: 1.85′). The variability of threshold measurements is also similar for the compound grating and the 2-c/d grating over this range of pedestals; it is generally smaller than the variability for 0.5-c/d gratings, even after normalizing for the 2:1 ratio of thresholds. 

In the next segment, covering pedestals of 20′ to 30′, thresholds for the compound grating approximately equal those for the 0.5-c/d grating (Figure 3a and 3b). The upper limit of this pedestal segment, 30′, is a 90° phase disparity for these gratings. In the final segment, covering pedestal disparities beyond 30′—one cycle of the 2-c/d grating—interference prevails: Thresholds for the compound grating are not limited by the thresholds of either component, and are higher, in both mean value and variability, than in simple-grating thresholds. 

Thresholds for 0.5- and 4-c/d gratings appear in Figure 4 for the same two observers. Compound gratings were tested at pedestals from 0′ to 20′, taking the 4-c/d component of the compound grating through more than a full cycle of disparity for one observer (Figure 4a), and from 0′ to 30′, two cycles, for the other (Figure 4b). Thresholds for the 0.5-c/d grating are those plotted in Figure 3. For the 4-c/d grating, thresholds are low at small pedestals and peak at a pedestal of 7.5′, a phase disparity of 180°. They resemble a scaled version of the 2-c/d data (Figure 3), having roughly half the spatial thresholds and therefore similar phase thresholds. Compound grating thresholds equal the 4-c/d thresholds at near-zero pedestals and increase to match the 0.5-c/d thresholds at pedestals of 15′ or 20′. At intermediate pedestal values, the comfpound threshold is below both component thresholds and at greater pedestals; beyond a cycle or so of the higher-frequency component, it exceeds them both. These results are similar to those for 0.5+2-c/d compounds, but occur within a compressed spatial pedestal range, scaled downward by approximately a factor of 2, reflecting the 2:1 ratio between the stimuli in their high-frequency components. 

Figure 4

 

Disparity increment thresholds as a function of disparity pedestal for simple and compound Gabor patches with spatial frequencies of 0.5 and 4 c/d. Scales are as in Figure 3; for 4-c/d stimuli, phase angles should be multiplied by 8. a and b. Thresholds for two observers. Error bars are ±1 SEM.

Whether the discrimination of depth of natural objects shows the facilitation seen here at moderate pedestals must be considered in light of the interference seen at large pedestals. From our data the facilitation is roughly a factor-of-two effect—a compound grating has a similar threshold as its high-frequency component grating at roughly double the pedestal disparity. . This extension of the range of fine-scale processing could confer important performance advantages—unless it is cancelled by cross-scale interference. Our two-frequency data give no indication of how a more continuous frequency spectrum might affect the balance between facilitation and interference. One possibility is further facilitation. This could happen if interactions occurred only between mechanisms selective to nearby scales and only within limited disparity ranges. Then the effect of multiple components could be a cascade of local interactions that is felt most strongly at the finest scale. This would occur if -to use the Marr & Poggio 1979, theory as an example- the coarsest-scale mechanism shifted the matching range of the next coarsest, which in turn shifted the next, and so on. The finest scale would receive the total accumulated shift, which could exceed the shift possible with two-frequency stimuli. Alternatively, the interference seen between two frequencies may be enhanced by the addition of more. 

To find out, we measured increment thresholds for the remaining combinations of the components we have used:2 + 4 c/d (one observer) and the three-component compound grating, 0.5+2+4 c/d (two observers). These sinusoids were combined with equal contrasts into bipartite Gabor patches. Relative phases between the components were randomized identically for upper and lower half-Gabor at each presentation. The procedures followed those used earlier. Figure 5 compares the thresholds with those for the other compound Gabor stimuli (Figures 3 and 4) and for Gaussian-distributed random-dot stereograms, also with zero-disparity envelopes, gathered on the same observers (and reported in Farell et al., 2004). 

Figure 5

 

Disparity increment thresholds for 0.5+2+4-c/d Gabor patches and random-dot stereograms (a and b) and 2+4-c/d Gabor patches (a only) as a function of disparity pedestal. Data from 0.5+2-c/d and 0.5+4-c/d stimuli from Figures 3 and 4, respectively, are shown for comparison. Upper and right phase disparities are scaled to the period of gratings containing 0.5-c/d components. Error bars are ±1 SEM.

The three-component compound produces the familiar pattern: facilitation at moderate pedestals where thresholds for the compound are below all three component thresholds, and interference at large pedestals (approximately one period of the higher frequency) where they exceed all component thresholds. Figure 5 shows that with the addition of components, the compound grating function progressively flattens and approaches the thresholds for the flat-spectrum random-dot pattern over the pedestal range yielding facilitation. 

Over most of this range, the three-component function is at or below the lower envelope of the two-component functions. It dips below the threshold values for the two-component subsets at pedestals proportional to the period of the component not shared by them. Thus, the absence of one of the three components leads to threshold elevation over a pedestal range related to the component’s period. Interference is strong when it comes, but facilitation dominates within regions where both facilitation and interference occur among the two-component combinations. Yet there is no evidence of interference in the data for the 2+4-c/d gratings. Here, thresholds over the substantial range of pedestals tested (a phase angle of 150° of the 180° possible) were similar to those for RDSs, which increased modestly, even as the pedestal size extended beyond Panum’s area. 

Discussion

Disparity increment thresholds are usually described as rising steeply with the distance of the stimulus from the horopter. The steep increase is taken as showing that fine disparity discrimination is a specialization of near-horopter vision. This view is entirely consistent with Marr and Poggio’s (1979) coarse-to-fine model, which achieves multi-scale stereo vision through eye movements that bring the horopter to the stimulus. It is also consistent with the size-disparity correlation, whereby high-frequency stimulus components contribute to disparity discrimination only near the horopter and independently of low-frequency components. 

This view needs to be reevaluated. Our data show that stimulus components of different scales do not contribute independently to the threshold function, their contributions to disparity discriminations are not restricted to particular ranges of disparities, and the disparity increment threshold function is not necessarily particularly steep. 

Spatial disparity thresholds for single-frequency gratings vary reciprocally with frequency; the thresholds display phase-disparity constancy. We found that combining pairs of single-frequency gratings has two interactive effects. First, facilitation: At moderate pedestals, thresholds for compound gratings are lower than thresholds for both component gratings. Second, interference: At large pedestals compound-grating thresholds are higher than both component thresholds. 

Facilitation

At small and moderate pedestals, disparity discrimination thresholds for compound gratings in our experiments are as low or lower than thresholds for the component gratings. Importantly, there is no super-resolution: In no case does the threshold for a compound grating fall below the lowest threshold measured for the components (for related results from occlusion and transparency junctures, see Farell, 1998, 2003, and van Ee, Anderson, & Farid, 2001). This result suggests that the mechanism limiting thresholds for high-frequency components also limits thresholds for compound gratings. These thresholds differ in the pedestals at which they are found, the compound-grating thresholds being displaced toward larger pedestals. This is consistent with a shift of the high-frequency matching range toward larger disparities, as in Figure 1, but doesn’t imply it; other explanations are possible. 

Thresholds for gratings—both single-frequency and multi-frequency gratings—are high overall compared to thresholds for RDSs. The RDS threshold function is as low as that of any of these components and flatter over a larger range of disparities than any of them, yet the threshold functions for the RDSs of Figure 5 are classically exponential (Farell, Li, & McKee, 2004). Grating thresholds systematically approach this shallow exponential function as additional spatial-frequency components are added. Thus, while the ability of humans to resolve disparity differences generally falls off as stimuli move away from the horopter, the components of a multi-scale stimulus do not contribute to this fall-off independently. The components interact, attenuating the threshold elevation at moderate pedestals. For broadband patterns (even those showing an exponential increase in threshold with standing disparity), off-horopter depth discrimination can be quite good, better than that for the individual stimulus components. 

Facilitation was not found in earlier studies, perhaps only because it was not sought in the right place. Facilitation appears in our data over a range of pedestals not systematically examined earlier. Smallman and MacLeod (1997) used component frequencies of 2 and 8 c/d. The smallest pedestal was 4′, a phase disparity of 192°, for their high-frequency component (there was no zero-pedestal condition). Rohaly and Wilson (1993) used frequencies of 3 and 12 c/d for one observer and 2 and 8 c/d for the other. The smallest nonzero pedestal, 2′, corresponds to phase disparities of 144° and 96° for the two high-frequency components. Only these smallest of the standing disparities lie within the range yielding unambiguous evidence of threshold facilitation in our data. Indeed, the only hint of facilitation in these earlier studies is at the smallest pedestal values. 

Facilitation may also be limited to disparities that are consistent with coherent objects, whose components all have equal spatial disparities. If only the lower-frequency component has a nonzero disparity, the threshold for detecting disparity in a compound grating is higher than the threshold for this component grating presented alone (Li & Farell, 2002). And if both components have nonzero disparities that are equal in phase rather than space, compound-grating thresholds are higher than thresholds for either component grating (Li & Farell, 2002). 

Interference

Disparity thresholds for simple gratings rise rapidly when a large pedestal is increased further, above phase disparities of about 60°. For compound gratings, large-pedestal thresholds can exceed the highest of the components thresholds. Such interference has been seen previously in the two-frequency increment thresholds of Smallman and MacLeod (1997), and may have contributed to the steep threshold increases typically seen at large pedestals in previous studies of disparity increments. The scale or spacing of spectral components may be important for this interference effect, because it is not evident in the data for 2+4-c/d gratings. 

It might be supposed that interference occurred when compound gratings weren’t fused. At large disparities, squarewave gratings appear as diplopic, whereas sinewave gratings of the same fundamental frequency are seen as single (Kulikowski, 1978). So, the high frequencies in our compound gratings might have narrowed the fusional range, beyond which diplopia might have elevated thresholds. However, Rohaly and Wilson (1993) found that diplopia thresholds for superimposed D6 patterns, one of high frequency and one of low, were generally as high as the threshold for the low-frequency D6 alone (and in one case actually higher), provided that both patterns had the same disparity. This provision held in our study — all components had the same spatial disparity — and neither diplopia nor depth reversals were noted by our observers. Moreover, our RDSs were outside Panum’s area at the largest pedestals, yet thresholds remained modest. 

RDSs may escape interference because of their low-frequency components, below the 0.5 c/d of the compound gratings, because of their aperiodic spatial structure, or because of the continuity of their spatial-frequency spectra. It is known that the presence of low frequencies, as well as high, contributes to fine stereoacuity for broadband patterns (Westheimer & McKee, 1980). Low-frequency contrast-envelope disparities can extend the disparity range of higher-frequency carrier components (McKee, Verghese, & Farell, 2004; Stelmach & Buckthought, 2003). However, in our study, the low frequencies would be those of the RDS pattern; the envelope’s disparity was fixed at zero. Still, the envelope is a possible source of interference. Specifically, the zero-disparity signal of the gratings’ Gaussian envelope, possibly transduced by a second-order pathway (McKee et al., 2004), could have interacted with the nonzero disparity signal of the low-frequency (0.5 c/d) component of the compound grating. The 2+4-c/d grating could have escaped interactions with the envelope because of its higher-frequency range. However, this leaves unanswered the question of why the large-pedestal thresholds for the 0.5-c/d grating are any less than those for the 0.5+2-, 0.5+4-, and 0.5+2+4-c/d gratings. Whatever the source of this interference effect, thresholds for RDSs and 2+4-c/d gratings appear not to be influenced by it. Therefore, the interference seen in the data for most of the compound gratings is not a general property of disparity discriminations of multi-scale stimuli. 

Disparity discrimination and stimulus appearance

Consider a compound grating with a disparity of 20'. For component frequencies of 0.5 and 2 c/d, the respective phase disparities would be 60° and 240°. If viewed separately, the component gratings would appear on opposite sides of the fixation plane. If viewed simultaneously and processed independently, they would appear as separate transparent surfaces, but this is not what observers saw. Over the full range of pedestal disparities tested, all of our observers reported seeing the 0.5+2-c/d and 0.5+2+4-c/d gratings as cohering in depth.2 Depth coherence has been reported also in earlier studies for other component frequencies with two-octave separations (Rohaly & Wilson, 1993; Li & Farell, 2002). The relative depth of 0.5-c/d and 4-c/d components (a three-octave separation) was harder to judge at the short durations used here, but it was not obviously different from the other cases.3 Any of several processes (not necessarily distinct from one another) could bring grating components into the same apparent depth plane, yielding a perception of coherent depth. These include disparity averaging, depth capture, restriction of the fine-scale fusional range, and a shift in the fine-scale matching range. Not only could these processes result in a coherent-depth percept, but also they could conceivably facilitate disparity discriminations. 

Wilson et al. (1991) and Rohaly and Wilson (1993) showed that a low-frequency stimulus with fixed disparity restricted the disparity range over which a superimposed high-frequency stimulus could be fused, lowering the diplopia threshold. Might a contraction of the fusional range be linked to the facilitation for detecting disparity increments seen in the present data? Apparently not. As noted above, when Wilson et al. (1991) presented high- and low-frequency patterns at the same disparity, diplopia thresholds were at least as high as those for the low-frequency pattern alone. And when the disparities of the patterns were manipulated independently, the fusion range of the high-frequency pattern was limited, but there was no consistent effect on stereoacuity (Rohaly & Wilson, 1993). So, the contraction of the fusional range does not seem to intersect the facilitation of disparity discriminations reported here. 

McKee and Mitchison (1988) binocularly offset a row of evenly spaced dots by one inter-dot period. Except at the row’s ends, each dot in one eye had a zero-disparity match in the other eye. Still, the entire row appeared at the depth given by the disparity at the edges (also see Ramachandran & Cavanagh, 1985; Ramachandran, 1986). Edge offsets also ‘capture’ sinusoidal gratings (McKee et al., 2004). Low-frequency grating disparities might act analogously, capturing higher-frequency gratings. This would give a compound grating a single perceived depth that could extend beyond the high-frequency depth range, much as if the fine-scale matching range had been shifted. Indeed, our 0.5+2-c/d gratings were seen in a single depth plane at all pedestal disparities. However, threshold facilitation appeared over only part of this range. Thresholds for the 0.5+2-c/d grating were low between pedestals of 0′ and 15′ and much higher between pedestals of 30′ and 45′, yet the 2-c/d component had the same modulo-360° phase disparities over both pedestal ranges and appeared in the same depth plane as the 0.5-c/d component over both ranges. Thus depth capture, like restriction of the range of fusion, should be regarded as distinct from the interactions that facilitate disparity discriminations. 

This argument applies as well to disparity disambiguation (Smallman 1995; Mallot, Gillner, & Arndt, 1996; Tsai & Victor, 2003). However, disambiguation, defined broadly, suggests a different way of looking at single-scale disparity increment thresholds and their link to multi-scale thresholds. Suppose the single-scale threshold function is shaped by two distinct sources of internal noise. Let the magnitude of the single-scale disparity increment signal be constant across the ±180° range of pedestal disparities. Let the magnitude of the first of these noise sources be similarly constant and that of the second increase with disparity such that the overall single-scale signal-to-noise ratio falls and thresholds rise as the pedestal nears and exceeds 90°. This second, increasing noise source might be correlated with stimulus appearance, including diplopia and depth ambiguity. Indeed, such perceptual effects could directly degrade task performance by increasing uncertainty about stimulus appearance. If some process (disambiguation) uses the low-frequency content of compound gratings to nullify this second source of noise (e.g., by imposing single vision or a consistent depth interpretation), then the remaining noise would give a constant signal-to-noise ratio and the disparity-discrimination function would be roughly flat across a broad range of pedestals, as seen for compound gratings on pedestals having high-frequency phase disparities between ±180°. By this scenario, the threshold facilitation observed for multi-scale stimuli is best understood as a consequence of eliminating an internal noise that arises when the stimulus occupies a narrow frequency band. 

Disambiguation depends, presumably, on a bias for coherent depth, on the assumption that if the ambiguous disparities of stimulus components are consistent with the components having the same depth, then they probably do have the same depth: for example, 0.5- and 2-c/d gratings with phase disparities of 60° and −120° (= 240°). This is an adaptive bias in environments in which most objects are opaque, and each visual direction has a single spatial disparity associated with it. We have shown elsewhere that a bias toward coherence, as opposed to transparency, is reinforced by object disparities that are relatively small and near horizontal (Farell & Li, 2004). 

Disparity discrimination and perceived depth

Given that the components of a compound grating appeared to cohere in depth, what was that depth? Depth-capture implies that the compound is seen at the depth of the “capturing” component, presumably the one with lowest frequency. The same perceived depth should result from a restriction of the fusional range of the higher-frequency component or a shift in its matching range. By contrast, disparity averaging (Schumer & Ganz, 1979; Parker & Yang, 1989; Rohaly & Wilson, 1994) implies a compromise between the perceived depths of the separate components. As a result, perceived depth could vary non-monotonically with disparity. Reversals would occur as the phase disparity of the higher-frequency component passed through integral multiples of 180°. For purposes of disparity increment detection, a reduced perceived depth at some pedestal might make the pedestal functionally equivalent to one of smaller size. The apparent reduction in depth could, conceivably, account for threshold facilitation, at least over part of the pedestal range. 

To investigate the matter, and to determine what consequences the facilitation of disparity discriminations has for perceived depth, measures of perceived depth were made using bipartite Gabor patterns. The lower half-Gabor contained a compound F + 4F grating (0.5+2 c/d). The upper half-Gabor contained only F, in cyclopean phase with the F component of the compound. The disparity of the compound grating was fixed during a run of trials, while the disparity of the simple grating varied from trial to trial in a constant-stimulus procedure. Two observers judged whether the upper or the lower half of the pattern was more distant. The set of comparison disparities was selected for each observer based on preliminary data to generate a psychometric function, the 50% point giving the disparity of the simple grating that yielded a perceived-depth match with the compound grating. Each of the components of the compound had a contrast of 0.1, as before. The contrast of the F grating was initially also 0.1. Additional data were collected with the F-grating contrast set to 0.2, a value yielding more nearly equal perceived contrasts for the two halves of the Gabor patch. This contrast manipulation turned out to be without effect on perceived depth. 

We measured perceived depth for compound gratings with spatial disparities between 0′ and 20′. It is at a value of 15′ that the phase disparity of the 4F (2 c/d) component equals 180°. It is in this vicinity that we would expect departures from proportionality between grating disparity and perceived depth to occur, if they occur at all. Figure 6 shows the results. On the abscissa are the fixed compound-grating disparities and on the ordinate the comparison simple-grating disparities that yield a perceived-depth match. In all conditions, perceived depths were very nearly equal when disparities were equal. There is no hint of a lessening of perceived depth of the compound grating as the disparity of the higher-frequency component approached or exceeded 18°. If anything, the compound grating is judged as appearing slightly farther than the simple grating at all disparities. Thus the higher-frequency component had no discernable effect on perceived depth over the range of disparities examined here, a result consistent with data of Boothroyd and Blake (1984) and Schor and Wood (1983). 

Figure 6

 

Disparity of comparison grating required to match perceived depth of test grating. The test grating (circles) was a 0.5+2-c/d Gabor patch, each component having a contrast of 0.1. The comparison grating was a 0.5-c/d Gabor patch with the same contrast. Lower and upper abscissas give phase disparities of the 0.5-c/d and 2-c/d test components, respectively. The ordinate gives comparison phase disparity at perceptual depth match. The dotted line has a slope of 1. Square and triangle symbols show data for 0.5-c/d test Gabor patches at contrasts of 0.1 and 0.2, respectively.

Over the same disparity range in which interactions between coarse and fine scales keep compound-grating disparity increment thresholds below single-frequency threshold values, they exert no influence on perceived depth. They change sensitivity without introducing bias. These findings are consistent with a shift in the fine-scale matching range (Marr & Poggio, 1979), implemented neurally (Nishihara, 1984; Quam, 1987). However, a matching-range shift cannot explain the interference that was also observed in the threshold measurements. One could suppose that high frequencies shifted the low-frequency matching range toward smaller disparities, but there is no support for this in data for diplopia thresholds or perceived depths. Alternatively, as discussed earlier, a single narrow-frequency-band stimulus might give rise to noise at large disparities, perhaps associated with diplopia, ambiguous depth percepts, or equivocal stereo matches. The addition of a disambiguating low-frequency stimulus, for which the disparity is a smaller phase angle, could attenuate this noise source and shift threshold disparities to larger pedestal values. Interference could result from the combined high- and low-frequency noise sources at disparities that are large on the scale of the lower-frequency component. This might be testable if external noise could be added selectively to one or another component of a compound stimulus (Pelli & Farell, 1999). 

Acknowledgments

Supported by National Eye Institute Grants EY12286 (BF) and EY06644 (SPM). 

Commercial relationships: None. 

Corresponding author: Bart Farell. 

Email: . 

Address: Institute for Sensory Research, 621 Skytop Road, Syracuse, NY, USA 13244-5290. 

Footnotes

Footnotes

¹  Popple and Findlay (1999) measured disparity increment threshold for a central random-dot disk as a function of the outside diameter of a surrounding random-dot annulus. Both patterns had the same pedestal disparity. Threshold decreased as the annulus became larger. This was interpreted as a vergence-driven cyclopean coarse-to-fine constraint. The effect persisted at durations well below vergence latency (40 ms), though, and scale-independent sources of the effect, including interference from the border of the annulus and the random-dot background, might have been at play.

Footnotes

²  Transparency would have been seen if components with a two-octave difference in frequency also differed from one another in orientation (Farell & Li, 2004).

Footnotes

³  With stimulus presentations of several seconds, the 4-c/d grating appeared to drift from the depth location of the 0.5 c/d to a position near the fixation plane if the compound grating had a positive disparity larger than a few arc minutes. The 4-c/d grating then appeared transparent. An edge-offset row of dots can similarly appear to migrate, from the depth specified by the edge disparity to the fixation plane, during long presentations (McKee et al., 1990). When disparities were negative, however, the 4-c/d grating appeared immediately in front of the 0.5-c/d grating and was not seen to drift. Thus, in both cases, the final phenomenal location of the fine grating was in front of the coarse grating.