Perceptual grouping via binocular disparity: The impact of stereoscopic good continuation

Abstract

Stereoscopic contextual effects are widely reported but are generally discussed in terms of 2-D Gestalt grouping principles, e.g., good continuation or closure. We propose that there are disparity-based grouping operations that are separable from 2-D grouping and instead depend on the distribution of binocular disparity information. Two experiments assess the impact of perceptual grouping via good disparity continuation. First, perceived depth magnitude is reduced for a multidot contour with a smooth disparity gradient compared to the end points in isolation. This reduction is eliminated when disparity jitter is introduced to the intermediate dots. Second, observers showed more efficient visual search for the continuous contour versus the discontinuous version. Therefore, when there is spatial support for interpretation of a slanted object, quantitative depth is reduced, but is rapidly detected in visual search. These results reflect the operation of disparity-based grouping, extending the 2-D principle of good continuation into the third dimension.

Introduction

The founders of the Gestalt movement were aware of the important role of the third dimension (depth) in the perceptual organization of visual space. As early as 1930, Kopfermann used figures whose components were drawn on transparent surfaces to show that the relative location in depth of the fragments was reduced when they were seen as a single closed figure (see Hartmann, 1935). Subsequently Koffka (1935) argued that “three-dimensional shapes are matters of organization in the same way as two-dimensional ones, depending on the same kind of laws” (p. 161). Modern researchers have used a variety of psychophysical paradigms and stimuli to evaluate the effects of 2-D configuration on stereoacuity. For instance, it is well documented that 2-D grouping cues such as convexity, collinearity and similarity can degrade disparity thresholds in amodal completion arrangements (see Hou, Lu, Zhou, & Liu, 2006; Liu, Jacobs, & Basri, 1999; Yin, Kellman, & Shipley, 2000). Others have shown that stereoacuity for a pair of vertical lines is elevated when they are perceived as parts of a closed rectangle (McKee, 1983; Mitchison & Westheimer, 1984; Westheimer, 1979). Implicit connections between features can also disrupt disparity discrimination: Fahle and Westheimer (1988) measured thresholds between two small dots and found that stereo-sensitivity systematically decreased as more dots were added along a linear disparity gradient. Notably, thresholds for two dots were elevated by the presence of a single dot positioned between them. Similarly, Mitchison and Westheimer (1984) reported that stereoacuity for two columns of dots were markedly increased when additional columns were added on each side to form a square lattice. These studies show that the perceptual outcome of connecting or intermediary components significantly affects stereoscopic resolution. 

Recently we used a configuration similar to that employed by McKee (1983) to assess the impact of configuration on suprathreshold depth estimates (Deas & Wilcox, 2014). We showed that quantitative depth estimates are consistently and markedly reduced when vertical lines are perceived as part of a closed figure. Our results cannot be explained by low-level disparity processes such as disparity averaging, inhibition, cue conflict, saliency, or other factors related to crowding and or lateral inhibition. Rather, we show that the degradation of depth percepts demonstrated in these and related experiments are a midlevel phenomenon resulting from figural grouping via closure and/or good continuation. In these experiments, any manipulation that weakened a closed-object interpretation of our stimuli resulted in a systematic release from the depth degradation effect, and consequently enhanced depth estimates (for a similar effect in 2-D crowding experiments, see Manassi, Sayim, & Herzog, 2012). 

Whereas it is clear that closure played a key role in reducing perceived depth from disparity in our stimuli (and those of McKee, 1983, and Westheimer, 1979), the perceptual grouping manipulation necessarily introduced a within-object disparity gradient, specifically a smooth disparity profile. Here we propose that the disparity profile itself contributes to the reduction in perceived depth via a form of good continuation in 3-D. It is well established that depth percepts for stimuli that contain linear disparity gradients are relatively poor. For example, slant perception for isolated surfaces with smoothly changing horizontal disparity is often significantly underestimated and slow to develop (e.g., Gillam, 1968; Gillam, Flagg, & Finlay, 1984; Mitchison & McKee, 1990; Rogers & Cagenello, 1989; van Ee & Erkelens, 1996). One explanation for this phenomenon is that constant disparity gradients provide weak impressions of depth in the absence of disparity contrast, i.e., the stereoscopic system is most sensitive to discontinuities (e.g., Brookes & Stevens, 1989; Gillam et al., 1984; Stevens & Brookes, 1988). In the case of isolated elements or points, this sensitivity results in high resolution depth estimates for isolated elements. However, when elements are perceived as part of a continuous surface, unless the surface contains discontinuities, the stereoscopic signal is weaker. 

Here we evaluate the role of a new grouping principle, which we call “good disparity continuation,” in determining suprathreshold depth percepts. We show that this form of 3-D grouping, like classic Gestalt organizing principles, has both costs and benefits. Using rows of dots, in Experiment 1 we manipulated the number of elements and the smoothness of the disparity gradient, and found that when disparity varied smoothly (exhibiting good disparity continuation) perceived depth was reduced. Based on performance in Experiment 1, we devised a visual search experiment using configurations with and without good disparity continuation. Our results show that good continuation through depth promotes rapid, parallel search. Critically, these effects break down when disparity discontinuities, which disrupt disparity continuation, are introduced. We propose that this disparity-based grouping cue is at least partially responsible for previous reports of degraded slant estimates for stimuli which contain smooth disparity gradients. 

Experiment 1a

Methods

Observers

Eleven observers participated in Experiments 1a and 1b. The sample size was predetermined, and based on similar (small n) experimental designs. Four were experienced stereoscopic observers with excellent stereoacuity (≤40 arcsecs). Seven were undergraduate students with little or no prior experience with psychophysical tasks. These seven students were recruited based on a prescreening assessment with the criteria of achieving at least 40 s of arc on the Randot^TM stereoacuity test. All observers had normal or corrected-to-normal visual acuity. The research was approved by the ethics board at York University and followed the tenets of the Declaration of Helsinki. 

Apparatus

Stimuli were generated using the Psychtoolbox package (Brainard, 1997; Pelli, 1997) for MATLAB on a Mac OS X computer. They were presented on a pair of LCD monitors (Dell U2412M) in a mirror stereoscope arrangement at a viewing distance of 64 cm. The monitor resolution was 1920 × 1200 pixels with a refresh rate of 75 Hz. At this resolution and viewing distance, each pixel subtended 1.45 arcmin of visual angle. The monitors were calibrated and matched prior to testing, and the gamma functions linearized. A chin rest stabilized head position during testing. Each observer's interocular separation was measured with a Richter digital pupil distance meter^TM. 

Depth estimates were made using a purpose-built touch sensitive sensor. A rectilinear SoftPot membrane potentiometer (SpectraSymbol) was mounted on a thin aluminum bar. The sensor strip was 200 mm long and 7 mm wide with a resistance of 10 kOhm and allowed linear measurements with a resolution of approximately 0.2 mm. Responses were read using an analog to digital converter and a 16-bit micro controller. The thumb was positioned against an adjustable rod at one end of the sensor, which was adjusted prior to testing to account for individual thumb size. The recorded voltage was converted to millimeters using a MATLAB script. This measurement technique has previously been validated and shown to be comparable to other types of assessment, such as on-screen rulers (Hartle & Wilcox, 2014). 

Stimuli

The stimulus comprised small (8.7 arcmin diameter) white (59.1 cd/m²) dots arranged in a horizontal row on a midgray background (15.6 cd/m²). At this size, each element's internal disparity gradient was imperceptible. The path length was fixed at 156 arcmin, and we varied the number of elements from two (one at each end) to seven (see Figure 1). The baseline test stimulus was two dots, positioned symmetrically about the midpoint of the display and laterally separated by 156 arcmin. To create the three-, five-, and seven-element conditions, intermediate dots were added between the outer dots (maintaining equal element spacing in a given configuration). The element lateral separations for the two-, three-, five-, and seven-dot configurations were 156, 78, 39, and 26 arcmin, respectively. 

Figure 1

 

Illustration (not to scale) of the stimulus configurations used in Experiment 1a. Observers judged the amount of depth between the two outer dots in each condition. Each row depicts one condition, with systematic increases in the number of elements forming the contour. The first column shows the patterns as a view-from-above, and the second column depicts stereograms of the stimulus configurations. The horizontal distance between the outer elements in each configuration was fixed and, for a given test disparity, the disparity gradient was constant for all path densities.

On each trial, the outer test dots were presented at one of a range of disparities (0°, 0.04°, 0.08°, 0.12°, 0.16°, and 0.2°). One dot had a crossed disparity and the other end had an equal amount of uncrossed disparity, such that the path was symmetric in depth about the zero disparity plane (the total disparity between the outer dots was 0°, 0.08°, 0.16°, 0.24°, 0.32°, and 0.4°, respectively). In the conditions with intermediate elements, these dots were positioned at a disparity determined by a linear extrapolation between the disparities of the two outer elements. The maximum disparity gradient (defined as G = Db/Sb, where Db is the binocular disparity difference between the endpoints and Sb is the binocular dot separation of the endpoints, averaged over both eyes) in our dot arrays was 0.17. At this gradient the stimuli are well within the fusable range, and much lower than the limit of 1 proposed by Burt and Julesz (1980). Note that for each disparity tested, the disparity gradient between the two end elements remained the same regardless of the number of intervening elements. Binocular disparity was introduced by shifting each half image in opposite directions relative to the central fixation point. Pilot testing prior to the main experiments verified that all disparities were within Panum's fusional area. 

Procedure

On each trial observers were asked to indicate the amount of depth they perceived between the two outer test dots. Observers positioned their thumb against the adjustable rest at the base of the sensor, and pressed the side of the nail of their index finger on the sensor to indicate depth magnitude. A small red LED (10.8° below the line of sight to the stimulus) illuminated when sufficient pressure was applied to the sensor strip. Observers were free to adjust their fingers until satisfied with their estimation. When ready, they pressed the spacebar to record their response and move on to the next trial. Between trials, observers repositioned their finger at the base of the sensor. Viewing time was unrestricted, and the experiment took place in a darkened room. 

We measured perceived depth for four conditions: two (baseline), three, five, and seven dots. The experiment consisted of 24 conditions (six disparities × four configurations), with each condition presented 10 times in random order (five left dot crossed and five right crossed). The 240 trials were completed in two blocks in a single session. Prior to testing, observers completed a brief practice session of 30 trials to familiarize themselves with the depth estimation technique. 

Theoretical depth from disparity

To simplify comparison of on-screen angular disparity to observers' depth estimates, we converted the stimulus disparities to theoretical depth in millimeters for each experiment, using the standard formula (see Howard & Rogers, 2012, p. 457). The average of the observers' interocular difference (IOD) was used in each experiment. For Experiment 1, the theoretical depth between the two test lines corresponding to disparities of 0°, 0.08°, 0.16°, 0.24°, 0.32°, and 0.4° were 0, 9.5, 19.0, 28.5, 38.0, and 47.5 mm, respectively (with average IOD = 60.1 mm). 

Results

Figure 2 shows the mean estimated depth for each configuration as a function of the theoretical separation in depth in millimeters. Because observers were told that some stimuli would have zero disparity, this response became stereotyped to the base of the sensor strip and had no associated variance. To avoid biasing the model fits, we excluded the zero-disparity estimates from analyses. 

Figure 2

 

Perceived depth as a function of the depth difference between the outer dot pair for four dot densities with smooth disparity gradient. Paths were composed of two (red circles), three (green squares), five (orange circles) and seven (blue squares) dots. The abscissa shows the theoretical depth, and the ordinate shows the estimated depth. Depth estimates are expressed as the equivalent theoretical disparity that would produce the depth at that viewing distance (see text). The dashed line indicates a gain of one and error bars represent SEM.

As the disparity separating the target pair increased, estimated depth increased linearly for all configurations. However, the amount of depth reported clearly depends on the number of dots in the configuration. Maximum depth is perceived in the two-dot condition, and in this case, closely followed theoretical predictions except at the largest disparity. In comparison, perceived depth is consistently and systematically reduced as more dots are added to the path. Note that the addition of just one dot (three-element configuration) is sufficient to significantly reduce perceived depth of the outer elements and the disruptive effect increases with increasing dot number. 

Statistical analyses confirm these observations. A repeated-measures ANOVA showed main effects of Configuration, F(1, 18) = 32.59, p < 0.001; η² = 0.77, and Disparity F(1, 14) = 83.35, p < 0.001; η² = 0.89. The interaction between Configuration × Disparity was also significant, F(4, 41) = 4.44, p = 0.004; η² = 0.31. Simple effects analyses were used to compare the differences between configurations. These contrasts revealed that perceived depth for all conditions was significantly different at the p = 0.001 level. Depth estimates for the two-dot condition were significantly higher than all other conditions at the p = 0.001 level [contrast with three-dot, F(1, 10) = 21.77, p < 0.001; five-dot, F(1, 10) = 38.54, p < 0.001; seven-dot, F(1, 10) = 42.45, p < 0.001, conditions]. Perceived depth for the three-dot condition was significantly higher than for the five-dot, F(1, 10) = 16.75, p < 0.001, and seven-dot, F(1, 10) = 26.76, p < 0.001, conditions. And the amount of depth estimated for the five-dot condition was greater than the seven-dot condition, F(1, 10) = 20.48, p < 0.001. 

The disparity gradient is necessarily affected by altering the relative disparity of the two end dots. However, for a given test disparity, the disparity gradient between those elements does not change with the addition of intermediate dots, as both the binocular distance (Sb) and the disparity difference (Db) remain unchanged. Despite this consistency, observers perform veridically in the two-dot condition, but disparity is systematically misperceived with the addition of intermediate dots. Therefore we can rule out explanations for the reduction in perceived depth seen here based on loss of fusion due to violation of the disparity gradient limit. Rather, we suggest a higher-level disparity-based grouping mechanism underpins this phenomenon. 

Previously we showed that perceptual grouping via closure reduces the amount of depth seen between parts of a figure (Deas & Wilcox, 2014). In the present study, we propose that the addition of intermediate dots to the isolated dots condition changed the interpretation of the distinct dots or objects to a single path via good continuation. As a result, perceived depth is reduced. It is possible, however, that the addition of the intermediate elements alone interfered with depth estimates, and that this reduction in perceived depth is unrelated to the disparity relationships per se. In part two of this study we evaluate this possibility. 

Experiment 1b

As outlined in Experiment 1a there is a significant reduction in perceived depth from disparity between two dots when intervening elements are added. We propose that this reduction reflects the operation of perceptual grouping via good disparity continuation; in Experiment 1b, we evaluate if the smooth disparity gradient is essential to this phenomenon using the five-dot configuration (shown in Figure 1C) and varying the smoothness of the disparity profile. If good disparity continuation is critical to the degraded depth effect in the preceding experiment, then jittering the disparity of the central elements should eliminate the effect and restore depth magnitude estimates to levels reported for isolated elements. On the other hand, if the reduction in perceived depth seen in Experiment 1a is simply due to the presence of additional elements, the pattern of results should be the same for all conditions. 

Methods

Stimuli

Depth magnitude was measured for a five-dot array with continuous and discontinuous disparity gradients. We assessed two discontinuous disparity profiles, defined by low and high amounts of disparity jitter. The continuous disparity contour was generated as described in Experiment 1a: Intermediate dots were positioned in depth by dividing the disparity between the end points in equal parts. In the discontinuous conditions, the binocular position of each intermediate dot was randomly jittered in depth (in the z axis only) relative to the continuous plane. For ease of explication, disparity jitter is described relative to a linear interpolation in depth between the disparities of the end dots. The jitter step limit was 25% of the total disparity in each direction (crossed, uncrossed) which corresponded to half of the total disparity range in a given run (test disparities: 0.08°, 0.16°, 0.24°, 0.32°, and 0.4°). Figure 3 illustrates the three conditions, the continuous disparity profile, and example profiles for the discontinuous conditions (low and high). To generate low disparity jitter, each dot was repositioned by one “step” on either side of the original position on the continuous plane. The position of each intermediate element was selected pseudorandomly from disparities ranging from ± 25% of the test disparity (Figure 3B). To create high jitter, each dot was repositioned by more than one step, but was constrained to not extend beyond the depth of the end dots. In this instance, the disparity sign of the elements neighboring the outer test dots would be reversed (Figure 3C). In both jitter conditions, element position was determined according to these rules, with two additional constraints: No element was positioned along the original (linear) path, and no dot extended beyond the disparity of the end dots. The disparity gradient limit was never violated within the path. Note that in all conditions, the path dot density was constant (i.e., five identical dots aligned horizontally); the differences in the appearance of the contours arose from disparity alone. 

Figure 3

 

A bird's eye view of the test conditions in Experiment 1b. Observers estimated the depth between outer test dots in horizontal contours of five dots. The dashed line indicates a linear path in depth between the two end dots. Three conditions were assessed, defined by different depth profiles. (A) Continuous disparity change (like that used in Experiment 1a). (B) and (C) depict possible versions of jittered conditions, where dots were repositioned in depth according to the constraints outlined in the text. Solid gray lines represent the maximum displacement in depth (disparity jitter). (A) No jitter, (B) “Low” jitter, (C) “High” jitter.

Observers and procedure

Eleven observers participated in Experiment 1b, all of whom had previously completed Experiment 1a. Observers were asked to assess the amount of depth between the two outer dots as described in Experiment 1a, for the same set of disparities (equally offset 0°, 0.08°, 0.16°, 0.24°, 0.32°, and 0.4°) The three conditions were randomly interleaved for a total of 180 trials (three conditions × six disparities × 10 trials per condition) and were completed in one test session. 

Results

Perceived depth (expressed in mm) for the conditions in Experiment 1b are shown in Figure 4. First, the data obtained in the zero-jitter, five-dot condition and two isolated dots condition replicates the loss of perceived depth seen in Experiment 1a (Figure 2). These data are compared with the jittered disparity profile results also depicted in Figure 4 and show that the amount of depth perceived within a simple dot path is contingent on its disparity profile. In all conditions, the path density is constant, but the differences in disparity relationships modulate the amount of depth perceived in these stimuli. First consider the low jitter condition (depicted in Figure 3B), when dots straddled the line of continuous disparity by one step (less than 25% of the total disparity for a given trial). This change to the stimulus increased the magnitude of depth percepts relative to the continuous disparity condition; even this small amount of disparity jitter was sufficient to “break” the disruptive effect. Displacing the interior elements by even larger increments (high jitter) resulted in a marked overestimation of the depth of the end dots, compared to the theoretical depth prediction. Statistical analyses confirmed these observations. There were significant main effects of both Configuration, F(2, 20) = 28.2, p < 0.001; η² = 0.74 and Disparity, F(1, 15) = 50.87, p < 0.001; η² = 0.84. There was a significant Configuration × Disparity interaction, F(8, 80) = 6.51, p < 0.001; η² = 0.39. Contrasts (using simple effects analyses) revealed that estimates for the continuous-disparity configuration were significantly different from both the small jitter, F(1, 10) = 15.50, p = 0.003; η² = 0.61, and the large jitter configurations, F(1, 10) = 40.12, p < 0.001; η² = 0.80. Furthermore, estimates in the high jitter configuration were significantly different from those obtained in the low jitter condition, F(1, 10) = 19.47, p < 0.001; η² = 0.66. 

Figure 4

 

Perceived depth for a five-dot contour as a function of slant in depth for three disparity profiles: continuous (green circles), discontinuous low jitter (blue squares), and discontinuous high jitter (orange squares). Perceived depth for two isolated dots (red circles) are included for comparison. The abscissa shows the theoretical depth, and the ordinate shows the estimated depth. The disparities are expressed in terms of the corresponding theoretical depth (see text). The dashed line indicates a gain of one, and error bars show SEM (at some points, the symbol size is larger than the error bar).

The separation of two dots in depth is underestimated when intervening elements are added on a linear disparity gradient; in this experiment we show that displacing dots away from the linearly interpolated path in depth results in systematic increases in perceived depth. The effects of these manipulations on perceived depth are not related to local disparity relationships (such as disparity gradient) that are presumed to occur at early stages of visual processing. Because the outer dots remain at a fixed relative disparity for all disparity profiles, the disparity gradient between them does not change when jitter is applied to the intermediate dots. While the application of large amounts of jitter might introduce disparity gradient violations between neighboring intermediate elements, the resultant diplopia should degrade, not enhance, perceived depth (Wilcox & Allison, 2009). In addition, given that other potential local influences, such as conflicting depth cues (perspective), assimilation, or normalization to the fronto-parallel plane should occur in all three configurations, these explanations cannot account for our results. Instead, these data are consistent with our previous work in showing that perceptual grouping, in this case via good disparity continuation, is responsible for the degraded depth percepts shown here. 

It is clear that maximum depth is perceived for the high jitter condition and is overestimated compared to veridical (and the two-dot version). This overestimation may be related to local depth differences between neighboring dots. With large amounts of jitter, the intermediate dots were positioned on or near the disparity of the outer test dots. While observers did not report this, it is possible that in this large jitter condition, the stimulus appeared more like two pseudotransparent planes. If so, observers may have judged the separation in depth of these two planes rather than the outer elements. This may have produced exaggerated depth estimates as it has been shown that larger disparity gradients can enhance perceived depth (Bülthoff, Fahle, & Wegmann, 1991). 

Experiment 2

Experiments 1a and 1b suggest that visual information in a scene is perceptually organized in terms of relative disparity. If so, we would expect that the visual system could exploit this information to aid detection. It has previously been shown that binocular disparity can improve detection of contours in random element displays (Hess, Hayes, & Kingdom, 1997; Uttal, 1983); however, these experiments did not explicitly manipulate good continuation while controlling for the depth of the end points, nor did they measure suprathreshold depth percepts. To avoid potential influences of monocular cues, we used a visual search paradigm similar to that used by Elder and Zucker (1993) to quantify the impact of 3-D Gestalt good continuation. Few studies assessed detection of slant in depth, but a common finding is that stereoscopic slant can be detected preattentively, i.e., it pop-outs out (Epstein & Babler, 1990; Holliday & Braddick, 1991; Sousa, Brenner, & Smeets, 2009). While these studies provide important information regarding slant detection, they do not speak to the role of stereoscopic grouping nor did they evaluate suprathreshold depth percepts. 

In Experiment 2, we use the stimuli described in Experiment 1b to permit direct comparison of the impact of a given disparity profile in these different contexts. We predict that “good disparity continuity” will act as a stimulus feature that is capable of guiding visual search: When it is present, performance will be independent of the number of distractor elements. 

Methods

Observers

Nine of the observers who participated in Experiments 1a and 1b also participated in Experiment 2. Two observers from Experiment 1 (undergraduate students) did not schedule a return visit. 

Apparatus

The apparatus was the same as that described in Experiment 1. 

Stimuli

As shown in Figure 5, the test display contained one target and 5, 9, or 13 distractor stimuli (display sizes of 6, 10, or 14 stimuli). The stimulus array consisted of multiple horizontal paths of five elements on a midgray background (15.6 cd/m²). As in Experiment 1, each circular dot was 8.7 arcmin, in diameter and the separation between the outer dots in each path was fixed at 78′, and each intermediate dot was laterally separated from its neighbor by 19.5′. The elements in a given path were pseudorandomly assigned one of three contrasts (Michelson contrasts of 0%, 29%, and 58%) to prevent 2-D grouping with nearby contours (via proximity and/or similarity). In these experiments the relative disparity of the outer pair of dots was fixed at 0.24°. Recall from the results of Experiment 1b (Figure 4) that at this disparity, observers consistently reported less depth when the stimulus had a smooth linear disparity profile (we will refer to that as “continuous”), but made veridical settings when discontinuities were introduced. We tested two conditions in separate blocks: In one the target had a continuous disparity profile and distractors were jittered in depth (discontinuous), and in the other the target had a discontinuous profile while the distractors were continuous. To create discontinuous paths, the outer dots were positioned in depth the same as the target (the depth range was fixed), but the disparities of the intermediate dots were jittered in depth using the high jitter manipulation described in Experiment 1b. The stimuli were displayed in a 5° × 5° area centered at the midpoint of the display. Each horizontal path was pseudo-randomly positioned within this region, with the constraint that adjacent paths were separated by at least 0.3° vertically and horizontally and had different contrasts. All path locations were refreshed on every trial. 

Figure 5

 

Examples of visual search displays for 6, 10, and 14 items. All contours were composed of five dots with equal lateral separation (images are not to scale). Search displays were presented within a fixed distribution zone centered on the midpoint of the screen.

In a follow-up control experiment, we verified that the search task used here was not influenced by any 2-D interpretation of, or information in, the stimuli. To do so, we presented the continuous target in an array of discontinuous distractors and asked observers to perform the task monocularly (one eye viewing the left image of the stereo-pair with disparity) and binocularly (both eyes viewing at zero disparity). We assessed ten trials for each condition at each array size. The results shown in Figure 6 confirmed that the task was impossible in the absence of binocular disparity. 

Figure 6

 

Follow-up results show search accuracy as a function of number of distractor paths for two monocular viewing conditions: the left-eye image of the disparate stereo pair (orange circles) and binocular viewing with all stimuli at zero-disparity (blue squares). Chance performance is depicted by the gray dashed line. Errors bars show SEM.

Procedure

At the beginning of each trial the search target was shown for 3 s. A fixation cross was then presented in the center of the screen (0.75s) and was replaced by the search array. When the observer located the target, they clicked a mouse button and their response time was recorded. The search display was replaced by a validation display in which each dot stimulus was replaced by a small square (8′ × 8′) centered on that stimulus' midpoint. The observer identified the position of the target by clicking on the appropriate square. If an error was made, the trial was discarded, and subsequently retested at random. The two conditions (continuous vs. discontinuous target depth profile) were run in separate blocks of 60 trials (20 trials for each of three distractor levels). Observers initially completed a practice block of 18 trials with feedback. 

In typical visual search experiments half of the trials contain the target, and the subject indicates whether it is present or absent. As described above, we used a modified visual search paradigm (after Elder & Zucker, 1993) in which the target is always present. This paradigm is preferable because it yields relatively low error rates (total average of 2.1% and were < 4.5% for all observers), is efficient and avoids negative biases introduced by long search times on difficult trials (see Elder and Zucker, 1993, for a more complete explanation). 

Results

Reaction time data are shown in Figure 7A. The results show that the targets with good disparity continuation were more readily detected (among disparity-jittered paths) than targets containing depth jitter (among continuous disparity paths). Whereas search speed for the discontinuous target depends strongly on the number of stimuli in the display (slope = 0.81 s/item, intercept = 1.1 s), search for the continuous target appears to be independent of distractor number (slope = 0.095 s/item, intercept = 2.36 s). A two-way ANOVA confirmed that there is a significant effect of disparity profile (continuous vs. discontinuous), F(1, 48) = 42.13, p < 0.001; η² = 0.47. In the continuous disparity target condition, there is no significant effect of the number of distractors, F(2, 24) = 0.58, p > 0.250, confirming that distractor number has no effect on search speed. However, there is a significant effect of distractor number in the discontinuous target condition, F(2, 24) = 4.80, p = 0.018. 

Figure 7

 

Search results (reaction time) for Experiment 2. (A) Five-dot contours with a continuous (blue squares) or discontinuous (red circles) depth profile. (B) Three-dot contours with a continuous depth profile (orange circles) or discontinuous profile (green squares), with the five-dot results replotted for comparison. Error bars show SEM (at some points, the symbol size is larger than the error bar).

The results shown in Figure 7A demonstrate that the visual system can take advantage of good disparity continuation to find a target among distractors. In Experiment 1a we report that perceived depth for the end dots can be enhanced (the effect of grouping disrupted) by reducing the number of dots in the path from 5 to 3. If the advantage provided in the visual search task is also tied to the effects of grouping, then a reduction in element number should produce corresponding reductions in the difference between search performance in the continuous and discontinuous target conditions described here. To evaluate this prediction, we repeated Experiment 2 with the same observers using three-dot target and distractor paths. The results are shown in Figure 7B (along with the five-dot results) and show that search efficiency deteriorates relative to the five-dot contour, and increasing the number of distractor paths does slow search times (slope = 0.38 s/item, intercept = 1.62 s). Note that while there is an effect of distractors in the three-element continuous condition, the impact is not as severe as in the five-element jittered target condition. This partial effect is consistent with the fact that perceived depth for the three-dot continuous path (shown in Figure 2) is not fully restored to veridical. On the other hand, performance for the three-dot discontinuous path (slope = 0.64 s/item, intercept = 2.58 s) was similar to the five-dot, F(1, 48) = 0.035, p > 0.250. This suggests that in the absence of good disparity continuation, performance is consistently poor regardless of the 2-D spatial support. 

The results presented here demonstrate a clear search asymmetry. Detection of a path with a linear disparity gradient embedded in distractors with jittered disparity was consistently more efficient than when the jittered path formed the target. This asymmetry shows that the disparity profile (and not the presence or range of disparities) is the critical factor. By isolating and modulating the disparity cue, we were able to verify that stereopsis alone does not benefit search; rather, the local disparity relationships between path components are important. Observers did not simply report the “odd-one-out,” nor were they able to perform the task by identifying the path with the unique depth profile; instead, good continuation through depth is the critical feature. However, this type of grouping is dependent on sufficient spatial support, as a continuous path with lower dot density (three dots) was not as well detected. This is consistent with findings that higher dot density results in stronger perceptual grouping along a contour (e.g., Kovács, 1996; Hess & Field, 1999; Uttal, 1983), and suggests that a certain amount of spatial support (via 2-D cues) is necessary to initially define a contour. Note that 2-D cues alone (such as collinearity, or spacing) were not sufficient to support detection for either type of target (Figure 6). Rather, the additional cues provided by stereopsis—specifically, good continuation in depth—mediate visual search. 

General discussion

We propose that the classic Gestalt principle of good continuation in 2-D has a disparity-based 3-D counterpart. This disparity-based grouping cue negatively influences perceived depth (Experiment 1a and b) but enhances the detectability of similar stimuli (Experiment 2). In Experiment 1a we report that the addition of intermediate dots along a linear disparity gradient between two isolated elements results in a systematic reduction in relative depth percepts. Importantly, the disruptive effect of intermediate dots critically depends on the presence of a continuous transition through depth: Displacement of the intermediate dots in depth to create discontinuities was sufficient to break the good disparity continuation, and depth estimates returned to their previous veridical levels (Experiment 1b). It is important to note that in Experiment 1b, the spatial support for the contours via 2-D cues such as good continuation was stable across conditions; thus, it was the differences in disparity relationships that modulated the perceived depth in these stimuli. We conclude that some minimum amount of 2-D support is necessary, but not sufficient, to generate the degraded depth effect illustrated here. Our results also allow us to rule out explanations based on low-level factors such as cue conflict, disparity gradient limits, and redefinition of the fronto-parallel plane. 

In our second experiment, we confirmed our hypothesis that the visual system can capitalize on good continuation in depth to enhance performance in a visual search task. A path of dots that traverses depth smoothly along a linear trajectory is detected more rapidly than a target that has sharp disparity discontinuities. Our stimuli were deliberately designed to eliminate all other cues to the target location; search was virtually impossible when viewed monocularly or binocularly at zero disparity. Taken together, our results suggest that the visual system can take advantage of good continuation though depth to detect contours, but that this advantage comes at the cost of perceived depth magnitude. This disparity-based grouping is likely governed by a number of constraints which are the subject of ongoing investigation. 

The observed benefits of perceptual grouping via disparity continuation are relatively straightforward and are similar to the benefits provided by classic Gestalt cues (e.g., Elder & Zucker, 1993; Field, Hayes, & Hess, 1993; Hess & Field, 1999; Kellman & Shipley, 1991; Kovács, 1996; Kovács & Julesz, 1993). However, the explanation for the observed reduction of suprathreshold depth percepts when elements are grouped is less obvious. In this and our previous series of experiments (Deas & Wilcox, 2014), we reported consistent reductions in perceived depth due to configural factors. In related work, other researchers have reported that stereoscopic thresholds are elevated when formerly isolated components form an object (Fahle & Westheimer, 1988; Hou et al., 2006; Liu et al., 1999; McKee, 1983; Mitchison & Westheimer, 1984; Westheimer, 1979; Yin et al., 2000). Despite a number of studies showing that thresholds are elevated substantially under conditions that connect isolated elements, it is not known how these conditions affect the perception of suprathreshold versions of the stimuli. One prediction would be that under conditions yielding high stereoacuity (small JND), a given suprathreshold disparity would produce greater magnitude estimation than when stereoacuity is low. However, studies have shown that there is not always a direct translation, that is, not all conditions that increase the stereoscopic threshold result in reduced depth of targets with suprathreshold disparities (Bedell, Gantz, & Jackson, 2012; Patel, Bedell, Tsang, & Ukwade, 2009). 

The effects of configuration on stereopsis at detection threshold have most commonly been explained by a form of disparity pooling or averaging (McKee, 1983; Mitchison & Westheimer, 1984; Vreven, McKee, & Verghese, 2002). While the neural processing underlying such disparity-based pooling is typically unspecified, a simple hierarchical model is often assumed. That is, binocular neurons in later visual processing areas extract more global shape information by pooling information from disparity selective neurons in V1. Disparity pooling or averaging may also be responsible for the results presented here and in our previous experiments (Deas & Wilcox, 2014). However, our results are not consistent with simple feed-forward hierarchical processing models. That is, we have shown that 2-D and 3-D grouping cues play a critical role in modulating the observed reduction in perceived depth. Instead, our results are more consistent with current models of cortical processing that support recurrent feedback between mid- and low-level processing (among others, see Deco & Lee, 2002; Desimone & Duncan, 1995; Lee & Mumford, 2003). As outlined by Markov and Kennedy (2013), there is compelling physiological and computational evidence for the existence, and importance, of both feed-forward and rapid feed-back networks in the visual system (Markov et al., 2014; Samonds, Potetz, Tyler, & Lee, 2013). To account for our results as well as configuration-dependent threshold elevation (Fahle & Westheimer, 1988; McKee, 1983; Mitchison & Westheimer, 1984; Westheimer, 1979) disparity pooling must be constrained by feedback which provides an object or surface-based representation of the stimulus. Importantly, as a result of this object-based pooling, the visual system sacrifices the precision and accuracy afforded by the relatively smaller receptive field sizes in earlier visual areas. It is likely that the disparity-based depth discontinuities are initially encoded in area V2 (von der Heydt, Zhou, & Friedman, 2000) but are subject to pooling based on representations of surfaces in a variety of extrastriate areas including IT, MT, and CIP (Hegde & Van Essen, 2005; Janssen, Vogels, & Orban, 1999, 2000; Nguyenkim & DeAngelis, 2003; Rosenberg, Cowan, & Angelaki, 2013; Verhoef, Vogels, & Janssen, 2010). 

The question remains why this disparity pooling, and resultant reduction in perceived depth, occurs. One possibility is that this pooling reflects smoothing operations which help resolve the correspondence problem (Marr, 1982; Marr & Poggio, 1976, 1979). Since it was proposed by Marr and Poggio, some form of “smoothness constraint” has been implemented in most models of stereopsis (among others see Lehky & Sejnowski, 1990; Qian & Zhu, 1997; Yuille, Geiger, & Bülthoff, 1991). The averaging or smoothing of disparities within a local region biases matches towards those resulting in constant disparity and therefore biases depth towards the fronto-parallel plane (Goutcher & Mamassian, 2005). To retain sensitivity to surface discontinuities, most models of stereopsis have relied on luminance or texture-defined edges to constrain disparity-based smoothing operations. Others have implemented constraints based on the rate of change of disparity to capitalize on abrupt changes in the disparity gradient that often occurs at object boundaries (Pollard, Mayhew, & Frisby, 1985; Prazdny, 1985; Yuille et al., 1991). The relationship between the disparity gradient and the degree of disparity pooling was assessed psychophysically by Bülthoff et al., (1991). In their experiments, estimates of the separation in depth of two adjacent elements decreased as their disparity gradient increased. Bülthoff and colleagues attributed this result to their disparity gradient manipulation. However, when the two targets were dissimilar, there was no disruption in perceived depth. Yuille et al. (1991) argued that this pattern of results reflects disparity-smoothing operations which are determined by the degree of matching ambiguity. They argue that matching ambiguity between similar elements forces the visual system to rely on a priori assumptions, such as smoothness, to solve the correspondence problem. As matching ambiguity increases, there is a corresponding increase in disparity smoothing. However, when neighboring elements are sufficiently different, for instance a line and a dot, the matching ambiguity is reduced and less smoothing (pooling) is required. This may be true; however, our results show that the amount of perceived depth between neighboring elements can be modulated without increasing matching ambiguity. 

It is possible that while matching ambiguity contributes to reduced depth percepts under some conditions, a more significant constraint on smoothing operations may be the salience of the object representation. In the case of Bülthoff et al.'s (1991) results, manipulation of the disparity gradient by changing element proximity may have also influenced the degree to which the neighboring stimuli were perceptually grouped as a single object. This interpretation is supported by the fact that the degraded depth percepts were eliminated when differently shaped elements were used. Similarly, we have shown that depth percepts can be restored by changing the perceived object cohesion with no change in disparity gradient (Deas & Wilcox, 2014). Moreover, in the experiments reported here, we were able to restore depth magnitude estimates by adding disparity jitter, a manipulation that should have exacerbated the correspondence problem and produced degraded depth estimates. 

In sum, we propose that object-based disparity smoothing operations are responsible for the degraded depth percepts shown reported here. A modified version of Yuille et al.'s (1991) computational model could account for our results, if the degree of object-based disparity smoothing were contingent on the strength of the object interpretation. Yuille and colleagues do not rule out such a contingency, but assume that an edge representation has been performed prior to the stereoscopic surface computations. We argue that, as outlined by Lee and Mumford (2003), the object inference from extrastriate cortical areas, which incorporates perceptual grouping information, is used to constrain and modulate this disparity smoothing. The result of this inference-based processing is reduced depth percepts for, and enhanced detectability of, perceptually coherent objects. 

Our research provides compelling evidence that disparity-based grouping has a significant impact on the suprathreshold appearance of objects and surfaces. In particular, the effect of smooth disparity gradients on perceived depth is an important consideration given that the stimulus configuration used to study stereoscopic processing could unwittingly introduce this bias, and distort subsequent results. Changes to the configuration that reduce the disparity-based grouping, might then be misattributed or over estimated. These results are likely most relevant to studies of perceived surface slant. It has been widely reported that observers consistently underestimate the slant of surfaces in depth defined by disparity (e.g., Gillam, 1968; Mitchison & McKee, 1990; Rogers & Cagenello, 1989; Stevens & Brookes, 1988; van Ee & Erkelens, 1996). Explanations for this phenomenon typically refer to the presence of depth cue conflicts (perspective, accommodation) or the inherent sensitivity of the stereoscopic system to depth discontinuities (Brookes & Stevens, 1989; Gillam et al., 1984). While more study is required, we propose that in existing studies the smooth distribution of disparities combines with 2-D figural grouping cues to support 3-D grouping via good continuation. The result, as shown here, is degraded depth percepts. 

Acknowledgments

This research was supported by NSERC funding to LM Wilcox and an Ontario Trillium award to LM Deas. 

Commercial relationships: none 

Corresponding author: Lesley M. Deas. 

Email: [e-mail]. 

Address: Centre for Vision Research, York University, Toronto, ON, Canada.