Collinear facilitation over space and depth

Abstract

The detection threshold of a Gabor target can be reduced by the presence of collinear flanking Gabors but is disrupted when the target and the flankers have different disparity. Here, we further investigated whether it is the depth or surface difference between the target and the flanker that causes the abolition of collinear facilitation. The target and the flankers were 1.6 cycle per degree vertical Gabor patches with a separation of three wavelength units between them. There were six viewing conditions: target and flankers were set (A) in the same frontoparallel plane in a collinear configuration, (B) at different disparities but embedded in the same slanted plane, (C) at different disparities in different frontoparallel planes (flankers occupied at the same depth), (D) at different disparities in different frontoparallel planes (flankers occupied at different depth), (E) in the same frontoparallel plane in a noncollinear configuration, and (F) at the same disparity but locally slanted. We measured the target contrast detection threshold with and without the flankers present with a temporal 2AFC paradigm with the Ψ staircase method. Strong collinear facilitation was observed when the target and the flankers were either in the same frontoparallel plane or embedded in the same slanted surface even though the target and the flankers were at different disparities. Our results suggest that it is the difference in surface assignment, not the difference in disparity per se, that causes the disruption of collinear facilitation.

Introduction

The fundamental problem of binocular vision is to solve the correspondence problem (Julesz, 1971), to correctly match an image element in the left eye to the corresponding image element in the right eye while excluding possible false matches. Several constraints that allow the visual system to solve the correspondence problem (Marr, 1982; Marr & Poggio, 1976; Pollard, Mayhew, & Frisby, 1985; Tyler, 1973, 1974, 1975, 1991) have been proposed. There is a uniqueness constraint that every point can only be at only one depth. This is implemented as the mutual inhibition among units tuned to different disparity. In addition, there is a continuity constraint, that the change of depth across space is constrained to be smooth. This is implemented as the mutual facilitation among units tuned to similar disparities. The disparity gradient constraint discovered psychophysically by Tyler (1973) was implemented computationally by Pollard et al. (1985) in the form of facilitation between neighboring units that occurs as long as the ratio between the depth and distance is less than 1. 

While such cooperation–competition algorithms are quite successful in depth computation, few studies empirically test the interaction between units tuned to different locations and depths. There are studies measuring the interaction across depth at the same location or across space at the same depth. For the former, Tyler and Kontsevich (2005) measured masking effects on elliptical Gaussian target detection. They reported that the masking effect depended strongly on disparity. In the frontoparallel plane, the masking effect extended about 30 arcmin on either side of the target and increased to a maximum of about 60 arcmin when the disparity between the target and the mask was about 40 arcmin and the effect disappeared when the disparity of the mask was more than 1 degree from the target. However, these authors focused on lateral interactions in the direction of the short (x) axis of the elliptical Gaussian targets and did not examine disparity interactions in the remainder of the space around the targets, including the direction of the long (y) axis. 

In the frontoparallel plane, interaction between stimuli at different locations is well known. The detection threshold of a Gabor target can be reduced by the presence of other Gabor patches nearby (Polat & Sagi, 1993, 1994). A similar effect was also observed with line segments (Wehrhaha & Dresp, 1998). Further studies investigated the relationship between target and flankers for different stimulus parameters such as spatial frequency, orientation, phase, temporal onset, chromaticity, image statistics, location of the flankers, distance between target and flankers, etc. (Cass & Spehar, 2005; Chen & Tyler, 2001, 2002, 2008; Huang, Hess, & Dakin, 2006; Huang, Mullen & Hess, 2007; Polat, 2009; Polat & Sagi, 1993, 1994, 2006; Solomon, Watson, & Morgan, 1999; Zenger-Landolt & Koch, 2001). It has been shown that the strongest facilitation occurs when the flankers were iso-oriented (Chen & Tyler, 2002; Polat & Sagi, 1993) and in phase (Solomon et al., 1999) with the target and were placed at locations three wavelength units away from the target at the collinear direction (Chen & Tyler, 2008; Polat & Sagi, 1993). The facilitation gradually reduced as the image parameters become less optimal (Chen & Tyler, 2002, 2008; Polat & Sagi, 1993). 

Here, we are interested in collinear facilitation across depth. When the flanker and the target are put at different depths, the interaction between them may involve both facilitation from collinearity and suppression from disparity computation. Huang et al. (2006) showed that collinear facilitation was disrupted when the target and the flanker were at different depth and argued that collinear facilitation was a purely monocular phenomenon. However, Huang et al. presented the target and the flankers in different frontoparallel planes. That is, the target and the flanker were not only at different disparities but also on different surfaces. Hence, it is not clear which factor causes the abolition of collinear facilitation. In this study, we further investigated how these two factors, disparity and surface context, influence collinear facilitation by placing targets and flankers not only at different disparities but also either in the same slanted plane or in different planes defined by the local disparity structure of the stimuli. 

The experimental design for this study was motivated by the theory (Tyler, 2005; Tyler & Kontsevich, 1995) that surface representation is a core principle of perceptual organization governing the interpretation of 3D images. According to this theory, facilitatory interactions occur only between targets that lie within the same perceived surface manifold (i.e., the array of flat and curved surfaces). A variety of disparity, Gestalt, and other forms of perceptual organization contribute to the perception of surface manifolds as extending across objects in the world, but the theory is that the net result of these cues is the perception of surfaces and that perceptual facilitation is restricted to operating within any such perceived surface manifold, regardless of its depth configuration. If conditions arise to block the continuity of a perceived surface, facilitation will not operate across the discontinuity. 

It should be clear that this theory is not tested by previous studies of frontoparallel stimuli, since they always lie within the same perceived surface, a requirement that applies both in monocular and binocular viewing. What those studies have shown is that there is an additional constraint, the collinearity constraint, that facilitatory interactions are largely restricted to collinear targets within the surface; such interactions are thus subject to the dual (orthogonal) requirements that the targets need to be both collinear and coplanar for facilitation to occur. (Another way to formulate this constraint is that facilitation is restricted to extend along one-dimensional manifolds in perceptual 3D space.) Previous studies of the effects of disparity on collinear facilitation have disrupted the surface continuity as they varied disparity, so it is not clear which factor was responsible for the disruption. Thus, our experiment is designed to vary both disparity and surface continuity as independent factors, in order to test the theory that the perceived surface continuity is the relevant factor controlling facilitatory interactions. 

Methods

Apparatus

The stimuli were presented on a CTX CRT monitor (38 × 26.5 cm), which were driven by a Macintosh computer running the OS 9 operating system. A Radeon graphic card was used to provide 10-bit DAC depth for each gun. The resolution of the monitor was 1920 × 1440 pixels with 60-Hz refresh rate and the mean luminance was 25.6 cd/m². At the viewing distance of 112 cm, the size of each pixel was 0.5′. The computer program for experimental control was written in Matlab with Psychophysics Toolbox(Brainard, 1997). The input voltage–output intensity function for the monitor was measured by a LightMouse photometer (Tyler & McBride, 1997) and the information was used to compute the linear look-up table to correct the nonlinear properties of the monitor. Participants viewed the display through a four-mirror stereoscope system. 

Stimuli

The stimuli were Gabor patches defined by the following equation: 

$\cos (2\pi x/T-\rho )=\cos (2\pi x/T-\theta )$

(1)

where L ₀ was the mean luminance, c was the contrast of the Gabor defined as (L _max − L _mean)/L _mean, T was the period of the carrier, σ was the standard deviation of the Gaussian envelope, and θ was the phase of the stimuli with respect to the center of the Gaussian of the window. At the viewing distance of 112 cm, the carrier spatial frequency was 1.6 cycles/degree with 0-degree phase angle. The space constant of the Gaussian envelope, σ, was 0.47 degree, and consequently, the bandwidth of the Gabor was about 0.72 octave. The size of the Gabor patch was set at 4σ to reduce the abutted edges. The contrast of the flankers was set to 0.5 and the target–flanker separation was set to 3 wavelength units (i.e., 1.88 deg from center to center) and the target was presented to the fovea. To ensure that there was sufficient contrast resolution for the low-contrast target presentation while high-contrast flanks were presented, the 8-bit luminance depth (256 steps) was divided into two parts, giving 62 steps for the flanks and 192 steps for the target. Each image pair contained two black/white vertical lines (from top to bottom of the whole screen) 5 degrees from the target center to aid peripheral fusion and to maintain the zero disparity planes. 

The contrast detection of a central Gabor target was measured under different three-dimensional (3D) configurations. The depth in an image was provided by the disparity in the left and right eye images. In order to make comparison across different conditions easier, the image for the dominant eye was always the collinear set of target and flankers and the image for the nondominant eye contained images with disparity manipulation (if not stated otherwise). 

We used the following 3D configurations in our experiments: (A) Frontal-collinear: The target and the flankers were set in the same frontoparallel plane by presenting the identical stimuli to both eyes. (B) Slant: The target and the flankers were at different disparities but perceived as on the same slanted plane. Assuming that the observer's dominant eye was the left eye, the orientation of each Gabor patch in the right eye image was rotated clockwise by 5 degrees; the upper flanker was shifted 16′ visual angle to the left, and the lower flanker was shifted 16′ to the right. If the observer was right eye dominant, the left eye image was constructed in the same way but in the opposite directions. (C) Shelf: The two flankers were in the same plane that was different from the target plane. This was achieved by shifting the position of the flankers 16′ to the right or left (depending on eye dominance). (D) Staircase: The target and the flankers were placed in different frontoparallel planes with one flanker in front of and the other behind the target. This was constructed with the same disparities as the Slant configuration except that the Gabors were always vertically oriented. (E) Frontal-noncollinear: The target and flankers were in the same frontoparallel plane, but all of the Gabor elements were oriented 5 degrees counterclockwise. (F) Sawtooth: While the centers of the target and the flankers all had the same disparity, they were each locally slanted as in (A) so as to appear in different parallel planes. To form this configuration, the orientation of Gabor elements was rotated 2.5 degrees clockwise in one eye and counterclockwise in the other eye (Figure 1). 

Figure 1

 

Stimulus configurations. The Gabor elements in each panel are the stimuli pairs presented to the left and right eyes. The black lines represent the perceived depth and slant for each Gabor element. The white lines denote how disparity of Gabor elements was changed in Experiment 2. (A) Frontal-collinear: The target and the flankers were set in the same frontoparallel plane by presenting the identical stimuli to both eyes. (B) Slant: The target and the flankers were at different disparities but perceived as on the same slanted plane. (C) Shelf: The two flankers were in the same plane that was different from the target plane. (D) Staircase: The target and the flankers were placed in different frontoparallel planes with one flanker in front of and the other behind the target. (E) Frontal-noncollinear: The target and flankers were in the same frontoparallel plane, but all of the Gabor elements were oriented 5 degrees counterclockwise. (F) Sawtooth: While the centers of the target and the flankers all had the same disparity, they were each locally slanted as in (A) so as to appear in different parallel planes. All examples above assume a left eye dominance.

For a control, we also had the corresponding monocular viewing conditions with the nondominant eye stimuli from Configurations (A) to (D) in the experiment. The purpose of these monocular conditions was to assess the effect of two-dimensional (2D) stimulus change in our data. 

In Experiment 2, we were interested in the effect of disparity size on flanker effect. We thus manipulated the disparity between the target and the flanker in the two eyes for the (B), (C), and (D) configurations from 0, 0.5, 2, 4, 8 to 16′. The orientation of Gabor elements in (B) was also changed accordingly. 

Procedures

A temporal two-alternative forced-choice (2AFC) paradigm with Ψ staircase procedure was used to measure the detection threshold of the Gabor patches with or without the presence of the flanking Gabors. The participants were asked to fixate the center of the screen and determined which interval contains the center Gabor patches. The pairs of vertical lines, which aid peripheral fusion, were present throughout the whole experiment. Subjects were pre-cued to trial onset by an audible tone and by a fixation point, which was an inverted T for one eye and upright T for the other eye. This was used to help subject maintain fixation at the horopter and to check the alignment between the two eyes. The fixation lines were presented for 200 ms and then disappeared. After a 200-ms delay, the stimulus was presented for 500 ms (temporal square pulse), then followed by a 1000-ms interstimulus interval (ISI, consisting of homogeneous mean luminance field) and the second interval. Auditory feedback was provided to the observers after they made the response in the given trial. 

In Experiment 2, in addition to the contrast detection to the central Gabor target, we also measured the depth discrimination performance. In one interval, the target and the flankers were in the same frontoparallel plane, while in the other interval, they were not. We thus measured the percentage correct for the observers' judgment whether or not the target and the flankers were at the same depth. 

Participants

Three naive subjects participated in the experiments. All had normal or corrected-to-normal vision. We confirmed that all observers can perceive our 3D configuration with 15′ disparity. Before each stimulus configuration, they were asked to confirm that they perceived the appropriate depth perception. 

Results

Experiment 1: Configuration effects

Figure 2 shows the flanker facilitation effect under various 3D configurations. The facilitation effect was defined as the threshold measured without flankers minus that measured with flankers of the same test condition. The thresholds are presented in dB unit, which is 20 times the log₁₀ of the linear contrast. The bars represent the data averaged across three observers and the error bars indicate the standard error. The collinear effect (5.5 dB or about an 88% increase in sensitivity) was most pronounced in the Frontal-collinear configuration (A). This facilitation is at the higher end of the collinear flanker effects reported in the literature (cf. Chen & Tyler, 2001). The flanker effect was slightly reduced in the other frontoparallel configuration (E), in which the orientation of Gabor elements were 5 degrees away from the collinear axis, even though the difference is not statistically significant (p = 0.13). Such result is expected. It was reported that the flanker effect is tuned to the difference in orientation (Chen & Tyler, 2002) and position (Chen & Tyler, 2008) between the target and flankers. Hence, the 5-degree shift from the collinear axis in this Frontal-noncollinear configuration (E) should produce a flanker effect that is slightly weaker than in the Frontal-collinear configuration (A). 

Figure 2

 

Configuration effect on collinear facilitation under binocular viewing. (A) Frontal-collinear. (B) Slant. (C) Shelf. (D) Staircase. (E) Frontal-noncollinear. (F) Sawtooth. The “*” denotes statistically significant difference from the Frontal-collinear condition at p < 0.05 and “**” at p < 0.01.

The flanker effect on a slanted surface (Slant configuration (B)) was similar to that in the frontoparallel surface. That is, a pitch of the surface had little effect on flanker effect even though the flankers and the target were at different disparities. Such a slant-invariant context effect has also been reported in line integration (Uttal, 1978) and symmetry detection (Sio & Chen, 2011). The flanker effect, however, was much reduced when the target and the flankers were not coplanar. Such a reduction of flanker effect was most pronounced in the Shelf configuration (C), in which the target was presented in front of the flankers, and the Sawtooth configuration (F), in which the target and the flankers were presented in parallel planes. Notice that, in the Sawtooth configuration, the target and the flankers were at the same (zero) mean disparity but individually slanted in depth. The Staircase configuration (D) showed a weaker but significant reduction of the flanker effect. 

To examine how our results can be explained by a change in the 2D configuration of the stimuli, we also measured the flanker effect monocularly with the nondominant eye stimuli from Configurations (A) to (D) (the dominant eye stimuli were all identical for those four configurations). Figure 3 shows the result. All configurations tested showed a facilitation effect. The facilitation in Configuration (A) was slightly greater than the other configurations, but the difference was not statistically significant as a repeated measures ANOVA also showed no configuration effect (F(3,6) = 0.67, p = 0.60 > 0.05). This effect is expected. Configuration (B) was basically a 5° rotated collinear condition. In Configurations (C) and (D), the flankers were shifted in position relative to the target, but the shift (16′) was quite small when compared with the size of the stimuli, implying a phase difference of only 60° between the target and the flankers resulting from the position shift. Note, moreover, that collinear facilitation is less sensitive to the phase difference between the target and the flankers as the facilitation is reported even when the target and the flanker were in opposite phase (Solomon et al., 1999). Hence, since there was no difference in facilitation across the monocular viewing conditions, the reduction in facilitation that we observed in the noncoplanar conditions of Figure 2 cannot be explained by the differences in the monocular stimuli. 

Figure 3

 

Configuration effect on collinear facilitation under monocular viewing. (A) Frontal-collinear. (B) Slant. (C) Shelf. (D) Staircase. Note the lack of effect of the configurations when viewed monocularly.

Experiment 2: Disparity effect

There may be a concern that our configuration effect, discussed above, could be due to a difference in depth discriminability among those configurations. That is, the observer might be more sensitive to the depth difference in the Shelf than in the Slant condition. Therefore, we measured the flanker effect and depth discriminability between the target and the flankers at different disparities. Hence, if there is a discriminability difference among configurations, this information would allow us to compare flanker effects across configurations with matched discriminability. 

It turns out, as shown in Figure 4a, that the depth discriminability did not differ among the 3D configurations. For all configurations tested, depth discriminability (measured as d′) increased with disparity beyond 1 arcmin. On the other hand, as shown in Figure 4b, the facilitation decreased with the increase of the disparity for the Shelf (C) and Staircase (D) configurations. That is, the disparity effect was similar for these two noncoplanar configurations. On the other hand, the facilitation for the Slant configuration (B) showed no significant variation with disparity. The results suggests a high correlation (−0.89) between discriminability and the strength of collinear facilitation in Configurations (C) and (D). That is, the disparity effect on flanker effect was higher for the noncoplanar than coplanar configurations. Such effect was not found in the Slant configuration. 

Figure 4

 

The disparity effect. (a) Discriminability, measured as d′ as a function of disparity. (b) The strength of collinear facilitation as a function of disparity.

Discussion

In this paper, we have shown that flanker facilitation for detecting Gabor targets occurred when the target and the flanker were coplanar, regardless of whether they were in the same frontoparallel plane or on a slanted surface. The facilitation deteriorated when the target and the flanker were not coplanar. Such deterioration even occurred in the Sawtooth configuration in which the target and the flankers were at the same distance from the observer but were locally slanted. These results cannot be explained by a difference in the 2D stimulus arrangement as we found no difference in the flanker effects when both coplanar and noncoplanar configurations were viewed monocularly. Neither can they be explained by a difference in depth discriminability among those configurations. We found no evidence that the observers had more difficulty in perceiving the depth difference in the noncoplanar than in the coplanar configurations. Hence, our result cannot be explained by depth discrimination per se. 

It should be mentioned that binocular summation played a role in the detection of the Gabor target. Without the flankers, the target detection threshold in the binocular viewing conditions, averaged across observers, was −4.4 dB or 40% lower than the threshold in the monocular viewing conditions. The facilitation effect per se, however, showed no binocular summation effect. As was shown in Figures 2 and 3, the facilitation in the frontoparallel plane was 5.5 dB for the binocular viewing condition but 7.1 dB for the monocular viewing condition. That is, the facilitation effect in the binocular condition was no greater than (i.e., not significantly different from) that in the monocular condition. Hence, our result cannot be attributed to binocular summation. 

In Marr and Poggio's (1976) model of stereopsis computation, it was proposed that the units tuned to different depth at the same location would inhibit each other while neighboring units tuned to the same depth should facilitate each other. They, however, made no assertion about the combination case of neighboring units tuned to different depth. Pollard et al. (1985) suggested that the neighboring units should excite each other as long as the disparity gradient, or the ratio between the depth and the distance, is less than 1. Our result, however, cannot be explained by disparity gradient. After all, the disparity gradient for the Slant configuration was the same as the Shelf and Staircase configurations. Yet, the flanker effect was different. In addition, we observed no flanker effect in the Sawtooth configuration even though all the image elements in that configuration were at the same distance. That is, there was no disparity gradient between the elements in that configuration. Hence, to sum up, the flanker effect in a 3D environment depends not on disparity difference per se but on coplanarity, whether or not the elements conform to a continuous surface regardless of the slant of that surface in depth. 

The elimination of flanker facilitation in the Shelf configuration was consistent with the result of Huang et al. (2006) in a similar configuration. They constructed the stereo collinear pair by presenting the target and flankers in counterphase in the two eyes and found no facilitatory effect relative to monocular input. Based on this result, and a similar disrupt of the flanker facilitation in a dichoptic viewing condition, Huang et al. suggested that the flanker facilitation is a monocular process. However, this interpretation is not supported by the extra conditions studied in the present paper. First of all, collinear facilitation was observed in the Slant configuration even though the target and the flankers were at different depths. This result implies that the critical factor is that the flankers are both collinear and (perceptually) coplanar, regardless of whether they were at the same disparity. 

In addition, in the present study, collinear pairs were used to evoke stereoscopic perception. We are hypothesizing that it does not matter whether or not the collinearity exists in the monocular stimuli; collinearity occurs when the image elements were collinear and coplanar and reduces when they were on different surfaces. That is, what is important for flanker facilitation is coplanarity, not depth. Huang et al. (2006) confounded these two factors. Instead, since perceiving a slanted surface also requires analysis of information from both eyes, collinear facilitation cannot be a monocular process. 

The 3D configuration-dependent flanker effect we report here suggests that the underlying mechanisms of surface representation are sensitive to the 3D structure of the image. That is, the processing of 3D structure should occur no later than the flanker effect. Since the flanker effect is, in general, considered as an early visual phenomenon manifested in the responses of V1 neurons (Chen, Kasamatsu, Polat, & Norcia, 2001; Polat, Mizobe, Pettet, Kasamatsu, & Norcia, 1998), our result may suggest that the surface representation occurs early in the visual processing. A similar statement has also been made by Shimojo, Kamitani, and Nishida (2001) with a very different experimental paradigm. 

An alternative view is that the differential facilitation effects are attributable to a higher level filtering process operating between the primary disparity processing stage and the final depth computation after the long-range interactions have come into play. This interpretation derives from the serial–parallel heterarchical model of binocular disparity processing of Tyler (1983, 1991), which identifies several stages between the primary disparity registration and the final depth signal that are necessary to account for the full scope of the reported data. In particular, there is a disparity cleaning stage that resolves the ambiguities of the correspondence problem to enforce the uniqueness constraint of a single depth solution at each spatial location and that is postulated to rely both on epipolar disparity inhibition and a lateral smoothness constraint of the depth solution to achieve its results. The smoothness constraint was originally proposed by Julesz (1971) and introduced as a computational device by Marr and Poggio (1976), but evidence for its operation as a principle of human disparity processing was provided by Tyler (1973, 1974, 1975). It should be recognized that, although smoothness does seem to be a key principle of the perceived depth signal overall, the opposite principle of an allowance for sharp breaks in the perceived depth map top provided for isolated depth patches is also a perceptual option (Tyler & Kontsevich, 2005). Thus, the visual system appears to have the capability of switching between a smoothness constraint and a sharp edge cut according to the weight of the evidence in the disparity field. How it chooses which principle to bring to bear at any given stereoscopic image location remains unresolved, however. 

Conclusions

We measured the target contrast detection threshold with and without the presence of the flankers under various 3D configurations for the target and the flankers. We showed that flanker facilitation occurs only when the target and the flanker were coplanar either on the same frontoparallel plane or on a slanted surface. The facilitation deteriorated when the target and the flanker were not coplanar. Such deterioration even occurred in the Sawtooth configuration in which the target and the flankers were locally slanted but at the same distance from the observer. 

Importantly, the result cannot be explained by differences in 2D stimulus arrangement as we found no difference in flanker effect when both coplanar and noncoplanar configurations were viewed monocularly. Finally, the result cannot be explained by a difference in depth discriminability among those configurations. We found no evidence that the observers had more difficulty in perceiving depth difference in the noncoplanar configuration than in the coplanar configurations. Our result cannot be explained by presence of a depth gradient across the Gabors because there was no facilitation effect in the Sawtooth configuration even though it has zero disparity gradient. Hence, we conclude that it is the difference in surface assignment, not the difference in disparity per se, that determines collinear facilitation, implying that the flanker facilitation requires not only collinearity but also coplanarity. 

Acknowledgments

This work was supported by NSC 96-2413-H-002-006-MY3 to CCC and AFOSR FA09-1-0678 to CWT. 

Commercial relationships: none. 

Corresponding author: Chien-Chung Chen. 

Email: [e-mail]. 

Address: Department of Psychology, National Taiwan University, 1, Sec. 4. Roosevelt Rd., Taipei 106, Taiwan.