Seeing depth coherence and transparency

Abstract

Gratings with different disparities are sometimes seen as transparent surfaces, each with a distinct depth, when they are superimposed, and sometimes they are seen as a coherent plaid confined to a single depth plane—stereo analogs of transparent and coherent motion. Briefly presented sinusoidal gratings of similar spatial frequencies are seen to cohere in depth. The resulting plaid generally appears in a depth plane different from that of either component grating viewed separately; the plaid may even appear on the oppose side of fixation from the component gratings. Under similar viewing conditions, squarewave gratings are typically seen as transparent. Objective measures, gathered here using depth-order discriminations, show that the perception of transparency between squarewave gratings requires a minimum disparity difference that varies with the gratings’ orientations. Gratings that are near orthogonal in orientation, or that give the plaid a near-horizontal disparity, favor the perception of coherence. Gratings that form a plaid having a large ratio of vertical to horizontal disparities favor the perception of transparency. The data are consistent with a Bayesian prior favoring single surfaces when disparities are small and near-horizontal. Disparities that are large or non-horizontal are more likely to be aperture disparities that result from viewing separate but overlapping surfaces. The sinewave-squarewave difference leads to the conclusion that coherence between components is required both for seeing a broadband pattern in a single depth plane and for seeing it in a different depth plane from other superimposed patterns.

Introduction

Except when viewing a ganzfeld or an object that fills the visual field, we view overlapping surfaces, each with a different depth. In the region of overlap, we see only the nearer surface, if this surface is opaque, or multiple surfaces in their distinct depth planes, if the nearer surfaces are transparent. Variations on these alternatives exist, for the surfaces might interact optically (as when the nearer surface belongs to a lens) or perceptually (as in camouflage). While stereopsis can reveal overlaid surfaces that would otherwise be lost to perception (Julesz, 1971; Pettigrew, 1990; McKee, Watamaniuk, Harris, Smallman, & Taylor, 1997), it can also obscure these surfaces. Examples of the latter are disparity averaging (Schumer & Ganz, 1979; Parker & Yang, 1989; Rohaly & Wilson, 1994), where two superimposed surfaces merge, yielding a single surface perceived at an intermediate depth, and depth capture (Mitchison & McKee, 1985; Ramachandran & Cavanagh, 1985; Ramachandran, 1986), where one surface appears at the depth of another despite their disparity, or disparity range, difference. Related to these is depth coherence, where stimuli with different disparities combine to yield a single pattern (Adelson & Movshon, 1984; Farell, 1997, 1998). The perceived depth of a depth-coherent pattern cannot be predicted from the disparities of the component stimuli alone, for the pattern’s disparity varies with the components’ orientations as well as their disparities (Farell, 1998, 2003). Changing the orientation even of a zero-disparity component can change the pattern’s perceived depth. 

Depth coherence is analogous to motion coherence (Adelson & Movshon, 1982). Superimposed drifting gratings are seen as moving coherently or independently, depending largely on the gratings’ similarity on a variety of spatial, temporal, and chromatic properties. By one well-known interpretation, the visual system initially registers object motion as velocities of one-dimensional (1D) stimulus components. It subsequently combines these velocities into a single 2D object-motion signal or segregates them into multiple transparent-motion signals (Adelson & Movshon, 1982), perhaps according to a Bayesian decision rule (Weiss, Simoncelli, & Adelson, 2002). Only if a similar component analysis underlies stereo perception of 2D patterns is “depth coherence” descriptive of a visual process, a process of combining distinct stimulus elements, each with its own disparity, to yield a single depth percept. If, instead, stereo matches are made between 2D features of the pattern-that is, if 1D components are monocularly conjoined before stereo-matching-then this would itself account for the perceptual coherence of superimposed gratings; there would be no separate depth signals to cohere. 

Evidence for a component analysis comes from studies of two types. First, disparity adaptation can enhance depth discrimination of a 2D pattern when the adapting disparity is near the disparity of the 1D components, but not when it is near the disparity of the 2D pattern (Farell, 1998). Second, it is the disparity of the spatial-frequency components of plaid patterns, not the disparity of the patterns’ luminance profiles, that limits fusion and the detectability of disparity (Levinson & Blake, 1979; Heckmann & Schor, 1989; Farell, 2003). 

A smoothness assumption has been a common feature of early modern models of stereopsis, usually implemented by inhibition between disparities in the same visual direction (Sperling, 1970; Julesz, 1971; Dev, 1975; Nelson, 1975; Marr & Poggio, 1976). Smoothness reinforces opacity; these models tend to see only one surface when presented with multiple disparities in one visual direction. Several newer models, though, succeed in segregating these surfaces and seeing transparency (Prazdny, 1985; Pollard, Mayhew, & Frisby, 1985; Marshall, Kalarickal, & Graves, 1996; Tsai & Victor, 2003; Zhao & Farell, 2004). Disparity averaging, depth capture, and depth coherence can all be regarded as failures to perceive transparency. In disparity averaging, loss of information—an unresolved disparity difference—is the reason, whereas in depth capture and depth coherence, multiple disparities are mapped onto a single depth. Failure to perceive transparency entails both a misjudgment of surface optical properties and a transposition of perceived surface distances.1 The transparency problem—representing multiple disparity values at a single visual direction—has received wide recognition in studies of human and machine vision. The opacity problem—perceiving multi-scale components at a single depth—has been widely under-appreciated. A broad-bandwidth pattern is readily seen at a depth that could be aliased by disparity detectors tuned to the scale of its visible higher frequency components. The higher harmonics of a squarewave grating with a phase disparity of 60°, for example, would appear with little depth, no depth, or reversed depth if presented in isolation with the same spatial offset. 

Humans perceive several varieties of transparency under naturalistic viewing conditions (Kersten, 1991) (and under some that are unnaturalistic, as well [Nakayama, Shimojo, & Ramachandran, 1990]). Transparency can be perceived, too, in random-dot and related stereograms (Julesz, 1971; Prazdny, 1985; Akerstrom & Todd, 1988; Weinshall, 1989; Ahuja & Farell, 1997, 1998a, 1998b, 1999; Gepshtein & Cooperman, 1998; McKee & Verghese, 2002). We believe that humans also experience depth coherence and depth capture very frequently—not when failing to perceive the transparency of overlaid surfaces, but rather when viewing opaque broadband patterns, whose components would otherwise appear smeared across different depth planes. In this study, we examine the boundary conditions for perceiving transparency between superimposed gratings, sinewave and squarewave. We examine static superimposed gratings that have the same spatial frequency; frequency differences and differences in direction of motion progressively increase the likelihood of perceiving transparency instead of coherence (Farell & Ahuja, 1996; Farell, 1997; Ahuja & Farell, 1997; Lankheet & Palmen, 1998). We use an objective test for distinguishing patterns that cohere and those that segregate into transparent components, supplemented by observers’ subjective judgments. 

Our method is to present two superimposed gratings and to measure disparity thresholds for discriminating their depth order. The grating pairs are shown in each of two intervals. In one interval, the target grating, identified by its orientation, has non-zero disparity and the non-target grating has zero disparity. In the other interval, these are reversed: The target grating has zero disparity and the non-target grating has non-zero disparity. The observer’s task is to discriminate depth order by reporting the interval in which the target grating was off the plane of fixation. If the gratings disparities are available to the observer as independent values, then the task should be about as easy as detecting the disparity of a grating presented singly; the threshold for depth-order discriminations should be similar to stereoacuity for gratings. However, if the gratings’ disparities combine to yield a single value (i.e., a single perceived depth), the task should be possible only if the observer knows how the gratings’ disparities map onto the plaid’s perceived depth. To regulate this knowledge, we supply or withhold feedback about the correctness of the response. 

Experiment 1 is an exploratory study of the method; Experiment 2 focuses on the findings of the first experiment and compares depth-order discrimination of sinewave and squarewave gratings; and Experiment 3 compares the influence of plaid parameters versus grating parameters on perceived transparency. 

Experiment 1

Methods

Stimuli

Stimuli consisted of two superimposed sinusoidal gratings, whose luminance modulations were added. The two gratings had the same spatial frequencies and contrasts, but different orientations. In each presentation, one grating of the pair had zero disparity and the other had non-zero disparity. We designate one grating the “target” grating and the other the “non-target.” The two were distinguished by their orientations. 

A trial consisted of two stimulus intervals, each 180 ms in duration and separated in time by 330 ms. Onsets of the intervals were cued by short tone bursts. The stimuli were identical across the two intervals except for grating disparity and phase. In one interval, the target grating had a variable disparity D and the non-target grating had zero disparity, where D is a disparity phase angle. These disparity values were reversed in the other interval: The target had zero disparity and the non-target had phase disparity D. The two stimulus configurations within a trial are sketched in Figure 1. The absolute phases of the gratings were independently randomized (identically for left- and right-eye half-gratings) in each interval; this translates the gratings (and the resultant plaid) unpredictably, disrupting local position cues without affecting disparity. 

Figure 1

 

Depth-order task. Two superimposed gratings are presented in each interval. One grating (Component A) has disparity that is zero in Interval 1 (marked by ×) and non-zero in Interval 2. The other grating (Component B) has the reverse assignment. The observer’s task is to report the interval in which the target grating (Component A, say) appeared with non-zero disparity. The two intervals present the same phase disparities across two differently oriented gratings; the perceived depths of the separate off-horopter gratings are not necessarily the same (Farell, 1999).

The grating spatial frequency was 1 c/d for the results presented here; other frequencies, both lower (0.5 c/d) and higher (up to 4 c/d), were examined but did not yield substantially different results. Contrast for 1 c/d gratings was fixed at 10% for all observers. Gratings were presented within hard-edged circular windows 7.8° in diameter. The windows were centered on black fixation squares whose angular subtense was 6’ on a side; they were visible throughout the run of trials. The windows and fixation squares had zero disparity; the only non-zero disparities were interocular carrier phase shifts. 

Gratings appeared on the left and right sides of a luminance-calibrated monitor controlled by a Macintosh computer and were viewed through a mirror stereoscope. Viewing distance was 57, 74, or 93 cm, depending on the monitor used. Pixels extended 2 arcmin on a side. Mean screen luminance was approximately 20 cd/m². Viewing was with natural pupils in a dimly lit room. 

A check was made using two observers to determine whether the results were specific to the window’s size or profile. Neither halving the window diameter nor switching to a Gaussian window profile appreciably altered the results. 

Procedure

Before each run of trials, the observer was informed of the orientation of the target and non-target gratings, which remained fixed throughout the run. The observer’s task was to select the interval in which the target grating had non-zero disparity. The sign of this disparity was positive. If the gratings were perceived as transparent, the off-horopter grating would be seen behind the fixation plane. However, if the gratings cohered, the resulting plaid might be seen in front of or behind the fixation plane. The plaid’s disparity depends both on the components’ disparities and on their orientations. Figure 2 shows how the apparent depth of a plaid composed of a 45° grating with positive disparity varies from “Near” to “Far” depending on the orientation of a superimposed zero-disparity grating. This variation in the sign of the plaid’s depth is predictable from the horizontal component of the plaid’s disparity. When one component has a disparity of zero, as was the case in the experiments reported here, the plaid has a disparity whose direction is parallel to the orientation of this grating (Farell, 1998, 2003). As shown in Figure 2, when the component gratings have orientations in the same quadrant, the plaid’s disparity will have opposing signs depending on which of these gratings has non-zero disparity. The perceived depth of the plaid is seen to reverse accordingly (Farell, 1998). 

Figure 2

 

Polarity of plaids’ horizontal disparities. The sign (positive vs. negative) of the disparity of a plaid containing a positive-disparity component grating oriented at 45° as a function of the orientation of a superimposed zero-disparity component grating with orientation given by the polar angle.

An observer of gratings segregated into different depth planes has depth-order information available directly. An observer of coherent gratings might deduce the grating depth order from insight into how plaid depth depends on component disparity and orientation, by access to a formula, table, or Figure 2, or from feedback about response accuracy. Auditory feedback was provided and withheld in separate runs of trials. Each observer received all the no-feedback conditions before receiving any of the feedback conditions. The intention was to prevent lessons learned from the feedback experience from being applied to no-feedback conditions. 

The non-target grating was nominally irrelevant to the task. However, because the disparity of the non-target grating complemented that of the target grating, the non-target could serve just as well as the target grating as the stimulus on which decisions are based. Reversing the roles of target and non-target does not alter the logic of the experiment; the designations merely establish a convention for consistent responding within and between conditions. Switching the target and non-target gratings need not matter empirically either, for disparity thresholds are typically a constant phase angle for gratings of different orientations (provided they are not too close to horizontal) (e.g., Farell & Ahuja, 1996; Morgan & Castet, 1997; see also Ogle, 1955; Ebenholtz & Walchli, 1965; Blake, Camisa, & Antoinetti, 1976). 

We measured the threshold disparity phase angle for discriminating the two depth orders presented on each trial. The value of the non-zero disparity D was varied across trials according to the QUEST algorithm (Watson & Pelli, 1983; King-Smith, Grigsby, Vingrys, Benes, & Supowit, 1994). Two independent QUEST staircases of 40 trials each were randomly intermixed across trials within each run. Each observer had three or four runs per condition. The measured thresholds (for 82% correct responses) were normalized by the disparity detection thresholds when the target grating was presented alone, yielding disparity threshold increments due to the presence of the non-target grating. 

We also collected subjective classifications of depth coherence and transparency on the same stimuli. Disparities for these classifications were under the control of a constant-stimulus procedure, for we were interested in the appearance of the stimuli at various disparities, not just the disparity threshold for a particular appearance. 

Observers were instructed to fixate on the central fixation squares throughout the trial. Observers initiated a trial by clicking with a mouse; they responded by clicking on-screen response labels that followed the offset of the second stimulus interval by approximately 0.5 s. 

Observers

The three observers had normal stereo vision and normal or corrected-to-normal acuity. Only one (one of the authors) was aware of the purposes of the experiment. 

Results

The subjective judgments take the form of binary classifications—coherent versus transparent—of superimposed gratings as a function of the gratings’ orientations and disparities. The objective measurements take the form of disparity threshold elevations for discriminating the depth order of the superimposed gratings, as a function of the gratings’ orientations. The three observers gave quite similar results both on threshold measures and on subjective classifications and data for the two tasks are in close agreement, so we present discrimination thresholds for one of them and, where applicable, supplement the threshold data with the subjective classifications. 

Figure 3 shows threshold elevations for sinewave gratings as a function of the orientation of the non-target grating. Disparity detection threshold for the target grating measured in the absence of the non-target grating provides the baseline thresholds. Threshold elevation is plotted as radial distance from the origin. Non-target orientation is given as the polar angle; each data point appears twice, with centric symmetry, in these 360° plots. Figure 3a shows data collected without feedback. For these data the target grating had an orientation of 45° (where 0° is horizontal); in the absence of feedback, target orientation had little affect on thresholds. Figure 3b shows data collected with feedback provided; targets were oriented at 30°, 45°, or 90° on different runs. 

Figure 3

 

Threshold disparity for the depth-order discriminations in Experiment 1, as a function of non-target grating orientation. Elevations express threshold grating disparity as a multiple of the disparity detection threshold for the target grating presented separately (upper left axis). Grating phase disparity thresholds are also shown (lower right axis). a. Observer received no response feedback; target orientation was 45°. b. Observer given feedback on each trial; target orientations were 30°, 45°, and 90°.

Without feedback, thresholds for discriminating depth order were many times the threshold for detecting the disparity of the target grating. Regardless of the observer, the disparity staircase approached 180°, the maximum permissible value, for most non-target orientations. Diplopia was evidently required for depth-order discrimination in these cases. Exceptions occurred for one observer when non-target orientations were 0° and 90°; here thresholds were elevated by a mere factor of 5 or so, as seen in Figure 3a. In these conditions, the disparity of either the non-target grating (in the 0° case) or the plaid (in the 90° case) is strictly vertical in one interval, a direction that may be distinctive and identifiably different from the disparity direction in the other interval. (The possibility that the results for no-feedback conditions may have been contaminated by an artifact is discussed below.) 

The threshold function was radically different when feedback was given, as seen in Figure 3b. Threshold elevation was smaller and more selectively dependent on both target and non-target orientation. Threshold elevation was greatest for non-target orientations centered at about 135° when the target grating was oriented at 45°, and at about 150° when the target was at 30° — that is, when target and non-target were mirror-images of each other, reflected about a meridian, either horizontal or vertical. This pattern applies roughly to 90° targets as well, for which thresholds were highest for non-target orientations nearest 90°. At mirror-image grating orientations, the disparities seen in the two intervals differ only in the sign of the vertical component of disparity. This is so whether disparity is measured on the component gratings or the plaid’s 2D features. With only a vertical disparity sign difference to distinguish the two intervals, task performance cannot be based on perceived depth. It must rely instead on other cues that, as is clear from the elevated thresholds in Figure 3b, come into play only at large disparities. 

Without feedback, depth order was inaccessible to observers except at very large disparities and perhaps in special cases involving vertical disparities (Figure 3a). This is not to say observers could not discriminate the two plaids presented on each trial; in general, they perceived the plaids at different depths. But they could not label the plaids as the task demands. This is made apparent by the effect of feedback. Depending on the relative orientations of the component gratings, the plaid could appear at a greater depth when the target grating has non-zero disparity or when the non-target grating has non-zero disparity (Figure 2). Feedback informs the observer that correct responses are those that select the interval in which the plaid depth is the greatest or the least (or on the near versus far side of fixation, in cases of depth reversals). For feedback to work, there must be a difference in the perceived depth between the two intervals. This excludes symmetrical plaids, for there is no perceptual effect of the sign of a uniform vertical disparity. 

Each of the three observers reported seeing the superimposed sinewave gratings as cohering in depth.2 We take depth coherence to be consistent with the pattern-depth hypothesis: A single depth—the perceived depth of the plaid—is the only variable on which the observer can reliably base decisions in this task; the observer does not have access to the disparities of the component gratings (Farell, 2003; see also Watt & Morgan, 1985; Welch, 1989; Olzak & Thomas, 1991). A plaid that is seen at a particular depth could in principle have a non-zero disparity on either one or both of its component gratings. Without feedback or knowledge of how plaid depth depends on grating disparity and orientation, deducing the actual disparities of the component gratings should not be possible. 

Figure 4 shows predictions based on the assumption that discrimination thresholds are a function solely of the horizontal components of the plaid’s disparities across the two intervals:  

 

where H₁ and H₂ are the horizontal vector components of the disparities presented in the two intervals, and k is a proportionality constant. This assumption is unrealistic in ways discussed below. However, predictions based on it are instructive both in their matches to the observed data and in their misses. For plaids with symmetrically oriented component gratings, the two horizontal disparities are equal and the predicted thresholds are infinite; these are marked as ∞ in Figure 4 and given a threshold elevation of 10, the approximate mean value across observers for these conditions. For the two oblique target orientations, 30° and 45°, this simplistic model does a reasonable, though inexact, job in capturing the orientation dependence of threshold elevations observed in the experiment. 

Figure 4

 

Predicted threshold elevation for depth-order discriminations with feedback, the conditions generating the data of Figure 3b. Predictions are based on the horizontal component of the disparity of the plaid formed by two superimposed sinewave gratings. Predicted thresholds are proportional to the reciprocal of the difference between the horizontal components of the disparity in the two intervals of a trial, normalized by the observer’s disparity detection threshold for single vertical gratings. Unlike real data, the predictions are not bounded by a 180° phase disparity ceiling; values exceeding this limit are marked with ∞ and set to match the approximate average observed threshold elevation value of 10.

The predictions fail conspicuously in two ways. First, threshold facilitation—a value below 1.0—is predicted when target and non-target orientations are similar (for 30° and 45° targets). However, facilitation is nowhere evident in the data. The reason for this discrepancy is that the predictions do not take into account the component-limit on threshold disparity for plaids: 2D feature disparities, even if large, are detectable only if the disparities of the 1D components equal or exceed their thresholds (van Ee, Anderson, & Farid, 2001; Farell, 2003). This limits the minimal threshold value for superimposed gratings to that for individual gratings. Hence, observed threshold elevation does not drop below 1.0. 

The second failure is the prediction of no threshold elevation for 90° targets. Predictions are based on horizontal disparities only; they ignore the effect of vertical disparities on the discriminability of horizontal disparities (Stevenson & Schor, 1997; Morgan & Castet, 1997; Farell, 2003). The effect of vertical disparity appears as elevated thresholds in Figure 3b for the pairing of the 90° target and near-vertical non-targets. Thresholds are elevated for this combination because it yields vertical or near-vertical plaid disparity in both intervals. 

Two gratings with orientations close to vertical produce elevated thresholds, but note the absence of an oblique counterpart. It may seem that gratings with similar oblique orientations would yield similar plaid disparities across the two intervals, and their horizontal components, being similar, would lead to threshold elevation. Yet threshold elevation is not seen in the predicted or the observed data. The reason is that when the gratings’ orientations are in the same quadrant, the sign of the plaid’s horizontal disparity reverses when one component grating and then the other is given a non-zero disparity, as sketched in Figure 2. The resulting depth reversal, with the plaid switching between “near” and “far” from one interval to the next, makes the discrimination easy (with feedback) and avoids any elevation of threshold. 

To examine how perceived transparency depends on component versus plaid disparity and to further test the pattern-depth hypothesis, we applied the depth-order task to two sets of plaids, one modeled after the symmetrical grating pairs of Experiment 1, for which only vertical disparities differed between intervals, and the other modeled after the asymmetrical pairs, for which both horizontal and vertical disparities differed. We examined both sinewave and squarewave gratings. The data of Experiment 1 indicate that sinewave gratings are perceived as coherent in all conditions tested. When presented under similar conditions, however, squarewaves can support the perception of transparency (Ahuja & Farell, 1997; Farell, 1998). 

Experiment 2 also sought to control for a potential artifact in the no-feedback conditions of Experiment 1. Observers in that experiment were asked to discriminate the depths of target and non-target gratings. An observer without access to grating disparities or to feedback might respond by guessing on a trial-by-trial basis (i.e., guessing inconsistently). Alternatively, he or she might respond by guessing on the basis of the perceived depth of the plaid, not knowing how this was related to the depth order of the gratings. For example, an observer might arbitrarily decide to select the interval with greater plaid depth—in effect, guessing consistently across trials. Consistent guessing in this way would produce a low threshold if the arbitrary choice happened to be the correct one, and a very high threshold otherwise. Therefore, low thresholds in the absence of feedback would be ambiguous, indicating that the observer was responding either veridically on the basis of grating depth or arbitrarily but luckily on the basis of plaid depth. (A third alternative is that observers were responding on the basis of plaid depth with knowledge of the non-obvious correlation between plaid and component disparities.) The results of Experiment 1 vitiate these distinctions; in the absence of feedback, thresholds were very high. However, the possibility exists for artifactually low thresholds to express themselves in similar experiments. Moreover, for the observer whose threshold elevations were reduced for horizontal and vertical gratings (Figure 3a), the possibility that the appearance of the resulting plaids led to consistent guessing within runs certainly exists, yielding a mix of high and low thresholds across runs and an intermediate threshold for the average. Steps were taken in Experiment 2 to minimize such nonessential influences. 

Experiment 2

Methods

Stimuli

Superimposed gratings were again used as stimuli; they were divided into two sets differing in orientation. Pairs of gratings in one set were oriented symmetrically about the vertical axis. Those in the other set were asymmetrically distributed about the vertical, differing from the symmetrical set by a 45° clockwise rotation. The orientation difference between components ranged from 20° to 160°. The orientation pairs in the symmetrical set were 80°/100°, 65°/115°, 45°/135°, 25°/155°, and 10°/170°, and in the asymmetrical set 35°/55°, 20°/70°, 0°/90°, −20°/110°, and −35°/125°. For any grating, either of two opposite perpendicular disparity directions is possible. The directions used in this experiment were such that the horizontal component of the perpendicular disparity of all the gratings had the same sign. As a result, each plaid was distinct under a rotational transformation from the other plaids within a set. The difference in disparity directions between pairs of gratings preserved the difference in their orientations. For example, pairs with an orientation difference of 20° had disparity directions that differed by 20°, not by 160°. 

Each plaid had two possible disparity directions, which were displayed in random order in the two intervals of a trial. One disparity direction arose when the target grating had non-zero disparity and the other when the non-target grating had non-zero disparity. The alternative disparity directions of the symmetrical plaids differed only in the sign of their vertical components, so their horizontal components were equal. The disparity directions of the asymmetrical plaids were asymmetrical about the horizontal meridian and so differed in the magnitudes of both horizontal and vertical components. Therefore, the perceived depth of the symmetrical plaids should be the same in the two intervals, provided the component gratings cohere, whereas the perceived depth of the asymmetrical plaids should differ from one interval to the next. Separate sinewave and squarewave plaids were created. Examples of both are shown in Figure 5. As in the earlier experiment, their spatial frequencies were 1 c/d, and their contrasts were 0.1. Sinewave plaids were expected to cohere in all conditions; there should be no measurable threshold for the perception of transparency nor, in the absence of feedback, for the discrimination of depth order. Squarewave gratings can appear as transparent, at least in some conditions (Ahuja & Farell, 1997; Farell, 1998), and this should allow a depth-order threshold to be obtained whether or not feedback is given. 

Figure 5

 

Stereograms made of superimposed sinewaves (a) and squarewaves (b). For each waveform, the upper stereogram contains one grating with zero disparity and another at non-zero disparity; and the bottom stereogram presents the same phase disparity values, now switched between the gratings. The task is to report the stereogram, of the two presented in separate intervals, in which a particular grating (the more vertical of the two, say) had non-zero disparity. The task is easy only if the gratings are seen as transparent. (Any jagged edges visible here are imaging artifacts that did not appear in the stimuli presented in the experiment.)

Procedure

The ability to discriminate grating depth order was measured, as in the previous experiment, by having observers select the interval in which the target grating’s disparity was non-zero. The ambiguity of low thresholds in no-feedback conditions discussed above was dealt with by collecting two sets of thresholds, one while each of the two components was assigned the role of the target. If the mean thresholds of these two conditions differed substantially (by more than 2 SEM), the higher of the two was taken as the threshold; otherwise the data were combined. The intention was to defeat the accidental success of the strategy of responding consistently, but arbitrarily, on the basis of perceived plaid depth. In fact, threshold were found not to depend on which of the two gratings was named the target; in all cases, the two thresholds were similar and were combined. 

Thresholds were measured using the QUEST algorithm, as in Experiment 1. Thresholds for sinewave stimuli presented under no-feedback conditions were measured a second time for two of the observers using a constant-stimulus procedure to check on the validity of high thresholds. It is possible that a non-optimal initial disparity estimate for QUEST may have driven disparities in early trials beyond the range that is useful for task performance, causing them to continue to rise throughout the run. The constant-stimulus procedure was used to determine whether there existed any disparity that would support depth-order discrimination of sinewaves in the absence of feedback. No such disparity was found and only the thresholds obtained with QUEST are reported below. Thresholds for all the no-feedback conditions were collected first, followed by the conditions with feedback. Other experimental details were as in Experiment 1. 

Observers

Three observers were run, none of whom participated in Experiment 1. 

Results

Disparity thresholds for discriminating the depth order of target and non-target plaid components are shown in Figure 6, with circular symbols indicating sinewave thresholds and square symbols indicating squarewave thresholds. Conditions for which no disparity threshold was measurable are marked with asterisks at phase disparities of 180°, the limiting value. 

Figure 6

 

Threshold phase angle for target grating in depth-order discriminations of Experiment 2. Circles: sinewave gratings; squares: squarewave gratings. Top row: symmetrical condition; bottom row: asymmetrical condition. Threshold phase disparities of 180°, the largest realizable value, are marked with asterisks to indicate a failure to perform the task reliably at any disparity. For most data points, error bars are smaller than the symbols.

Sinewave gratings

For superimposed sinewave gratings, depth order was discriminable in only one condition: asymmetrical grating pairs with feedback provided. In this condition, thresholds were generally low—approximately at single-grating values—and showed only modest effects of the gratings’ relative orientations, rising as the orientations converged. 

The inability to discriminate sinewave depth order in the other conditions is consistent with a lack of access to task-relevant information other than plaid depth. Plaid depth fails to distinguish the two different plaids formed from symmetrical gratings, for these plaids differ only by the sign of their vertical disparities. Perceived depth does distinguish the plaids formed from asymmetrical gratings; these plaids differ in the magnitude, if not also the sign, of their horizontal disparities. Because the polarity of plaid depth corresponds to grating depth order in a peculiar and non-obvious way (Farell, 1998, 2003; see Figure 2), feedback is required for reliable discrimination of these disparities. 

The interaction of orientation symmetry and feedback is dramatic, governing whether the two sinewave depth orders can be discriminated at all. The interaction depends on the coherence in depth of the gratings: Little effect of symmetry, feedback, or their interaction would be expected if the gratings were perceived in separate depth planes. Also, these effects are not what would be expected from contrast masking between the gratings. Consistent with the threshold data, observers reported seeing sinewave gratings as cohering in depth in all conditions. 

Squarewave gratings

Waveform matters. Observers were able to discriminate squarewave depth order in all cases, regardless of grating orientation or the availability of response feedback (Figure 6). Yet there was an effect of grating orientation on the threshold disparity for depth ordering, with thresholds increasing as the orientation of the gratings deviated from vertical. This effect is seen only in the symmetrical condition, where the two gratings’ orientations deviate equally from vertical and extend further from vertical than in the asymmetrical condition. Observers’ subjective classifications confirm that the square-waves appeared to cohere in depth at disparities below the thresholds of Figure 6, which for symmetrical gratings exceeded the threshold for detecting disparity at all but the smallest orientation difference (20°). Only at or beyond disparities that support depth-order discrimination did observers consistently report these gratings as appearing transparent. 

For superimposed squarewave gratings to appear transparent, the grating disparity must have a minimum amplitude that depends on the orientations of the component gratings. This threshold disparity for transparency ranges over nearly a log unit, from the disparity detection threshold (an interocular phase shift of about 4°–5°) for near-vertical gratings to approximately 40° for near-horizontal gratings (Figure 6). The influence of grating orientation on perceived transparency cannot be reduced to a failure to detect small disparities of individual gratings, for disparity detection thresholds show only a small variation across grating orientations that are not near the horizontal (Farell & Ahuja, 1996). Squarewave transparency might still depend directly on grating disparity direction; for example, horizontal transparency might best be mediated by the near-horizontal disparities of near-vertical gratings. We examined the alternative: that the disparity direction of the plaid predicts thresholds for depth-order discrimination and perceived transparency. 

Plaids’ 2D disparities encompass two distinct disparity components—horizontal and vertical—to which much evidence attests as having distinct roles in depth perception (see Howard & Rogers, 1995). To assess their influence on squarewave depth-order thresholds, we examined the horizontal and vertical disparity components of the disparity of the plaid formed by the symmetrical grating pairs, plaids whose disparity components are identical in absolute value across intervals. Using the data for one grating condition (20° angular separation, averaged over feedback and no-feedback conditions) as a reference, predicted thresholds are derived for other conditions based on their ratio of component horizontal to vertical disparities:  

 

where THR_δ is the predicted threshold for disparity direction difference THR, THR_ρ is threshold for the reference condition (here 20°), H and S are the relative sizes of the vertical and horizontal components of the plaid’s disparity, and S is the ratio of sensitivities to horizontal and vertical disparities (typically about 2.0), measured with vertical and horizontal gratings, respectively, of the same spatial frequency used to create the plaids. The reciprocal of S is added to vertically shift the entire function to match the observed threshold for the 20° condition used as a reference. 

Figure 7 compares the predictions and the thresholds for symmetrical squarewaves taken from Figure 6. Overall, the fit is quite good.3 Thus, depth-order thresholds are proportional to the ratio of horizontal and vertical plaid disparities, discrimination being more difficult when the vertical component is small relative to the horizontal component than when it is large. Figure 7 implies that the weights given to horizontal and vertical disparity components in perceiving coherence versus transparency are unchanged, except for the orthogonal case, as the disparity direction varies between the near-horizontal (10°) and the near-vertical (80°) (cf. Farell, 2003). The picture that emerges, put simply, is that horizontal plaid disparity—the horizontal disparity of the 2D features created by superimposed squarewave gratings—is taken as evidence for coherence and vertical plaid disparity is taken as evidence for transparency. 

Figure 7

 

Circles show depth-order discrimination thresholds for squarewaves in the symmetrical condition of Experiment 2, as a function of the component disparity direction difference. Data from feedback and no-feedback conditions are averaged. Dotted line shows predictions based on the ratio of the horizontal and vertical components of the plaids’ disparities. The ratio was normalized by relative sensitivities for vertical and horizontal disparities measured on horizontal and vertical squarewave gratings, respectively, which averaged 2.2 across observers for 1 c/d gratings, with horizontal gratings (vertical disparities) having higher thresholds. The ratio was used to scale the observed thresholds for the 20° disparity direction difference. Observer S3.

Whether horizontal and vertical disparity components play the same role for other grating patterns is only partially open to question. Probably the asymmetrical squarewave pairs behave similarly, but this is hard to verify, for the asymmetrical pairs produce small horizontal:vertical disparity ratios in one of the presentation intervals. While consistent with their low discrimination thresholds, this makes the asymmetrical pairs a weak test of the effect of the horizontal:vertical disparity ratio. In any case, the generality of the ratio as a criterion for transparency perception is most obviously jeopardized by sinewaves, which cohere regardless of the plaid disparity direction. We will take up the difference between sinewaves and squarewaves in Discussion. 

Experiment 3

Interpreting the effect of the horizontal:vertical disparity ratio is complicated by the fact that the symmetrically oriented grating pairs from which this ratio was extracted impose correlations that could hide the source of the ratio’s effect. The symmetry results in correlated disparity directions between the two component gratings and in correlated disparity directions between the components and the plaid. A new set of plaids, one that allows the orientations of the component gratings to vary independently, was used to investigate the separate influence of component and plaid disparities. 

Methods

In the single interval presented on each trial, one of two superimposed squarewave gratings, the target, had a non-zero disparity. This disparity was fixed at a phase angle of 30°. The orientation of the target grating was fixed across a run of trials at 0°, 15°, 45°, 75°, 90°, 105°, 150°, or 165°; the target’s perpendicular disparity direction was equal to these orientation values minus 90°. The non-target grating, always with zero disparity, varied in orientation from trial to trial, randomly taking on the values 15°, 30°, 60°, 75°, 90°, 105°, 120°, 150°, or 180° (but skipping the orientation that matched the target’s). The task was simply to classify the superimposed gratings as to their appearance, either as appearing in the same depth plane (coherent) or in separate depth planes (transparent). The percept was not necessarily binary: Some gratings, especially those with similar, near-horizontal orientations, might appear joined across depth planes, looking somewhat like a propeller. Cases like this, lacking complete segregation and transparency, were to be classified as coherent. 

The judgment required of the observers was subjective and no feedback followed the response. Data were collected from two observers, one who had participated in Experiment 2; each received 20 trials per orientation combination. Other experimental details were unchanged from those of the previous experiment. 

Results

The probability of perceiving coherence is shown in Figure 8 for each combination of target and non-target orientation. Data are plotted separately for each target orientation. The polar angle in these plots gives the grating orientations; each data point corresponds to a different non-target orientation and the arrows indicate the target orientation for that plot. It is clear that each target orientation (with the partial exception of the horizontal target) supported the perception of both depth coherence and transparency. This is not surprising; with target disparity fixed at a moderate value and judgments subjective, the observer’s criterion will likely vary from run to run, keeping the ratio of coherent versus transparent classifications away from extreme values. Interest lies in the variation of the two classifications with the relative orientations of the gratings. 

Figure 8

 

Probabilities of reporting depth coherence between target and non-target squarewave gratings for Experiment 3. Probability is given by radial extent and non-target orientation is given by polar angle. Target orientations are shown by arrows. Each graph is for a different target orientation and contains 8 or 9 data points; those not visible are superimposed with others at a value of 0. Data points for 180° non-target gratings are plotted a second time at 0°.

There was a strong tendency for gratings to be classified as cohering in depth when their orientations were roughly orthogonal. There was also a trend for coherence to be more likely when the non-target grating was not only near-orthogonal to the target but also more horizontal than the orthogonal direction. The disparity direction of the plaid, which is parallel to the non-target grating, was predictive of perceived coherence only through this bias toward the horizontal. 

A measure of these trends over all conditions in the experiment can be seen in Figure 9. Here probability of coherence is plotted as a function of the difference between the directions of the components’ perpendicular disparities (even if the amplitude is zero). These disparity directions are directly related to the gratings’ orientations: A 90°-target grating would differ in disparity direction by 30° from a 60°-non-target grating and by −30° from a 120°-non-target grating. The derivative of the best-fitting polynomial is zero at disparity direction differences of −11° and +117°, the former being the function’s minimum and the latter its maximum. These values approximate 0° and 90°, the values expected if coherence depended solely on the relative disparity directions (or orientations) of the gratings. Their shift away from 0° and 90° marks the influence of absolute orientation, whereby near-horizontal plaid disparities are more likely to result in coherence than are more-vertical disparity directions. 

Figure 9

 

Coherence probability in Experiment 3 as a function of the difference in the gratings’ perpendicular disparity directions. Disparity directions are signed, and therefore the difference nominally extends beyond 180°. The best-fitting cubic polynomial, with a maximum at +117° and a minimum at −11°, is also shown. Note that points plotted on the far left, at or near −90°, are supported by only the few conditions in which the target orientation is at or near vertical.

Discussion

Seen briefly, superimposed sinewaves of the same spatial frequency appear to cohere in depth, despite a disparity difference between them. They form a plaid at a single depth plane. Superimposed squarewaves, on the other hand, appear as transparent surfaces, occupying separate depth planes, provided the disparity difference between them is large enough. Disparity thresholds for depth-order discriminations are closely correlated with these appearances. 

It is possible to discriminate the depth order of sinewaves when the plaids they form differ in perceived depth and when feedback allows this depth difference to be mapped onto an appropriate response. In these cases, observers judge plaid depth, not component depth or disparity. Our observers appear incapable of perceiving the individual depths of component sinewaves or accessing their disparities directly at the brief durations in which the stimuli were presented. Their only available disparity cue is the perceived depth of the plaid. 

By contrast, the depth order of squarewave gratings can be discriminated without the aid of feedback, consistent with the perception of these gratings as transparent. However, though they support transparency, superimposed squarewaves are not perceptually independent. The threshold for perceived transparency and depth-order discrimination varies with the orientation of the component gratings and can be many (∼8) times higher than the squarewave grating stereoacuity. Disparity detection thresholds for single gratings, it will be recalled, do not vary greatly with grating orientation outside a threshold-elevated region of near-horizontal orientations, showing at most a factor-of-two difference between vertical and near-horizontal (∼10°–15°) (Farell & Ahuja, 1996). Therefore, the dependence of depth-order thresholds on grating orientation cannot be reduced to an effect of grating properties. It must depend on the interaction between gratings, or directly on properties of the plaid. 

For any pair of gratings, there is a range of disparities centered on zero that gives rise to the perception of coherence; transparency, if seen at all, appears outside this range. For static sinewave gratings presented briefly, there appears to be no region of transparency. For squarewaves gratings, the perception of coherence gives way to the perception of transparency at disparities that are subject to several influences. Large ratios of horizontal-to-vertical plaid disparity components are more likely to cohere and those with small ratios are more likely to segregate. Orthogonal and near-orthogonal grating orientations make coherence more likely. Near-horizontal zero-disparity gratings are more likely to cohere with other gratings than are non-horizontal gratings. 

Is there a single factor underlying these diverse effects? We think that plaid disparity is such a factor : plaid disparities that are small and horizontal lead to depth coherence and those that are large and non-horizontal lead to transparency. This is consistent with the effects of relative grating orientation in addition to the effect of the size and direction of plaid disparities. For gratings having different horizontal disparities, it correctly predicts that orthogonally oriented gratings, which minimize plaid disparity magnitude for constant grating disparities, will be perceived as relatively coherent and that gratings of similar orientations, which produce large plaid disparities, will tend to be perceived as transparent. It also predicts the effect of non-target grating orientation—near-horizontal non-targets are associated with perceived coherence and near-vertical non-targets with perceived transparency—because the plaid’s disparity direction is parallel to the non-target grating. 

Why is there this particular association between plaid disparity and the perception of coherence and transparency? We suggest that the question the visual system is trying to answer when deciding between coherence and transparency is whether the presented 2D disparity—the disparity of the coherent components—is intrinsic or incidental: intrinsic to an object imaged under the current viewing conditions or an incidental alignment of multiple objects, an artifact of aperture viewing. A coherent object is the preferred interpretation in the former case, and overlaid objects, either transparent or opaque depending on surface interactions, in the latter case. The question must arise often in naturalistic visual conditions, where aperture viewing can transform horizontal or epipolar disparities into the full array of 2D-disparity directions (Farell, 1998; Malik, Anderson, & Charowhas, 1999). The problem is typically ambiguous, with no unique solution. To find a solution that most likely accords with the state of the world, the visual system may adopt a bias favoring a coherent-depth interpretation for disparities that are small and horizontal and a transparent interpretation otherwise. 

Many of the limits seen or supposed in stereoscopic vision—nearest-neighbor matching, the size-disparity correlation, the smoothness and opacity constraints, as well as disparity averaging, depth capture, and depth coherence—can be understood as the outcome of adapting a prior probability favoring small disparities, absolute and relative. When applied to superimposed gratings, such a prior supports depth coherence. This is so because the range of disparities of single surfaces (the coherence option) is restricted compared to the range of aperture disparities generated by multiple superimposed surfaces (the transparency option). The size of an aperture disparity depends jointly on the disparities and relative orientations of the contours (Farell, 1998, 2003; Malik, et al., 1999; van Ee et al., 2001). Even if all of the superimposed surfaces (e.g., gratings) have small disparities, they can still produce an arbitrarily large aperture disparity at their intersecting or overlapping contours, provided only that at least one of the surface disparities is non-zero. The distinction applies to disparity direction, as well. In naturalistic viewing conditions, single-surface disparities have an anisotropic distribution of directions, greatly compressed in directions near the vertical; by contrast, the direction of aperture disparities created from overlapping surfaces will be distributed nearly uniformly. These different probability distributions are consistent with a Bayesian prior that associates coherent depth with small and horizontal 2D pattern disparities, which in turn is consistent with the plaid-depth hypothesis: The disparities are those of plaids, not gratings. 

What appears discrepant are data for plaids with horizontal target gratings. Horizontal targets cohere with vertical non-targets; they also cohere with oblique non-targets that segregate from other targets (e.g., 0° target, 45° non-target, and 90° target, 45° non-target plaids; see Figure 8). In both cases, the vertical disparity of the coherent plaid is a large proportion of the horizontal disparity, a condition that should support transparency, not coherence. Yet it is clear that these cases require special treatment. The horizontal component of the disparity of horizontal contours is indeterminate and the vertical component is problematic; it cannot be attributed to aperture viewing and its most likely source is the non-horizontal disparity of the object to which contour belongs (as would be produced by a vertical fixation disparity). Because horizontal contours carry no information about depth in the absence of the support offered by non-horizontal contours of the same object, opting for coherence over transparency may be an optimal default strategy in dealing with them. 

The sinewave-squarewave difference

A number of models of stereoscopic vision incorporate inhibition between units tuned to different disparities (e.g., Nelson, 1975; Marr & Poggio, 1976; Sperling, 1970). This helps to solve the correspondence problem but impedes the perception of transparency (Marr, Palm, & Poggio, 1978; Akerstrom & Todd, 1988; Prazdny, 1985). Depth coherence, viewed as a failure to perceive transparency, might also be an outcome of disparity inhibition. If so, the dependence of depth coherence on component spatial frequency (Farell & Ahuja, 1996; Farell, 1997) and orientation (Figure 8) suggests that disparity inhibition is tuned to these variables. But why should disparity inhibition, or other possible causes of depth coherence, operate on sinewave gratings differently from squarewave gratings or other broadband patterns, such as random-dots, that can be perceived as transparent (Akerstrom & Todd, 1988; Prazdny, 1985; Weinshall, 1989; Ahuja & Farell, 1997, 1998a, 1998b, 1999)? 

Sinusoidal gratings of the same orientation and separated by two octaves in frequency are seen to cohere in depth when their disparities are equal. They continue to cohere even at rather large disparities where, because of the frequency difference, the gratings appear on opposite sides of fixation when viewed separately (Farell, Li, & McKee, 2004). The same applies to a squarewave grating. Because of the components’ frequency difference, only a subset possesses a ± half-cycle disparity range that includes the disparity of the grating as a whole. Thus low-frequency channels may not respond to small grating disparities and high-frequency channels may respond inconsistently or ambiguously to large disparities. Therefore, a squarewave grating (or other broadband pattern) depends on channel interactions to be perceived as coherent in depth.4 If the interactions are disparity selective, operating only among the channels with similar frequency tuning that have the smallest disparity difference between them, then components within a pattern will cohere and components from different patterns will not cohere. The result of this relative-disparity minimization is transparency, except between two similar-frequency sinewaves, which will cohere. This scheme has some structural affinities with the PMF algorithm (Pollard et al., 1985), which uses disparity gradients to iteratively select corresponding features from the set of possible matches. PMF can recover transparency; however, it has no mechanism for segregating squarewaves while causing sinewaves to cohere. 

Our data suggest that given a large-enough disparity difference between them, almost any briefly presented squarewave grating will segregate in depth from another squarewave grating with differing orientation. Sinewave gratings are governed by the obverse scenario, cohering across all disparity differences. A systematic exploration of the difference between the two will likely reveal a complex dependence of coherence on stimulus spatial frequency, orientation, the bandwidths of spatial frequency and orientation, and disparity. For example, sinewave gratings with the spatial frequency and contrast ratios of the fundamental and third harmonic components of squarewave gratings tend to be perceived as segregated in depth from a similar superimposed compound grating differing in orientation and disparity (Farell & Ahuja, 1996), but they tend to cohere when their orientations are the same (Li & Farell, 2002). Here, too, a solution can be sought in a generalized expectation for discrete objects to have small, predominately horizontal disparities. 

Akerstrom and Todd (1988)

Akerstrom and Todd (1988) found no effect on depth segregation of an orientation difference between surfaces. However, their stimulus was quite different from ours and so is the meaning of their data. They made stereograms from elements—line segments—that were spatially local and 2D. Correlating the lines’ orientation with disparity had no effect on perceived segregation; it didn’t matter whether the near surface had one line orientation and the far surface had another, rather than both surfaces having the same mixture of two orientations. The purpose of testing the effect of correlation was to see whether surfaces would segregate more readily if they differed along more dimensions than disparity alone. Orientation in Akerstrom and Todd’s study, unlike orientation in the experiments reported here, did not systematically affect the 2D spatial properties of the monocular stimuli or the disparities between them. For the purposes of their experiment, orientation could have been replaced by a non-spatial dimension—as, in fact, it was: Akerstrom and Todd (1988) found that correlating color with disparity did affect depth segregation, enhancing it. In our experiment, a change in grating orientation changed the spatial properties of the plaid, including its disparity magnitude and direction. The transition between coherence and transparency observed here, unlike that of Akerstrom and Todd (1988), was mediated by a change in disparity, with disparity direction setting the gain for changes in disparity amplitude. 

Acknowledgments

Supported by National Eye Institute Grant EY R01-12286 to BF. 

Commercial relationships: none. 

Corresponding author: Bart Farell. 

Email:  

Address: Institute for Sensory Research, Syracuse University, Syracuse, NY. 

Footnote

Footnotes

 Seeing transparent surfaces as opaque is certainly less frequent in naturalistic viewing conditions than the inverse—seeing opaque surfaces as transparent, which arises from diplopia and, relatedly, from uniocular viewing, the state from which we almost continuously see the sides of our nose.

Footnotes

²  Large grating disparities (e.g., phase offsets of 150°) can result in an indistinct appearance of depth or an unstable or ambiguous depth percept, rather than the vivid depth typically seen with moderate disparities, yet it was the plaid as a whole that appeared indistinct, unstable, or ambiguous, not just the grating with non-zero disparity.

Footnotes

³  The fit is not so good at 90°—orthogonal gratings. This is not a fluke. We have consistently observed that near-orthogonal squarewave gratings cohere at a larger disparity than expected from their difference in disparity directions, regardless of the absolute directions of these disparities. Orthogonally oriented patterns have been observed to interact (e.g., by cross-orientation inhibition or sharpening of selectivity) in extracellular recordings (Bishop, Coombs, & Henry, 1973; Bonds, 1989, Dragoi, Sharma, Miller, & Sur, 2002) and in psychophysics (Olzak & Thomas, 1991; Clifford, Wyatt, Arnold, Smith, & Wenderoth, 2001; Dragoi et al., 2002; Yu, Klein, & Levi, 2002), and for both simultaneous and sequential grating presentations. As described in the Discussion, we think our data can be explained without reference to any of these effects.

Footnotes

⁴  A possible constraint on the pattern is that its Fourier spectrum be continuous on the scale of the bandwidths of the channels on which the pattern information is processed (Farell et al., 2004).