Evidence for relative disparity matching in the perception of an ambiguous stereogram

Abstract

To compute depth from binocular disparity, the visual system must correctly link corresponding points between two images, given multiple possible correspondences. Typically, model solutions to this problem use some form of local spatial smoothing, with many physiologically inspired models doing so implicitly, through the use of local cross-correlation-like procedures. In this paper we show that implicit smoothing, without the explicit consideration of relative disparity, cannot account for biases in the perception of a novel ambiguous stereo stimulus. Observers viewed a stereogram consisting of multiple strips of periodic random-dot patterns, perceived as either a slanted surface, or a triangular wedge in depth, and reported their perception in a 4AFC task. Biases in the perception of this stimulus are shown to depend upon the stimulus configuration in its entirety, and cannot be accounted for by low-level preferences for disparity sign. Such results are not consistent with local smoothing effects arising solely at the level of cross-correlation-like absolute disparity detectors. Instead, our results suggest the presence of smoothing constraints that consider the differences in disparity between neighboring image regions. These results further suggest that such smoothing generally biases matching toward solutions that minimize relative disparity, regardless of the presence of changes in disparity sign.

Introduction

Two front-facing eyes, with overlapping visual fields, afford the human visual system two distinct views of any single scene. The small differences between the images arising from these two views are known as binocular disparities, and are highly informative of the three-dimensional structure of a scene. In order to make use of this information, disparate points in the two images, corresponding to the same single point in three-dimensional space, must be correctly linked together. This is problematic in the case of complex images, such as those arising from natural scenes, as the correct set of corresponding points must somehow be chosen amongst multiple possible correspondences. Deciding which points to link together in binocular image pairs is known as the correspondence or stereo matching problem, and is a central issue to be addressed for biological and artificial visual systems seeking to exploit binocular disparity information (cf. Julesz, 1971; Marr & Poggio, 1979; Scharstein & Szeliski, 2002). 

Finding the correct solution to the correspondence problem requires the use of prior constraints in order to limit the disparity to a subset of possible matches. These constraints are often grounded in a conception of the regularity of the natural world. Common examples are constraints derived from the notion of a continuity principle (Marr & Poggio, 1976, 1979). This continuity principle states that disparity should vary smoothly across the image since “… matter is cohesive, it is separated into objects, and the surfaces of objects are generally smooth” (Marr, 1982, p. 113). 

Physiologically inspired models of disparity measurement tend to consider the correspondence problem in terms of the responses of populations of disparity energy units (Ohzawa, DeAngelis, & Freeman, 1990, 1996, 1997). These model disparity detectors have been shown to be excellent predictors of the responses of disparity selective neurones in V1 (Cumming & DeAngelis, 2001, although see Read, Parker, & Cumming, 2002, and Haefner & Cumming, 2008). In disparity energy based models, the correspondence problem is solved by eliminating multiple false peaks in the response of a population of energy units tuned to different disparities. Multiple peaks arise in the disparity energy response due to both the quasi-periodic nature of the filter responses, and the structure of the stimuli. 

Elimination of false peaks is often achieved by applying a spatial weighting function (Fleet, Wagner, & Heeger, 1996; Qian, 1994; Qian & Zhu, 1997). This offers a means of applying implicit, continuity derived, local smoothing constraints to the disparity measurement process, and is functionally equivalent to a local cross-correlation (Fleet et al., 1996). Such implicit smoothing tends to bias disparity measurement toward solutions that minimize relative disparity locally. This biasing is achieved despite the fact that energy based disparity detectors only measure absolute disparity (i.e. they measure disparity in retinal coordinates such that measured disparity changes with changing fixation). 

The receptive field structures of binocular neurones implement local smoothing functions directly. Binocular simple cells in V1 tend to have receptive fields with identical orientation and spatial frequency tuning in each eye (Nienborg, Bridge, Parker, & Cumming, 2004), differing only by a phase or position-shift, equivalent to a translation of one receptive field relative to the other. Similarly, binocular energy based models of disparity measurement usually make use of model neurones with such characteristics (e.g. Fleet et al., 1996; Prince & Eagle, 2000; Qian & Zhu, 1997). As a consequence, these neurones will respond most strongly to appropriate stimuli containing purely translational disparities, in which the image sample drawn from one eye's image corresponds to an identical sample drawn from the other eye's image, with the difference in location of the samples corresponding to the disparity tuning of the each neurone. Such responses will be reduced when corresponding regions of the two images are related by more complex transformations. This preference for translational disparities is dependent on identical orientation and spatial frequency tuning in the two eyes' receptive fields (i.e. on receptive fields that differ only by a translation). Other transformations, such as a rotation, contraction or shearing of one eye's image relative to the other, introduce differences in the orientation or size of features between the two images. Responses to such images could be maximized if orientation or spatial frequency tuning also differed between the two eyes. 

Local weighted spatial summation of disparity energy responses results in a calculation equivalent to local cross-correlation (Fleet et al., 1996). Banks, Gepshtein, and Landy (2004) have shown that the limit on the frequency of cyclopean depth modulations that can be seen is well predicted by a model that measures disparity through cross-correlation. Similarly, Filippini and Banks (2009) have shown that the performance of such a model is determined by the disparity gradient in the stimulus, and not by absolute disparity or cyclopean frequency. This disparity gradient limit to performance is well matched by human data (Burt & Julesz, 1980; Filippini & Banks, 2009) and is a consequence of the preference for translational disparities inherent in the cross-correlation approach. As disparity gradient increases, the variation in disparity within the correlation window will also increase. This reduces the maximum correlation between samples from the two eyes, thereby increasing the false matching problem. 

Given the ability of the energy model to predict the responses of disparity selective V1 neurones, and the relative success of disparity energy based models of correspondence matching, it would seem that many aspects of smoothing in disparity processing occur at the level of V1 neurones. However, strong responses can be elicited in V1 neurones tuned to disparities that do not correspond to perceived depth (Cumming & Parker, 2000). The perception of depth on the basis of disparity is therefore not fully determined until after this processing stage. 

Results from single cell recording studies have shown that, unlike disparity selective cells in V1, neurones in later visual areas respond to relative disparities (Bredfeldt & Cumming, 2006; Cumming & Parker, 1999; Thomas, Cumming, & Parker, 2002). Relative disparity is the difference between the absolute disparities of two points. Many neurones in V2 respond more strongly to the presence of an appropriate cyclopean edge than they do to a region of uniform disparity. These relative disparity effects can therefore be contrasted with effects such as the disparity gradient limit, which has been accounted for in terms of the expected deterioration in the encoding of absolute disparity arising from the presence of local disparity variation, as discussed above. Unlike absolute disparity, relative disparity does not change with changing fixation. Such results suggest that the correspondence problem is solved either in relative disparity coordinates or in the transformation to such coordinates, rather than in the absolute disparity coordinates of the energy model. This view is also supported by psychophysical results (Glennerster, McKee, & Birch, 2002; Petrov & Glennerster, 2006). 

If relative disparity encoding does play a role in solving the correspondence problem, then one may expect additional spatial smoothing at this later stage of processing, in addition to that implemented at an initial, energy-based stage. That is, one may expect to find spatial smoothing mechanisms that consider the differences in disparity between multiple detectors. While recent, physiologically inspired, models of disparity measurement tend to rely on spatial smoothing mechanisms that make no explicit measurement of relative disparity (Banks et al., 2004; Chen & Qian, 2004; Qian & Zhu, 1997; Read, 2002), many early, feature-based, models used explicit relative disparity defined constraints to solve the correspondence problem (e.g. Marr & Poggio, 1976, 1979; Pollard, Mayhew, & Frisby, 1985). This is an important consideration when comparing models. For disparity energy or cross-correlation derived models, there is no concept of smoothness or surface continuity beyond the extent of the largest local cross-correlation window size. For such models, there is also no explicit rule to determine which match is preferred, given multiple ‘smooth’ correspondence solutions. 

In this paper, we examine whether there is indeed a role for relative disparity processing in the resolution of the stereo correspondence problem. We use a novel ambiguous stereogram to address this issue (see example in Figure 1a). The stimulus is comprised of multiple periodic random-dot strips, and is balanced across four alternative solutions to the correspondence problem. Observers perceive this stimulus as either a convex or concave wedge shape, or as a ground (top-far) or ceiling (top-near) slanted plane (Figure 1b). The balancing of matching alternatives means that supporting context (i.e. local structure) is available, in equal measure, for multiple solutions to the correspondence problem. As such, this stimulus presents difficulties for models of disparity estimation that rely on constraints and measurements defined only in terms of absolute disparity. 

Figure 1

 

(a) Example of the experimental stimulus, at a luminance ambiguity of zero. In this example, the stimulus is exactly balanced between all dominant correspondence solutions. Readers able to free fuse should see one of the correspondence matches shown in (b). These consist of two ‘wedge’ type matches (upper row), and two ‘slanted plane’ matches (bottom row). Observers report the perception of either a convex or concave wedge, or a ground or ceiling plane.

The results of a local cross-correlation model of disparity measurement, detailed below, are equally compatible with each of these solutions. Therefore, any observer biases, indicating a preference for some solutions over others, must reflect constraints in addition to those embodied in this model. One simple possibility is that there might be a preference for some values of absolute disparity over others. For example, Hibbard and Bouzit (2005) demonstrated a matching bias in ambiguous stereograms that depended on the location of the stimulus in the visual field: observers tended to prefer matches such that targets in the top half of the visual field were seen as farther than fixation, and targets in the bottom half as closer. 

Such disparity sign biases would necessarily be independent of the stimulus presented to an observer. As such, if disparity sign biases, rather than explicit relative disparity biases, are used to solve the correspondence problem in our ambiguous stimulus, one should be able to predict observers' matching biases from those arising when the constituent parts of the stimulus are presented in isolation. In this paper we measure observers' matching biases with both our complete, four-way ambiguous stimulus, and with top and bottom halves of the stimulus independently. We show that biases arising from half-stimulus presentation cannot be used to successfully predict matching bias with the complete ambiguous stimulus. Our results therefore suggest that the human visual system solves the stereo correspondence problem through the consideration of relative disparity. 

There are various ways in which such relative disparity biases could operate. In the stimulus shown in Figure 1a, alternative matching solutions are consistent with distinct definitions of ‘smooth’ matching. Matches to either the convex or concave wedge solutions (Figure 1b, top row) allow for the minimization of relative disparity at the expense of a discontinuity in depth at the point of the wedge. Conversely, matches to either the ‘ground’ or ‘ceiling’ plane solutions (Figure 1b, bottom row) allow for the minimization of discontinuities in depth, at the expense of a greater overall change in disparity in the stimulus. Wedge solutions therefore minimize relative disparity, at the cost of introducing a change in the sign of disparity gradient from one region of the image to another. Alternatively, slanted plane solutions minimize changes in sign, but increase the magnitude of relative disparity. It is not immediately apparent which ‘smooth’ solution should be preferred. 

Current psychophysical evidence suggests that the visual system makes use of local biasing toward minimal relative disparity matching solutions, but only to the extent that such solutions are favored over nearest neighbor matches (Goutcher & Mamassian, 2005; Zhang, Edwards, & Schor, 2001). This observed preference for minimal relative disparity matching, over nearest neighbor matching, is consistent with the proposed use of cross-correlation-like mechanisms for the encoding of binocular disparity. Within these stimuli, minimal relative disparity solutions offer close to the pure translational disparities favored by cross-correlation mechanisms. This is not the case for the nearest neighbor matches. Such matches require opposing shifts and so contain shearing or expansion-compression transformations. 

Cross-correlation mechanisms cannot, in isolation, account for biases in the perception of stimuli such as that shown in Figure 1a, due to the balancing of alternative matching solutions. One may, however, expect subsequent relative disparity processing stages to further constrain matching in a manner consistent with the biases arising from earlier processing, under the assumption that the visual system would not indulge in computations that it must later undo. Here, we examine whether, in resolving the correspondence problem in our balanced ambiguous stimulus, the visual system continues to prefer the minimization of relative disparity, or whether it switches to a bias that minimizes the perception of depth discontinuities (i.e. minimizes changes in disparity gradient sign). 

The ambiguous stereogram

The novel ambiguous stereogram used in this study comprised a series of periodic random-dot strips (Figure 1a). Each strip was 6.3° long and 21 arcmin high, at the 53 cm viewing distance. Each strip was windowed by a horizontal Gaussian distribution of standard deviation 1.58°. Neighboring strips were separated by a gap of 6.3 arcmin. The full experimental stimulus comprised seven strips, and thus measured 6.3 × 3.08°. Individual pixels subtended 2.1 arcmin. 

Each stimulus strip was constructed by creating repeated sequences of paired random-dot units, a and b. Random-dot patterns were binary with a 1:1 ratio of black to white pixels. Each unit extended to the full height of the stimulus strip. Unit width varied between strips, depending on the required disparity magnitude (see below), but was always constant within each strip. In one eye, repeated sequences were paired in the order ab, while in the other eye, the repeated order was ba (see Figure 2a). This reversal of sequence order produces a reliable matching ambiguity in the stimulus, where an ab pair in one eye may match to an ab pair with either crossed or uncrossed disparity, of equal magnitude, in the other eye. 

Figure 2

 

Illustration of the general structure of the stimulus and the manipulation of luminance ambiguity. (a) The stimulus is built of random dot units a and b, where a and b are of equal size but contain different binary random dot patterns. In one eye, these units are arranged in a repeating ab sequence. In the other eye, they are arranged in a repeating ba sequence. By adjusting the mean luminance of ab units in both eyes, correspondence matching can be biased towards uncrossed disparities, as in the outward shift in (b), or towards crossed disparities, as in the inward shift in (c).

The magnitude of the resulting disparity is equal to half the total size of the ab pair (i.e. equal to the size of a single a or b unit). Unit width, and therefore disparity magnitude, increased with the strip's vertical eccentricity. Flanking strips closest to the central strip consisted of ab pairing of width 8.4 arcmin. The next most eccentric strips consisted ab pairings of width 16.8 arcmin. Finally, the most eccentric strips consisted of ab parings of width 25.2 arcmin. ab pairings thus varied between 4 × 10 and 12 × 10 pixels in size. The central strip was non-periodic (i.e. was composed of a single extended random-dot pattern, not a repeating sequence of ab pairings), with no experimentally introduced disparity. 

Given this arrangement of ab width and eccentricity, disparity magnitude in the stimulus increases from zero in the central strip, to ±4.2, ±8.4 and ±12.6 arcmin with increasing strip eccentricity. This results in four possible global matching solutions. These four alternative stimulus interpretations are illustrated in Figure 1b. Alternative stimulus interpretations fall into two main categories: observers perceive the stimulus in either a ‘wedge’ or ‘slanted plane’ configuration. In the ‘wedge’ category each strip is matched in the same direction, while in the ‘slanted plane’ category the direction of matching reverses either side of the central, zero disparity strip. These two main categories can then be subdivided into two further categories, depending on matching directions. Wedges are perceived as either convex or concave, while slanted planes are either top-far, ‘ground’ planes, or top-near, ‘ceiling’ planes. 

The stimulus so far described is wholly ambiguous with regards to matching. There is no information in the stimulus biasing matching toward one solution or another. Such bias can be introduced by adding a luminance pedestal to specific sets of ab pairings (Figures 2b and 2c). If an increase in mean luminance is applied to alternate sets of ab pairings in both eyes, with a decrease in mean luminance applied to neighboring ab pairings, one would expect matching to be biased toward the solution that maximizes luminance similarity between the two eyes (Goutcher & Mamassian, 2005). The direction of the preferred match would be dependent upon the phase of luminance modulation applied to each stimulus strip, in each eye. The magnitude of luminance biasing would be dependent upon the amplitude of the luminance modulation. Figure 2b illustrates an example of a luminance modulation applied in the uncrossed disparity direction. As the reader will note, here ab pairings in the left eye (upper row) match with right eye (lower row) ab pairings shifted to the right (i.e. an outward shift). This leaves a single unmatched b unit on the left of the right half image, and a single unmatched b unit on the right of the left half image. Figure 2c illustrates an example of a luminance modulation applied in the crossed disparity direction. Here ab pairings in the left eye match with ab pairings in the right eye shifted to the left (i.e. an inward shift). In this case a single a unit is left unmatched on the left of the left half image and on the right of the right half image. In both crossed and uncrossed disparity cases, unmatched regions conform to the expected monocular regions arising in the event of half-occlusions (Shimojo & Nakayama, 1990). 

We shall refer to our patterns of luminance modulation, which bias matching toward crossed or uncrossed disparities, as the luminance ambiguity level. Luminance ambiguity varies between ±1, with the magnitude of the luminance ambiguity value indicating the amplitude of luminance modulation. For a luminance ambiguity of zero, the mean luminance of each ab pairing is 11.7 cdm⁻². For a luminance ambiguity of ±1, the mean luminance of the ab pairings will be either 6.6 cdm⁻² or 16.8 cdm⁻². 

For half stimulus presentation, positive luminance ambiguity values indicate a bias toward the top-far ‘ground’ plane solution. Negative values indicate a bias toward the top-near ‘ceiling’ plane solution. For whole stimulus presentation, luminance ambiguity can be defined on two axes, the wedge axis and the slant axis. On the wedge axis, negative luminance ambiguities indicate a bias toward the concave wedge solution, while positive luminance ambiguities indicate a bias toward the convex wedge solution. On the slant axis, negative luminance ambiguities indicate a bias toward the ‘ceiling’ plane solution, while positive luminance ambiguities indicate a bias toward the ‘ground’ plane solution. 

The cross-correlation model

Before examining observers' performance in our psychophysical task, we begin by analyzing the correspondence problem in our ambiguous stimulus. To do so, we first find disparities consistent with our stimulus, using a normalized local cross-correlation approach (Banks et al., 2004; Filippini & Banks, 2009; Palmisano, Allison, & Howard, 2006). In order to measure correlation between local image patches at different disparities, each half image I _L and I _R is first windowed by Gaussian envelopes W _L and W _R , defined, as: 

$\begin{array}{c}{W}_{L(x,y;x_0,y_0,\delta )}=e^{\frac{-{(x-x_0+\frac{\delta }{2})}^2-{(y-y_0)}^2}{2{\sigma }^2}}\\ W_{R(x,y;x_0,y_0,\delta )}=e^{\frac{-{(x-x_0+\frac{\delta }{2})}^2-{(y-y_0)}^2}{2{\sigma }^2}}\end{array},$

(1)

where σ is the standard deviation of the Gaussian, and determines the size of the window. x ₀ and δ determine the horizontal location of each envelope, and y ₀ determines its vertical location. δ determines the disparity in the position of the envelope between the two eyes. Given these envelopes, we can define local windowed image regions R _L and R _R as the product of the images I _L and I _R and the Gaussian windowing functions, centered on the image coordinates x _L and X _R where: 

$\begin{array}{c}{\mathbf{x}}_L=(x_0-\frac{\delta }{2},y_0)\\ {\mathbf{x}}_R=(x_0+\frac{\delta }{2},y_0)\end{array}.$

(2)

This gives us the following equations for R _L and R _R , the windowed local image regions in left and right half images: 

$\begin{array}{c}{R}_L(x,y;x_0,y_0,\delta )=I_L(x,y)W_L(x,y;x_0,y_0,\delta )\\ R_R(x,y;x_0,y_0,\delta )=I_R(x,y)W_R(x,y;x_0,y_0,\delta )\end{array}.$

(3)

 

The normalized cross-correlation for these windowed image regions, at the disparity δ is then calculated using the following equation: 

$c(x_0,y_0,\delta )=\frac{\sum _{(x,y)}\left[R_L\right(x,y;x_0,y_0,\delta )-{\mu }_L]\left[R_R\right(x,y;x_0,y_0,\delta )-{\mu }_R]}{\sqrt{\sum _{(x,y)}\left[R_L\right(x,y;x_0,y_0,\delta )-{\mu }_L{]}^2}\sqrt{\sum _{(x,y)}\left[R_R^2\right(x,y;x_0,y_0,\delta )-{\mu }_R{]}^2}},$

(4)

where μ _L,R is the mean intensity of the image region. 

Figure 3 shows a cross-section of the output from the local cross-correlation model, when presented with an example of our four-way ambiguous stimulus at a luminance ambiguity of zero. As can be seen, the cross-correlation procedure has no problem extracting the x-shaped cross depicting the disparities available in the stimulus. For each location in the image, there are therefore at least two disparity values producing equally strong correlations, and there is no straightforward means to select between the two in order to solve the correspondence problem. 

Figure 3

 

(a) Cross-section of the output of a normalized cross-correlation model of disparity measurement applied to the experimental stimulus. The standard deviation of the cross-correlation window was 4.2 arcmin. Readers should note that disparities consistent with the stimulus are easily extracted, but that no unique solution to the correspondence problem is evident. (b) Depth map obtained from (a), given a decision rule where the disparity with the greatest correlation output is chosen. Note that the obtained depth map does not conform to any of the observed percepts.

Let us assume that the cross-correlation output is the result of a process that measures the luminance of each point in the image, but that this measurement is affected by Gaussian distributed random noise. This noise gives us some small degree of variability in correlation output that would not otherwise be present. If we now use a simple decision rule, where the chosen disparity at any given image location is the disparity with the largest correlation value, we can obtain a disparity estimate for each point in the scene. However, there is no guarantee that such estimates will consistently match any of the stimulus interpretations commonly perceived by human observers (see Figure 3b). 

We can, however, further constrain the cross-correlation model to produce outputs equivalent to the perceptions of human observers through the use of template matching procedures (e.g. Filippini & Banks, 2009; Wallace & Mamassian, 2004). In applying a template matching approach, it can be straightforwardly shown that the stimulus is unbiased with regards to the probability of choosing each of the four alternative templates. To do this, the cross-correlation output was calculated for 1200 sample images at a range of luminance ambiguity values, each corrupted by additive Gaussian noise of mean zero and standard deviation 0.09 cdm⁻². A correspondence solution was then chosen by taking the sum of the product of the cross-correlation output with each of the four templates, and selecting the one that gave the greatest response. The templates corresponded to the four interpretations of the stimulus (ground plane, ceiling plane, convex wedge, concave wedge), having a value of one at the appropriate disparities, and zero everywhere else. 

Figure 4 shows the probability of choosing each of the four templates, given different levels of luminance ambiguity. Figures 4a and 4b show that luminance ambiguity manipulations very quickly bias the response of the cross-correlator toward a single percept. At a luminance ambiguity of zero (Figure 4c), matching preference is not significantly different from chance (i.e. the probability of obtaining each percept is 0.25). This absence of matching bias is found regardless of correlation window size, indicating that a lack of matching bias is not simply indicative of the absence of disambiguating information within the window boundaries. 

Figure 4

 

Confirmation of the lack of bias in the cross-correlation output. (a) Effects of wedge axis luminance ambiguity manipulations. Negative luminance ambiguities bias the stimulus towards the concave wedge solution. Positive luminance ambiguities bias the stimulus towards the convex wedge solution. (b) Effects of slant axis luminance ambiguity manipulations. Negative luminance ambiguities bias the stimulus towards the ‘ceiling’ plane solution. Positive luminance ambiguities bias the stimulus towards the ‘ground’ plane solution. (c) At a luminance ambiguity of zero, the probability of obtaining each template match is not significantly different from 0.25, regardless of correlation window size.

It should be noted that a lack of matching preference is not limited to the ambiguous stereogram presented in this paper. The inability of cross-correlation models to match only to those correspondence solutions perceived by human observers is also a general problem. These problems will arise with any periodic stimulus that contains only 180° phase disparities, even if the angular magnitude of those disparities changes across the vertical extent of the image. 

Consider any vertical sinusoidal grating pair placed in anti-phase between the two eyes. Such an image pair will produce disparities equal to ±1 wavelength, and so on in multiples of wavelengths. If we multiply the spatial frequency of this grating by a value b in half of the stimulus, it will now contain disparities equal to ±1/b of the original wavelength. So, either side of fixation, we now have disparities equal to 1/b and 1 original wavelength, plus all multiples of these, although only disparities of ±1/b will tend to be perceived, consistent with a preference for small disparities (Banks & Vlaskamp, 2009; Prince & Eagle, 2000; Qian & Zhu, 1997; Read, 2002). This means that, no matter the point in the image, a local cross-correlator will always have the same information telling it that the disparity is in front of fixation as it does that the disparity is behind fixation. Note that this is the case no matter the base spatial frequency of the stimulus, or the magnitude of the change in spatial frequency, provided that the anti-phase relationship between eyes is maintained, and that changes in spatial frequency only occur vertically. A change in spatial frequency along the horizontal will result in contraction and expansion disparities that will reduce the response of a cross-correlator. The stimulus used in this paper is a specific example of this general case. 

A simple cross-correlation based model predicts no preference for any of the possible interpretations of our ambiguous stimulus over any other. Any response bias shown by human observers would demonstrate the presence of additional constraints on the matching process. At their simplest, such constraints could take the form of a preference for small disparities as mentioned above, or for one sign of disparity over the other. Hibbard and Bouzit (2005) have shown that matching bias in an ambiguous stereogram is affected by the height of the stimulus in the visual field and by the distance at which it is viewed. Similarly, a bias for small disparities has been observed in psychophysical studies (Mallot & Bedeau, 1990; McKee & Mitcheson, 1988) and applied to models of disparity measurement (e.g. Lages, 2006; Prince & Eagle, 2000; Read, 2002). 

If bias in the perception of our ambiguous stereogram is explainable purely through the operation of simple disparity sign biases, then any bias observed when participants were presented with the whole stimulus in the 4AFC experiment, should reflect the bias observed when participants were presented with each half of the stimulus independently. In other words, matching bias should not depend upon the entire stimulus configuration. Response biases that could not be predicted in this way would indicate the presence of more complex constraints, taking into consideration the broader context of potential disparities present in the stimulus. Psychophysical experiments were performed to determine whether, and in what form, such constraints are evident in the interpretation of our ambiguous stereogram. 

Psychophysical experiments

Psychophysical methods

Apparatus

Stimulus display and data collection were controlled by an Apple G4 computer, running Matlab (copyright Mathworks Inc), in combination with the Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). Stereoscopic stimuli were displayed using a mirror stereoscope, in a single reflection, two-monitor configuration. Each half-image was displayed on a NE AccuSync LCD72VM monitor, running at a resolution of 1024 × 820 pixels. Monitors were calibrated using the Spyder2Pro calibration device (copyright Datacolor Inc) to ensure linear grayscales over identical luminance ranges. Luminance calibration was checked using a Minolta LS110 luminance meter. Stimuli were presented at a viewing distance of 53 cm. At this distance a single pixel subtended 2.1 arcmin. Participants completed the experiment in a darkened room with head movements restricted using a chinrest. 

Design and procedure

Observers were asked to fixate at the center of the stimulus, and were presented with a reference marker for 1600 ms prior to the presentation of the stimulus, in order to aid fixation. Like the central strip of the ambiguous stimulus, the fixation marker had no disparity offset (i.e. it was presented at the depth of the screen). Each stimulus presentation lasted 600 ms. 

Participants completed the experiment in two parts. In one part, participants were presented with the complete, 4-way ambiguous, stereo stimulus. On such occasions, participants were required to report their perception of the stimulus in a four alternative forced-choice task (4AFC). In this task, participants responded as to whether they perceived the stimulus as a convex wedge, a concave wedge, a top-far slanted surface (a ‘ground’ plane) or a top-near slanted surface (a ‘ceiling’ plane). In the other section of the experiment participants were presented with either the top or bottom half of the stimulus, complete with the zero disparity central strip. In this case, participants were required to report their perception of the stimulus in a two alternative forced-choice task (2AFC). Participants responded as to whether they perceived the section of the stimulus they were presented with as a top-far or top-near slanted plane (a ‘ground’ or ‘ceiling’ plane, respectively). 

Blocks of the 2AFC and 4AFC experiments were randomly interleaved, with each participant completing at least 10 blocks of each experiment. This amounted to at least 40 repeated trials of each luminance ambiguity value, with at least 80 repeated trials at a luminance ambiguity of zero. Each participant was tested at 13 different luminance ambiguity values. In the 4AFC experiment these values were tested along both wedge and slant axes. In the 2AFC experiment the same range of luminance ambiguities was used for both top and both half presentations. The range of luminance ambiguities was varied between participants so as to obtain the full range of responses for the fitting of psychometric functions (although this was not possible for all observers). 

Prior to beginning the experiment, participants were presented with unambiguous versions of the stimuli, in order to familiarize them with the multiple perceptual interpretations. 

Participants

Ten observers participated in each part of the experiment, including both authors. The remaining eight participants were naïve with regards to the experimental design and hypotheses. Nine participants were experienced psychophysical observers. The remaining observer, an undergraduate student, had some limited experience in participating in psychophysical studies. All participants were taken from the population of staff and students at the University of St Andrews and had normal, or corrected-to-normal, visual acuity. 

Psychophysical results

Measuring matching bias

Figure 5 shows the probability of obtaining a response in line with each of the wedge and slanted surface percepts in the 4AFC task. As expected, the manipulation of luminance ambiguity succeeded in altering matching preference. However, effects of luminance ambiguity manipulation differed between observers, with only slight changes in matching preference found for two observers (Obs. 1 and 3). Figure 5a shows the effects of luminance ambiguity manipulation for one observer (Obs. 5). Results at a luminance ambiguity of zero, when the stimulus is objectively ambiguous, are shown in Figure 5b for all observers. At this luminance ambiguity, observer responses in the 4AFC task show marked matching preferences, although the direction and extent of these preferences differ between observers. The most common preference is for wedge solutions, although some observers (Obs. 2, 6 and 7) display a strong tendency for the perception of slanted plane solutions. Notably, the two observers who show a reduced effect of luminance ambiguity manipulation (Obs. 1 and 3) also show some of the strongest asymmetries in matching preference. 

Figure 5

 

Matching bias observed in the 4AFC task (a) Effects of luminance ambiguity manipulation on matching preference are shown for one observer (Obs. 5), as proportions of each category of response in the 4AFC task. Left panel shows effects of varying luminance ambiguity along the wedge axis. Right panel shows effects of varying luminance ambiguity along the slant axis. (b) Matching preferences for all observers at a luminance ambiguity of zero (i.e. where the stimulus is objectively ambiguous). Matching preference is shown as the proportions of each category of response in the 4AFC task.

In order to understand the direction of the observed biases, one could simply look at the proportion of wedge responses relative to slanted plane responses. However, if strong biases exist for a particular sign of match in a particular part of the image (e.g. for ground plane matches in the upper half of the stimulus), then a simple comparison of wedge and slanted plane responses would not prove sufficient. Given such a situation, asymmetric preferences for one particular wedge solution, or one particular slanted plane solution might give the appearance of a reduced preference for wedge or slanted plane type solutions. 

To counteract this possibility, we instead examine conditional probabilities for the perception of a ‘ground’ surface in one half of the stimulus ‘a’, given the perception of either a ‘ground’ or ‘ceiling’ surface in the other half of the stimulus ‘b’. Conditional probabilities were defined as follows: 

$\{\begin{array}{c}p(Groun{d}_a|Groun{d}_b)=\frac{1}{2}(\frac{N_{g}round}{N_{g}round+N_{c}oncave}+\frac{N_{g}round}{N_{g}round+N_{c}onvex})\\ \\ p(Groun{d}_a|Ceilin{g}_b)=\frac{1}{2}(\frac{N_{c}onvex}{N_{c}onvex+N_{c}eiling}+\frac{N_{c}oncave}{N_{c}oncave+N_{c}eiling})\end{array},$

(5)

where N_response indicates the number of responses of a particular type (ground, ceiling, convex or concave) in the 4AFC task. The first response proportion term in each of these equations gives the conditional probability for perceiving a ‘ground’ surface in the top half of the stimulus, while the second response proportion term gives the conditional probability for perceiving a ‘ground’ surface in the bottom half of the stimulus. 

The two conditional probability terms reflect the two categories of matching preference that may arise from our stimulus, consistent with two distinct forms of matching bias. If, for a given luminance ambiguity value, the conditional probability p(Ground _a ∣Ceiling _b ) is greater than the conditional probability p(Ground _a ∣Ground _b ), then matching is biased toward wedge solutions, and therefore toward the minimization of relative disparity. If the opposite relationship holds, matching is biased toward slanted plane solutions, and therefore toward the minimization of changes in disparity gradient sign. Note that the complementary conditional probabilities p(Ceiling _a ∣Ground _b ) and p(Ceiling _a ∣Ceiling _b ) may be readily derived from those reported. 

Conditional probability data are shown for one example observer (Obs. 5) in Figure 6a, together with fitted psychometric functions. This observer shows a marked difference between these two conditional probabilities, which may be quantified by finding the 50% threshold on the fitted psychometric functions (point of subjective equality—PSE). This point indicates the level of luminance ambiguity at which a ‘ground’ surface is perceived in one half of the stimulus on 50% of trials. If the PSE occurs at a lower luminance ambiguity level for the conditional probability p(Ground _a ∣Ceiling _b ) than for the conditional probability p(Ground _a ∣Ground _b ) then, given equal luminance ambiguity values, the observer is more likely to perceive a wedge shape than a slanted surface, indicating a preference for the minimization of relative disparity over the minimization of changes in disparity gradient sign. This observer shows a marked preference for the minimization of relative disparity. Together, these results show that all 10 observers showed a significant matching bias. For 8 of the observers, this bias was in favor of wedge interpretations, while the remaining 2 showed a significant bias in favor of planar interpretations. Overall, therefore, there was a significant tendency for observers to show a bias consistent with minimizing relative disparity. That the results for the 2 observers who did not follow this trend were significant, however, shows that these reflect genuine individual differences, rather than simple sampling error. 

Figure 6

 

Results of the conditional probability analysis. (a) Conditional probabilities obtained for a single observer in the 4AFC experiment. Each plot shows the effects of luminance ambiguity manipulation (x-axis) on the conditional probabilities p(Ground∣Ground) (y-axis, left-hand plot) and p(Ground∣Ceiling) (y-axis, right-hand plot). To measure matching preference from these data, the luminance ambiguity at the 50% point (point of subjective equality–PSE) is found, as is the conditional probability value when luminance ambiguity is equal to zero. (b) The difference in PSEs, for p(Ground∣Ceiling) − p(Ground∣Ground), was found for each observer. PSE differences indicate a significant preference, in 8 of the 10 observers, for p(Ground∣Ceiling) matches. (c) The difference in conditional probabilities p(Ground∣Ceiling) − p(Ground∣Ground) was found for each observer, at a luminance ambiguity of zero. Again, conditional probability differences indicate a significant preference, this time in 6 of the 10 observers, for p(Ground∣Ceiling) matches.

Figure 6b shows the differences between p(Ground _a ∣Ceiling _b ) and p(Ground _a ∣Ground _b ) PSEs for each of the ten observers. Negative values indicate a preference for matching to wedge shapes, while positive values indicate a preference for matching to slanted surfaces. Error bars show bootstrapped 95% confidence intervals. Eight of the ten observers show significant negative differences, while the remaining two observers show significant positive differences. Two of the eight observers with significant negative differences show very little effect of luminance ambiguity variation on their perception of the stimulus. Instead, psychometric functions for these observers show ceiling and floor performance. While this does indeed show a strong bias in the direction specified by a calculation of the difference between conditional probability PSEs, quantifying this difference in terms of extrapolated threshold values does not seem satisfactory. We therefore also offer an alternative means of analyzing differences between the two conditional probability measures. 

Matching bias can also be assessed through examination of the difference between conditional probabilities p(Ground _a ∣Ceiling _b ) − p(Ground _a ∣Ground _b ) when luminance ambiguity is equal to zero (i.e. when the stimulus is objectively ambiguous). The results of this analysis are shown for each observer in Figure 6c. In this case, negative values indicate a preference for matching to slanted surfaces, while positive values indicate a preference for matching to wedge shapes. Using this analysis, six of the ten observers show a significant positive difference, while two observers have a significant negative difference. These results concur with those of the threshold analysis. The remaining observers do not show significant differences for the zero luminance ambiguity analysis, although their biases were significant in the threshold analysis. A related samples t-test on the zero luminance ambiguity data shows a significant positive difference between conditional probabilities (t ₉ = 2.5574, p < 0.05) across all participants, indicating a significant bias for the minimization of relative disparity. 

Comparison of predicted and measured matching bias

The biases observed in the 4AFC task cannot be taken in isolation. Although the difference between conditional probabilities suggests that the entire stimulus configuration is considered in resolving the correspondence problem, it may still be the case that low-level biases are responsible for the observed matching preferences. In order to examine this possibility, we compare the results obtained in the 4AFC task, to those obtained in the 2AFC task, when observers were presented with independent halves of the ambiguous stimulus. In this experiment, observers were asked to state whether the half ambiguous stimulus appeared as a top-far slanted ‘ground’ plane, or a top-near slanted ‘ceiling’ plane. A summary of the results for these conditions is shown in Figure 7. 

Figure 7

 

Matching preferences shown by our observers for the presentation of the top and bottom half-stimuli. Figures show (a) the probability of ‘ground’ responses as a function of luminance ambiguity for top and bottom stimuli, for a single observer (b) the probability of obtaining a ‘ground’ response with the objectively ambiguous (i.e. zero luminance ambiguity) stimulus for top and bottom stimuli, for all observers.

Figure 7a plots the proportion of ‘ground’ responses as a function of luminance ambiguity level, for one example observer (Obs. 5). Results are plotted separately for stimuli presented in the top and bottom half of the screen. As can be seen, luminance ambiguity manipulation again altered matching preference. Interestingly, for some observers, effects of luminance ambiguity manipulation were different in each half of the stimulus. Several observers showed reduced effects of luminance ambiguity in one half of the stimulus, although the stimulus half that showed this reduced effect differed between participants. 

At a luminance ambiguity of zero, when the stimulus is objectively ambiguous, biases in observer responses can be taken as indicative of underlying biases in the correspondence matching process. Biases for all observers are shown in Figure 7b. Seven of the ten observers were biased toward ‘ground’ plane matches in the top half of the stimulus. Of those seven observers, five were biased toward ‘ceiling’ plane matches in the bottom half of the stimulus, with the other observers biased toward ‘ground’ plane matches. Of the three remaining observers, one was biased toward ‘ceiling’ matches in the top half of the stimulus, and ‘ground’ matches in the bottom half of the stimulus, while another was biased toward ‘ceiling’ plane matches in the bottom half of the stimulus, but showed relatively unbiased responses in the top half. A final observer showed no particular bias in response in either half of the stimulus. 

If biases in the 2AFC task can account for matching preferences observed with the whole ambiguous stimulus, one should be able to predict these matching preferences from the results of the 2AFC, half-stimulus task. Such predictions for the probabilities of perceiving convex or concave wedge shapes, and ‘ground’ or ‘ceiling’ planes arise straightforwardly from probabilities for the perception of ‘ground’ and ‘ceiling’ planes in each half of the stimulus, as follows: 

$\{\begin{array}{c}p(Convex)=p(Groun{d}_{top})(1-p(Groun{d}_{bottom}\left)\right)\\ p(Concave)=(1-p(Groun{d}_{top}\left)\right)p(Groun{d}_{bottom})\\ p(Ground)=p(Groun{d}_{top})p(Groun{d}_{bottom})\\ p(Ceiling)=(1-p(Groun{d}_{top}\left)\right)(1-p(Groun{d}_{bottom}\left)\right)\end{array}.$

(6)

 

Predictions for the probabilities of perceiving each matching solution at a luminance ambiguity of zero are shown in Figure 8a. If these predictions are a good match for biases observed in the 4AFC task, then there is no need to invoke correspondence matching mechanisms or operations, beyond simple disparity sign biases, to account for the experimental data. However, if half-stimulus-derived predictions cannot account for whole stimulus results, then the correspondence matching process must be affected by the whole stimulus configuration. 

Figure 8

 

(a) Predicted matching preferences for each observer when luminance ambiguity is equal to zero. (b) Histograms of errors for the predictions shown in (a). x-axis shows prediction errors as a proportion of the ‘error’ observed in the 4AFC task. y-axis plots the probability of obtaining an error of that magnitude. Red lines show the size of proportional error accounting for 95% of the variance. If this is less than one, then the ‘error’ observed in the 4AFC task is significantly different from the errors arising from predicted data.

In general, there are strong similarities between predicted and observed results. The pattern of biases predicted from the results of the 2AFC task is qualitatively similar to that found with whole stimulus presentation. Such qualitative similarities are evidence of similar underlying processes. This is not surprising. The use of cross-correlation-like procedures for disparity measurement, coupled with the effects of low level biases for disparity sign and magnitude should still be evident in the perception of the complete four-way ambiguous stimulus. However, qualitative similarity does not mean that predictions derived from biases in the 2AFC task can wholly account for the biases observed with whole stimulus presentation. 

A series of Monte Carlo simulations were conducted in order to formally compare predicted and observed matching preferences. For each observer, these simulations looked at the variability in predicted matching preferences by repeating the 2AFC experiment 100,000 times, with response probabilities for each stimulus interpretation determined by each observer's experimental data. On each repeat of the experiment an error term was calculated, describing the difference between the simulated prediction and the initial prediction. This error term was defined as follows: 

$\begin{array}{c}{E}_{simulated}=(P_{convex}-S_{convex}{)}^2+(P_{concave}-S_{concave}{)}^2\\ +(P_{ground}-S_{ground}{)}^2+(P_{ceiling}-S_{ceiling}{)}^2\end{array},$

(7)

where P _response is the predicted probability of the response arising from data in the 2AFC task, and S _response is the probability of a response arising from the Monte Carlo simulations. The variability of prediction errors arising from the simulations was compared to the ‘error’ observed in the 4AFC task. This error E _observed was defined in the same way as the simulation error E _simulated , replacing the S _response terms with the relevant response probabilities observed in the 4AFC experiment. 

The two error terms may be compared by examining the E _simulated error as a proportion of E _observed as follows: 

$E_{proportion}=\frac{E_{simulated}}{E_{observed}}.$

(8)

 

Figure 8b shows, for each observer, the probability of obtaining proportional errors E _proportion of varying magnitudes. The probability of E _proportion having a value greater than or equal to 1 (i.e. of obtaining a value of E _simulated greater than or equal to E _observed ) was less than 0.01 for nine of the ten observers. For the remaining observer (Obs. 3), the probability of obtaining a proportional error greater than or equal to 1 was 0.068. This result is shown in the red lines in each plot on Figure 8b, which indicate the value of required to equal 95% of the errors found in our simulations. This value is less than one for nine of the ten observers. 

Thus, for nine of our ten observers, data obtained with half stimulus presentation cannot adequately predict matching biases arising when observers were presented with the whole ambiguous stimulus. Such a result is consistent with the idea that the stereo matching process takes into account the whole stimulus configuration, and does not rely solely on the absolute disparity outputs of a cross-correlation derived model, combined with low-level disparity sign biases. 

Although predictions for the sole remaining observer did not differ significantly from his measured bias, it should be noted that this observer's responses were highly biased toward the convex wedge matching solution, far more so than other observers. Furthermore, although biases for the majority differed significantly from predictions, the direction of the bias tended to remain the same (although this was not the case for observers 4 and 7). It would therefore appear that significant disparity sign biases operate in the correspondence matching process. 

That the majority of observers showed a discrepancy between the predictions from the half-stimulus presentations, and the results obtained in the 4AFC condition, supports the conclusion that the results for the 4AFC condition represent a matching constraint defined in terms of relative disparity, rather than a result arising from the presence of local matching constraints in the top and bottom of the stimulus, defined in terms of absolute disparities. 

General discussion

The results of the experiment detailed here show that observer biases in the perception of an ambiguous stereogram cannot be explained by disparity estimation models that consider correspondence resolution, and the implementation of smoothing constraints, only in terms of the action of cross-correlation-like absolute disparity detectors. Our results instead suggest that the resolution of the correspondence problem must include processes that consider the disparity estimates arising from multiple detectors, and the differences between them (i.e. relative disparities). Below, we consider the implementation of smoothing constraints in previously published, biologically inspired models of correspondence matching/disparity measurement, and suggest ways in which such models may be extended to account for our findings. 

Cross-correlation approaches to stereo matching

The use of smoothing constraints in the resolution of the stereo correspondence problem is commonplace. A variety of smoothing constraints have been considered, which have ranged from explicit matching rules based on a principle of surface continuity (Marr & Poggio, 1976, 1979; Pollard et al., 1985), to the implicit use of spatially extended correlation windows (Banks et al., 2004) or the spatial weighting of disparity energy units (Fleet et al., 1996; Qian & Zhu, 1997). Marr and Poggio's (1979) early model of correspondence matching explicitly considered issues of surface smoothness in the formulation of a continuity constraint. When confronted with a choice between matches of the same magnitude, but opposite sign, their model would choose the match that preserved continuity of disparity sign (rather than continuity of disparity gradient sign). Similarly, existing psychophysical data also support the idea of a minimization of relative disparity in binocular matching (Goutcher & Mamassian, 2005; Zhang et al., 2001). 

This conception of ‘smoothness’ as the minimization of relative disparity is consistent with recent, cross-correlation based approaches to the modeling of disparity measurement. Cross-correlation models favor solutions that minimize relative disparity locally, as the disparities in such solutions are closer to translations of image regions between the two eyes. If we apply mechanisms with this pure translational bias to random-dot stimuli, we find that solutions that minimize relative disparity will have higher correlations than solutions with greater changes in disparity. Cross-correlation mechanisms have no way of measuring ‘smoothness’ in terms of the minimization of changes in sign of disparity gradient. 

In considering this intrinsic bias of cross-correlation based models toward minimal relative disparity solutions, it is important to note that such a bias is limited to the local scale at which cross-correlation calculations are made. This is an important consideration as, while such local biasing is sufficient to find the correct solution in random-dot stimuli, it is insufficient in many classes of ambiguous stereoscopic stimuli. In almost any periodic binocular stimulus, local bias is not enough to ensure that, for example, a single continuous surface is perceived, rather than a series of unconnected strips in depth. This can be seen in the application of a maximal correlation decision rule, as shown in Figure 3b. This difficulty arises because the local structure at each point in the image is consistent with each possible interpretation. The only options available to solve this problem are to bias disparity measurement mechanisms toward particular signs or magnitudes of disparity, or to consider processes that compare the outputs of multiple cross-correlation-like detectors. 

Let us consider a simple mechanism that measures binocular disparity through cross-correlation and, in the event of an ambiguous response, relies on prior matching constraints that favor disparities of particular magnitudes and signs. A good first assumption for such a model would be that the preferred magnitude of disparity would be small. Such a bias is consistent with the distribution of disparities present in natural scenes (Hibbard, 2008) and in model environments (Hibbard, 2007; Langer, 2008; Read & Cumming, 2004) and has proven useful in several existing models of disparity estimation (Prince & Eagle, 2000; Qian & Zhu, 1997; Read, 2002). Hibbard and Bouzit (2005) have also presented psychophysical evidence of biases in disparity sign. They found that the perceived sign of disparity in an ambiguous stereogram depended upon the location of the stimulus in the visual field. Alternatively, a simpler disparity sign bias may be produced if we consider the possibility that observers possess small, consistent, fixation disparities. Consistent fixation disparities, in combination with a bias for small disparities would result in a bias for particular signs of disparity, since the sign of disparity would correlate with a smaller disparity magnitude. 

Although such absolute disparity biases would resolve correspondence ambiguity in periodic stimuli, matching in such a model will not depend on the global structure of the stimulus with which it is presented. This means that an identical disparity estimate would be produced at any given point in the image whether the model were presented with a complete ambiguous stereogram, or an isolated ambiguous strip. Such behavior is not observed in our human participants. Instead, matching biases change significantly with the presentation of our complete ambiguous stimulus, compared to the case where only a portion of the stimulus is presented. Such changes in matching preference present a direct challenge to any existing model that considers disparity estimation only at the level of local cross-correlation or disparity-energy detection, and are only consistent with a mechanism that considers the outputs of multiple disparity detectors. In other words, with a mechanism that considers relative disparity. 

Minimization of relative disparity

If correspondence matching/disparity measurement does indeed require a processing stage that considers the relative disparity content of alternative matching solutions, then it is important to establish the nature of the rules and biases operating at such a stage. In the first instance we are concerned with ascertaining the relative disparity structures preferred by the system. 

The results of the present experiment suggest that the matching process is heavily biased toward solutions that minimize relative disparity across the image. This bias will result in surface discontinuities when such discontinuous surfaces have a smaller overall change in disparity. Such results are consistent with earlier findings suggesting a bias for the minimization of relative disparity (Goutcher & Mamassian, 2005; Zhang et al., 2001), with findings that suggest that the visual system compares the total change in disparity in alternative matching solutions (Goutcher & Mamassian, 2005), and with the perception of discontinuous disparities arising in the Venetian blind effect (Banks & Vlaskamp, 2009; Cibis & Haber, 1951). In the case of the Venetian blind effect, discontinuous surfaces are but one interpretation of an ambiguous stereoscopic stimulus that might equally be perceived as a continuous surface slanted in depth. Banks and Vlaskamp (2009) suggest that the perception of discontinuous surfaces in this instance may well arise from a bias for small disparities, although a role for relative disparity minimization cannot be ruled out. 

In addition to biases for the minimization of relative disparity, we also find evidence, in two observers, of a bias toward solutions that minimize changes in disparity gradient sign. In this instance, large changes in disparity are preferred when they minimize surface discontinuity. Such results suggest that substantial individual differences exist in matching strategies, which may indicate a role for voluntary eye movements and/or top-down processing in disparity measurement. 

The distinction between a bias for the minimization of relative disparity and the minimization of changes in sign of disparity gradient reflects the need for distinct constraints on matching that reflect the statistical structure of the natural environment. Any constraint on matching is only useful in that it aids the computation of disparity structures likely to arise in the world. A bias for the minimization of relative disparity therefore suggests that our disparity measurement system is suited to an environment containing small changes in disparity locally, but with many local discontinuities. This would suggest that stereopsis is useful for the perception of rough textured surfaces. Conversely, matching constraints do not appear to ease the processing of large planar surfaces. This suggests that disparity is either not a preeminent cue for the perception of such surfaces, or that such surfaces do not tend to present the visual system with large scale matching problems due to disambiguation from other sources of information. 

Relative disparity processing

The results presented here show that relative disparity processing must play a role in the resolution of the correspondence problem. Here, we consider alternative approaches to the production of relative disparity related constraints. 

Perhaps the simplest possibility, suggested by an anonymous reviewer, is that relative disparity processing is related to active vergence eye movement strategies. Let us assume that small fixation disparities, or disparity sign biases result in matches to a particular disparity plane. If we suppose that such matches elicit vergence eye movements to that disparity plane, then a bias for small disparities would resolve the correspondence problem, producing the observed bias for the minimization of relative disparity. 

A mechanism of this kind would predict an increase in the disparity sign biases required to explain a preference for the minimization of relative disparity (i.e. the observed bias for the minimization of relative disparity should be greater than the predicted bias). This is only seen in some of our observers, with others showing general weakening of bias compared to predictions (Obs. 3 and 7). Such a mechanism also cannot explain biases for the perception of slanted surfaces (Obs. 2 and 6). As such, although vergence eye movements may play a role in correspondence matching, they are not sufficient to account for the changes in bias found here. Direct evidence that this cannot be a complete account for our results was obtained by repeating the 4AFC experiment for two of our observers, with a presentation of 150 ms, too short to allow vergence eye movements. The bias shown by these two observers was not affected by removing the possibility of vergence eye movements, meaning this cannot explain the presence of the bias. 

An alternative solution is to create a bias toward the minimization of relative disparity through processes of lateral excitation and inhibition. Such processing is evident in some of the very earliest models of correspondence matching (Marr & Poggio, 1976). Recently, Samonds, Potetz, and Lee (2009) have shown that the correlation of activity of neighboring disparity sensitive V1 neurones changes over time in a manner consistent with the action of excitatory and inhibitory lateral connections. Connections of this kind would ensure that neighboring points of an ambiguous stereogram are matched to similar disparities, allowing for the perception of continuous surfaces. 

Finally, relative disparity processing could be explicitly created through the use of receptive fields that vary in disparity space. One may already observe such processing in the template decision stage of many cross-correlation type models. For example, Filippini and Banks (2009) use a template matching decision rule to allow their cross-correlation model to perceive (i.e. make a decision about) the direction of slant in a sawtooth stereogram. Their decision template acts on the output of a local cross-correlator, where no decision has been made about the disparity at any single point in the image. As such, it is the decision stage of the model that effectively rejects or accepts matches as false or true. The same is true of our own application of cross-correlation in this paper. 

Of course, such templates are not suitable for a generalized model of correspondence matching. Instead they offer a useful means of modeling a limited family of possible matching solutions in a defined psychophysical task. However, a truly general model of correspondence matching could make use of distinct sets of disparity space templates in order to remove spurious response peaks, or, as in the case of our ambiguous stimulus, select preferred, surface-based matching solutions, given a multitude of disparity responses of similar magnitude. The effects of such templates would correspond to cyclopean and hypercylopean levels of processing (Tyler & Kontsevich, 1995). 

Acknowledgments

This research was supported by BBSRC grant no. C005260/1 (PBH) and by RCUK Fellowship no. EP/E500722/1 (RG). We thank Prof. Roger Watt for his helpful comments on data analysis. 

Commercial relationships: none. 

Corresponding author: Ross Goutcher. 

Email: [e-mail]. 

Address: University of Stirling, Stirling, FK9 4LA, UK.