How does the visual system combine information from different depth cues to estimate three-dimensional scene parameters? We tested a maximum-likelihood estimation (MLE) model of cue combination for perspective (texture) and binocular disparity cues to surface slant. By factoring the reliability of each cue into the combination process, MLE provides more reliable estimates of slant than would be available from either cue alone. We measured the reliability of each cue in isolation across a range of slants and distances using a slant-discrimination task. The reliability of the texture cue increases as |slant| increases and does not change with distance. The reliability of the disparity cue decreases as distance increases and varies with slant in a way that also depends on viewing distance. The trends in the single-cue data can be understood in terms of the information available in the retinal images and issues related to solving the binocular correspondence problem. To test the MLE model, we measured perceived slant of two-cue stimuli when disparity and texture were in conflict and the reliability of slant estimation when both cues were available. Results from the two-cue study indicate, consistent with the MLE model, that observers weight each cue according to its relative reliability: Disparity weight decreased as distance and |slant| increased. We also observed the expected improvement in slant estimation when both cues were available. With few discrepancies, our data indicate that observers combine cues in a statistically optimal fashion and thereby reduce the variance of slant estimates below that which could be achieved from either cue alone. These results are consistent with other studies that quantitatively examined the MLE model of cue combination. Thus, there is a growing empirical consensus that MLE provides a good quantitative account of cue combination and that sensory information is used in a manner that maximizes the precision of perceptual estimates.

*^S*

_{d}and

*^S*

_{t}of the slant of a surface based on disparity and texture cues, respectively. Assume further that errors in these estimates are uncorrelated and have variances

*σ*

^{2}

_{d}and

*σ*

^{2}

_{t}. If we combine the two estimates linearly, the rule that yields the minimum-variance, unbiased estimate is a weighted average that satisfies (Cochran, 1937) where and

*r*

_{d}and

*r*

_{t}are the

*reliabilities*of the two cues (e.g., ). Furthermore, if errors associated with the individual estimators are Gaussian, no other (nonlinear) rule has lower variance.

*^S*that is most probable given the image data. We assume the image data can be segregated into those data

*I*

_{d}used to estimate slant from disparity and

*I*

_{t}used to estimate slant from texture. Thus, we choose the value of

*^S*that maximizes

*p*(

*^S*|

*I*

_{d},

*I*

_{t}). Applying Bayes’ rule, and assuming that the two cues are conditionally independent, we derive The first two terms on the right side of the equation are the likelihood functions for each cue characterizing the probability of observing the image data if

*^S*is the actual slant. The last term is the prior distribution, which is the probability of observing

*^S*in the scene, independent of the image data. If the likelihoods and prior are Gaussian, the MAP estimate has the same form as the minimum variance, linear combination estimate where

*^S*

_{d}and

*^S*

_{t}are the maximum-likelihood estimates the observer would have made from each cue in isolation (the mean of the respective Gaussian distributions), and

*^S*

_{p}is the mean of the prior. The

*r*

_{i}are the reliabilities of the respective distributions (likelihoods and prior). If the prior has large variance relative to the individual cue likelihoods, Equations 4 and 5 reduce to Equations 1 and 2, which also yields the most likely slant to have caused the current sensory data (i.e., it is the maximum-likelihood estimate or MLE). For our conditions, the variance of the individual cues is much smaller than the prior’s variance (see Ideal observer models in Discussion), so we will use Equations 1 and 2 throughout.

*^S*is The variance of

*^S*is lower than the variance of either single-cue estimate.

*r*

_{i}in Equations 2 and 6) and empirically tested predictions for both the appearance and discrimination thresholds for stimuli when both cues were present (provided by Equations 1, 2, and 6). All five reported that the combination is quite close to the one predicted by those equations. In these five experiments, the variances of estimates derived from single cues were measured by conducting two-interval, forced-choice (2IFC) discrimination experiments when only one cue was informative. For example, Ernst and Banks (2002) conducted size-discrimination experiments for vision alone and haptics alone and then fit cumulative Gaussians to the two psychometric functions. The variance parameter of the Gaussians provided estimates of the variances of the underlying visual and haptic estimators. Equations 1, 2, and 6 were then successfully used to predict the results of two-cue (visual-haptic) experiments.

^{2}.

*S*

_{d}=

*S*

_{t}) or they could be in conflict. In the no-conflict case, homogeneous Voronoi-textured surfaces were projected directly to the two eyes. In cue-conflict cases, we first calculated a perspective projection of the texture with slant

*S*

_{t}at the Cyclopean eye (Figure 2, left panel). We then found the intersections of rays through this Cyclopean projection with a surface patch at the disparity-specified slant

*S*

_{t}(Figure 2, middle panel). The markings on this latter surface were then projected to the left and right eyes to form the two monocular images (Figure 2, right panel).

*slant*estimates. In the disparity-alone condition, this means our measure must reflect the process of scaling the disparity signal into units of slant. If the task in the disparity-alone condition were done without normalizing the disparity signal, the psychometric data would not reflect errors introduced by the scaling process (which, within the framework of weighted-linear cue combination, is essential for combining disparity and texture signals), and we would underestimate the variance of disparity-based slant estimates. We therefore looked for evidence that observers scale the disparity signal for distance in a discrimination task with our disparity-alone stimuli. Observers performed the slant-discrimination task with the disparity-alone stimulus, but with the comparison stimuli appearing at different distances relative to the standard stimulus. We found that most observers (importantly, JMH and ACD) take distance into account when performing the slant-discrimination task; that is, they do not perform the task by only comparing the disparity gradients in the two stimulus intervals. The details of the experiment and results are described in 2. The results of this control experiment support the assumption that the disparity-alone measurements provide valid estimates of the reliability of the disparity-based slant estimator.

*S*and four reversal rules—3-down/1-up, 1-down/3-up, 2-down/1-up, and 1-down/2-up—to sample points along the entire psychometric function. At least eight staircases were employed for each psychometric function for ACD and JMH, which corresponds to approximately 350–450 trials per function (each staircase was terminated after 12 reversals). At least two, but typically more, staircases were employed for RM and MSB. In each session, at least four interleaved staircases were run: two base slants (one positive and one negative to avoid adaptation) with two staircases each. Viewing distance was fixed in each session.

*S*

_{b}= ±60, ±30, or 0 deg) and the other cue was perturbed. The left panel depicts the conflict stimuli when disparity was perturbed and the right panel shows the stimuli when texture was perturbed. The perturbed cue had incremental slants of ±10, ±5, or 0 deg relative to the unperturbed cue, so the conflict was always small. In previous work with quite similar stimuli, a difference of 10 deg between the disparity- and texture-specified slants was generally not detectable (Hillis, Ernst, Banks, & Landy, 2002). The five possible perturbed cue values are represented along the abscissa and ordinate in the left and right panels, respectively. On each trial, a conflict stimulus and a no-conflict stimulus were presented and the observer indicated the one containing the apparently greater slant. No feedback was given. The value of the no-conflict stimulus was varied according to staircase procedures to map out the psychometric function. At least four staircases were run per experimental session: two conflict conditions for each of two base slants.

*point of subjective equality*(PSE), the value of the no-conflict stimulus with the same average perceived slant as the conflict stimulus. The enlarged bold symbols—the dark-blue circle in the left panel and black diamond in the right—represent two particular conflict stimuli. Δ represents the incremental slant of the perturbed cue in the conflict stimulus, and

*δ*represents the increment given to the no-conflict stimulus as the staircase procedure varies its slant. As

*δ*is increased and thereby the slant of the no-conflict stimulus is increased (represented in the figure by displacement up along the main diagonal where

*S*

_{t}=

*S*

_{d}), the observer will be increasingly likely to report that it had greater slant than the conflict stimulus. At some value of

*δ*, the no-conflict stimulus will on average have the same apparent slant as the conflict stimulus; this is the PSE. If the cue weights are constant across small variations in slant, we can determine the weights from this value of

*δ*.

*S*

_{d}=

*S*

_{t}+ Δ =

*S*

_{b}+ Δ (where

*S*

_{b}is the base slant), the expected value of the estimated slant of the conflict stimulus is Now consider the no-conflict stimulus. From Equations 1–2 and the fact that

*S*

_{d}=

*S*

_{t}=

*S*

_{b}+

*δ*, the expected value of the estimated slant of the no-conflict stimulus is

*S*

_{c}=

*S*

_{nc}and from Equations 7 and 8, we have

*σ*

^{2}

_{d}and

*σ*

^{2}

_{t}, or equivalently, the reliabilities

*r*

_{d}and

*r*

_{t}in Equations 2 and 6). To estimate these variances, we fit the psychometric data with a cumulative Gaussian using a maximum-likelihood criterion. The standard deviations of the resulting functions were divided by (because the psychophysical procedure was 2IFC) to yield estimates of the standard deviations of the underlying slant estimators (Green & Swets, 1974). We call these just-noticeable differences (JNDs) because they represent the slant difference that is correctly discriminated ∼76% of the time.

*HSR*) (Backus et al., 1999).

*HSR*=

*α*

_{L}/

*α*

_{R}, where

*α*

_{L}and

*α*

_{R}are the horizontal angles subtended by a surface patch in the left and right eyes. Plotted in these units, JNDs do not vary systematically as a function of viewing distance. This implies that the increase in slant-discrimination threshold is caused only by the geometric relationship between distance and disparity and not by greater error in the calculation of disparity nor by greater error in estimates used to scale for distance (such as vergence; Equation 2 in Backus et al., 1999). JNDs plotted in these units increase with increasing |ln(

*HSR*)|. This increase may reflect difficulties in solving the binocular-matching problem as the disparity gradient (which is linearly related to

*HSR*) increases (Banks, Gepshtein, & Landy, 2004; Burt & Julesz, 1980). The increase may also reflect the fact that surfaces with large |ln(

*HSR*)| contain fewer points near the Vieth-Müller Circle where stereoacuity is highest. |ln(

*HSR*)| increases more rapidly as a function of slant at near distances (indicated by the fact that JMH’s data at high base

*HSR*s all come from the near viewing distance). For example, a change in slant from 60 to 70 deg results in a change in ln(

*HSR*) from 0.58 to 0.97 at 19.1 cm and from 0.06 to 0.1 at 171.9 cm. (We did not plot the point at 70 deg, ln(

*HSR*) = 0.97, because thresholds were infinite.) We will return to a discussion of the effects of distance and base slant in Discussion: Comparison of observed and expected effects of slant and distance on disparity- and texture-based JNDs.

*JND*=

*α*·

*e*

^{β|x|}, where

*x*is slant and

*JND*is in deg [texture], or

*x*is ln(

*HSR*) and

*JND*is |Δln(

*HSR*)| [disparity], and

*α*and

*β*are free parameters). This was done by performing a maximum-likelihood fit to all of the raw psychometric data for a given condition (texture or disparity), varying

*α*and

*β*. The curves and the data are shown in the left and right columns of Figure 4. The curve fits represent a fit to the data at all three viewing distances. Thus, they give us a way to estimate disparity and texture reliability between slants where we have measurements, and they also allow us to interpolate across distance. While the reliability of the disparity cue to slant,

*HSR*, does not vary systematically with distance, the relationship between

*HSR*and slant varies significantly with viewing distance. Figure 5 shows how the reliability of disparity slant estimates varies with slant and distance, based on the curve fits to JMH’s data. The reliability of the disparity cue to slant decreases as distance increases and the reliability of the disparity cue varies with base slant in different ways at different distances. At near distances the disparity cue is more reliable than the texture cue (and hence, should be given more weight according to the MLE model).

*d*is viewing distance and

*i*is the inter-ocular distance. Slant from disparity (for tilt = 0) is given to close approximation by Thus, errors in the disparity and distance estimates will both yield errors in the estimated slant. We calculated the distribution of slant estimates for different viewing conditions under the assumption that the errors in

*HSR*and

*μ*can be represented by additive, independent noises. Specifically, we conducted a Monte Carlo simulation to determine the standard deviation of slant estimates

*σ*

_{^S}from Equation 11. The noises were Gaussian with mean = 0. We adjusted the noise standard deviations,

*σ*

_{μ}and

*σ*

_{HSR}, to obtain simulation JNDs similar to the observed JNDs. The simulation results are displayed in Figure 12. The left panel shows

*σ*

_{^S}as a function of distance (the curves representing different base slants) and the right panel shows

*σ*

_{^S}as a function of base slant (the curves representing different distances).

*σ*

_{^S}, is roughly proportional to viewing distance for all base slants (left panel). This result is expected from Equation 11 because

*d*≈

*i/μ*, so fixed additive noise in

*μ*has an increasing effect with distance. We found that

*σ*

_{^S}was proportional to distance for a wide range of

*σ*

_{μ}and

*σ*

_{HSR}; the key assumption is that the noise in disparity normalization is fixed and additive in vergence. The data points in the lower left panel are JNDs from observer JMH; clearly, his discrimination thresholds increased monotonically with increasing distance in much the same way as the simulation. The data from ACD were similar. Thus, the distance effect we observed in the disparity-alone experiment is expected if error in disparity normalization is additive in units of vergence.

*σ*

_{^S}is inversely related to the absolute value of slant. This relationship was observed for all values except when

*σ*

_{μ}≫

*σ*

_{HSR}. The relationship is expected from Equation 11 because

*^S*

_{d}≈

*k*ln(

*HSR*), so fixed additive noise in

*HSR*has progressively less effect on

*σ*

_{^S}as base slant increases. The data points in the lower right panel are JNDs from observer JMH; data were similar for the other three observers. At viewing distances of 57.3 and 171.9 cm, JMH’s discrimination thresholds decreased monotonically with slant magnitude much like the simulation’s standard deviations.

*HSR*. Does this assumption make sense? It does when

*HSR*is not significantly different from 1, which was true for distances of 57.3 and 171.9 cm (see Figure 4). However, when

*HSR*is quite different from 1, points on the surface fall where stereo-acuity is low and problems arise in solving the binocular correspondence problem (Burt & Julesz, 1980).

*HSR*and the horizontal gradient of horizontal disparity are closely related, where

*DG*is an approximation to the disparity gradient (Howard & Rogers, 2002). From Equations 10 and 11 when

*d*is small and

*S*is large,

*HSR*is quite different from 1 and thus

*DG*will be quite different from 0. Burt and Julesz (1980) and others have shown that binocular correspondence becomes difficult when |

*DG*| deviates significantly from 0 and breaks down altogether when |

*DG*| ≈ 1. Recent results indicate that this is probably a by-product of a matching process that is similar to cross-correlating the two eyes’ images to estimate the disparity in a region of the visual field (Banks et al., 2004).

*DG*| increases rapidly as a function of slant at the short distance, so we expect performance to be worse at that distance for large slants. JMH’s data exhibited this effect. His discrimination thresholds at 19.1 cm increased with slant, which is inconsistent with the assumption that the sole source of error in disparity measurement is additive in

*HSR*(Figure 12, lower-right panel, gray curve). They were higher than predicted for |

*slant*| ≥ 30 deg which corresponds to a higher disparity gradient (|

*DG*| ≥ 0.19,

*HSR*≥ 1.2) than occurs at 57.3 and 171.9 cm. Thus, the base-slant effect in the disparity-alone experiment is expected if error in disparity measurement is additive in

*HSR*except when

*HSR*deviates significantly from 1 where problems arise in solving correspondence.

*slant anisotropy*. The phenomenon is most striking when the texture gradient specifies a frontoparallel plane, as is usually the case with random-element stereograms. The phenomenon is not observed when disparity and texture signal the same depth variation, as occurs with real surfaces (Bradshaw, Hibbard, van der Willigen, Watt, & Simpson, 2002; Buckley & Frisby, 1993). These observations strongly suggest that slant anisotropy is caused by conflicting disparity and texture signals in conventional random-element stereograms. They also suggest that texture is generally given more weight for tilt 0 (as in our experiments) and less weight for tilt 90 (as in Knill & Saunders, 2003). By the argument presented here, this may be due to reduced disparity reliability for tilt 0 than for tilt 90 because there is no obvious reason for the reliability of the monocular texture cue to depend on tilt. There may, however, be differences in the steps required to combine the texture and disparity signals for different tilts. The issues involved in transforming texture-gradient signals into the same coordinates for combination with disparity signals are taken up in 4.

*μ*is vergence, and

*γ*is azimuth (the angle between the head’s median plane and the Cyclopean line of sight) (Backus et al., 1999).

*μ*is estimated both from extra-retinal signals concerning the eyes’ vergence and from the horizontal gradient of vertical disparity (Rogers & Bradshaw, 1995). When vertical disparities are large, as occurs with large stimuli at close range, they are the predominant means for estimating distance. However, when vertical disparities are unreliable because the stimulus is small (Rogers & Bradshaw, 1995), or because the texture contains no horizontal contours (Helmholtz, 1910), the eyes’ vergence becomes the predominant means of estimating distance and the accuracy of disparity normalization drops (Rogers & Bradshaw, 1995).

*γ*is used to correct disparities; it is estimated from extra-retinal, eye-position signals and from the magnitude of vertical disparities (Backus et al., 1999). When vertical disparities are large, as occurs with near stimuli subtending a large angle, they are the predominant means of estimating azimuth. When the stimulus is short or when vertical disparities are unmeasurable, eye position becomes the predominant means and the accuracy of disparity correction suffers (Backus et al., 1999).

*μ*is again the vergence angle,

*HSh*is horizontal shear disparity (Banks et al., 2001) and

*τ*is the cyclovergence of the eyes (the difference in the eyes’ torsion).

*HSh*must be normalized for distance by an estimate of

*μ*and corrected for cyclovergence by an estimate of

*τ*(Banks et al., 2001; Howard & Kaneko, 1994).

*S*and the distance is

*d*; for the part on the right, it is where

*γ*is the azimuth. The distance to the intersection of the line and plane is The left and middle panels of Figure 16 show how

*S*

_{γ}and

*d*

_{γ}vary with azimuth for different base slants and

*d*= 19.1 cm. Because the local slant and distance vary with azimuth, the statistically optimal weights for the texture and disparity cues should vary with azimuth.

*γ*< 0), slants

*S*

_{dγ}and

*S*

_{tγ}approach zero and distance

*S*

_{γ}decreases. Our data (Figure 6) show that texture weight is relatively low when the absolute value of slant is ∼0 and distance is short. Thus, if the weights used in combining slant estimates are determined locally, one would expect the texture weight in this situation to be lower on the right than straight ahead. (The changes in local slant and distance with changes in azimuth are unaffected by the direction in which the eyes are looking; they are determined only by the positions of surface points relative to the head. Thus, when we say “on the right” or “straight ahead,” we refer to the head-centered azimuth of a line of sight from the Cyclopean eye and not necessarily the azimuth of fixation.) For leftward azimuth (

*γ*> 0), the slants become increasingly negative and distance increases; the texture weight in this situation should be higher on the left than straight ahead.

*S*

_{γ}should change with azimuth for observer JMH. The cue-conflict stimulus in the calculation had a disparity-specified slant

*S*

_{d}of −25 deg and texture-specified slant

*S*

_{t}of −10 deg. Distance

*d*to the midpoint was 19.1 cm. If weights are determined locally for each azimuth

*γ*, the observer must associate local disparity-defined slant

*S*

_{dγ}with its corresponding variance

*σ*

^{2}

_{dγ}and the texture-defined slant

*S*

_{tγ}with its variance

*σ*

^{2}

_{dγ}. The right panel of Figure 16 shows the expected change in slant as a function of azimuth. With leftward azimuth, the slant estimate approaches the texture-specified slant, and with rightward azimuth, it approaches the disparity-specified slant. The result would be an apparently concave surface as schematized in the right panel of Figure 15. If the disparity- and texture-specified surfaces were swapped (

*S*

_{d}= −10;

*S*

_{t}= −25 deg), the result would be an apparently convex surface. If the disparity and texture specified the same slant, as they would with most real surfaces, the result should be an apparently planar surface.

*γ*= −25,

*S*

_{d}= −25, and

*S*

_{t}= −10 deg. At this azimuth, the field of view was ∼70 deg wide. The stimulus was clipped by an elliptical window to make the outline shape an unreliable cue to slant. The room was completely dark except for the display so the screen’s frame could not be seen. If weights are set locally, this cue-conflict stimulus should appear concave. We also created a viewing situation in which the surface should appear convex—

*γ*= −25,

*S*

_{d}= −25, and

*S*

_{t}= −40 deg—and another in which the surface should appear planar—

*γ*= −25,

*S*

_{d}= −25, and

*S*

_{t}= −25 deg. Seven observers (five naïve) viewed the displays and reported whether they appeared concave, planar, or convex. Five of the seven (three naïve) reported that the stimuli predicted to look concave and convex actually looked that way; the other two said that the stimuli all appeared concave or planar but that the one predicted to look concave appeared the most concave. We asked them to order the three stimuli according to the amount of perceived concavity, and all seven ordered them in the predicted order.

*P*(

*^S*) in Equation 3) has a negligible effect. This assumption is usually justified (e.g., Ernst & Banks, 2002) by assuming that the variance of the prior is much greater than the variances of the likelihoods. Is this the case in the estimation of surface slant? It is reasonable to assume that the distribution of surface slants in the world is uniform, particularly for tilt = 0. But if that distribution is uniform, the probability of observing slant

*S*at the retina will be proportional to cos(

*S*) because steeply slanted surfaces project to smaller retinal images. Equations 7–9 show how the observers’ judgments will be affected by the stimulus values and weights. If we add the prior into those equations, Equation 9 becomes

*w*

_{p}depends on its inverse variance, or reliability, relative to the estimator reliabilities (Equation 5). As long as the prior’s variance is large relative to the estimators’ variances,

*w*

_{p}will be small and will have no discernible effect on the data. The prior distribution is proportional to cos(

*S*), which is defined from −90 to 90 deg. Such a half cosine has a standard deviation of ∼40 deg. The standard deviations of the disparity and texture estimators (Figure 4) ranged from 1–20 deg. They were the highest when distance = 171.9 cm and slant = 0 deg. Then the standard deviations of the disparity and texture estimators were 20.4 and 8.0 deg for JMH and 16.1 and 5.3 deg for ACD. We can use Equation 5 to calculate the expected

*w*

_{p}for those conditions:

*w*

_{p}= 0.034 and 0.016 for JMH and ACD, respectively. Those represent the largest possible influence of the prior distribution on the results and

*w*

_{p}is still quite small. Further, the weights given to texture and disparity generally sum to one (Figures 3S–5S), indicating that the prior received little or no weight in all of our conditions. We conclude that the prior had no discernible influence on our results.

*σ*

_{n}. We also assume that

*σ*

_{n}has the same value in the single-cue and two-cue experiments.

*˜r*

_{d}and

*˜r*

_{t}represent the measured reliabilities from the disparity-alone and texture-alone experiments (the measured reliabilities include the effects of decision noise). In the two-cue experiment, we measured the weight observers actually assigned to disparity and texture and those weights were presumably affected only by the visual system’s estimates of the uncertainties of the disparity and texture estimators. In other words, Equation 2 rather than Equation 19 describes what the observed weights should be. Decision noise should, therefore, affect the predicted weights (Equation 19) and not the observed weights in the two-cue experiment. To determine the consequences of decision noise, we calculated the predicted and observed weights for a variety of situations. We set the sum of estimator variances to one (

*σ*

_{d}

^{2}+

*σ*

_{t}

^{2}= 1) and varied

*σ*

_{t}

^{2}from ∼0 to ∼1.

*σ*

_{n}

^{2}was set to 0, 0.1, 0.32, or 1. The left panel of Figure 17 shows the results. The predicted texture weight (

*w*

_{tP}) is plotted as a function of the actual texture weight (

*w*

_{t}). Naturally, the prediction (diagonal dashed line) is perfect when

*σ*

_{n}

^{2}= 0 because Equations 2 and 19 are then identical. For

*σ*

_{n}

^{2}> 0, the predicted weights deviate from the observed. When

*w*

_{t}is greater than 0.5 (and hence greater than

*w*

_{d}),

*w*

_{tP}is less than

*w*

_{t}. When

*w*

_{t}is less than 0.5, the opposite occurs. When

*w*

_{t}≈

*w*

_{d}(

*w*

_{t}= ∼0.5), the effect of decision noise is negligible. Thus, if decision noise were sufficiently large in our experiments, it should cause error in the PSE data when

*w*

_{t}is either much larger or much smaller than

*w*

_{d}. This circumstance occurred when the viewing distance was 19.1 cm and the base slant was 0 deg and when the viewing distance was 171.9 cm (Figures 8 and 9). With the exception of distance = 171.9, base slant = 0, the agreement between predicted and observed PSEs is excellent in these cases. This implies that uncertainty due to decision noise (and other additive noises) was small relative to the uncertainty of the underlying slant estimators.

*σ*

_{n}

^{2}was equal to the variance of the combined estimate (Equation 6), so

*σ*

_{n}

^{2}varied with

*w*

_{t}; in particular, it had lower values when

*w*

_{t}= ∼0 or ∼1. Thus, their simulations showed very little effect of decision noise on predicted weights. Knill and Saunders also modeled constant-variance noise like we did, but they did not show the results of that analysis.

*σ*’s): Now we make the two-cue measurements in order to compare the observed and predicted JNDs. In the two-cue experiment, the visual system would weight the cues as in Equations 1 and 2, and the decision noise would again affect the threshold measurement. Thus, the JND we measure in the two-cue experiment is

*σ*

_{n}= 0, but when

*σ*

_{n}> 0, the decision noise has different effects on the predicted and observed two-cue JNDs. To determine how additive decision noise could affect the interpretation of the JNDs, we calculated the ratio of observed JND (Equation 21) divided by the predicted JND (Equation 20). In doing the calculations we again set the sum of estimator variances to 1 and varied

*σ*

_{t}

^{2}from ∼0 to ∼1.

*σ*

_{n}

^{2}was again set to 0, 0.1, 0.32, or 1. The right panel of Figure 17 shows the results. The JND ratio is plotted as a function of the texture weight. The dashed horizontal line represents the ratio when

*σ*

_{n}

^{2}= 0. As

*σ*

_{n}

^{2}increases from 0 to 1, the observed JND becomes larger than the predicted. The ratio is largest at ∼1.3 when

*w*

_{t}= 0.5 and

*σ*

_{n}

^{2}= 1. The JNDs we observed in the two-cue experiment were generally quite close to the predicted values (Figure 11), so this analysis suggests that we can rule out the presence of decision noise whose variance is greater than approximately half the sum of the estimator variances.

*δ*(where

*δ*is the increment or decrement given to the base slant in order to obtain a threshold), (2) texture slant = 0 and disparity-specified slant = 45 +

*δ*, and (3) texture slant = 45 and disparity-specified slant = 45 +

*δ*. The results are plotted in Figure A1, which shows the just-discriminable change in slant as a function of dot number for the three conditions. For observer JMH, discrimination thresholds decreased as dot number was increased from 2 to 32 and then thresholds reached an asymptote by 32–64 dots. With 64 dots, disparity-based thresholds were essentially as low as they could be. The results were more complicated for observer ACD. When the disparity-specified slant was 0 deg, her data were quite similar to JMH’s. However, when the disparity slant was 45 deg, thresholds decreased from 4–64 dots and then increased with more than 64 dots. One would observe such an effect if the conflicting texture signal, which was not informative for the discrimination task, was given increasing weight with increasing dot number. Thus, ACD may have given some weight to the uninformative texture signal when shown the random-dot stimulus at base slants different from 0 deg. We discuss this point in regard to her two-cue data in Summary of results in Discussion.

*slant*estimates. In the disparity-alone condition, this means our measure must reflect the process of scaling the disparity signal into units of slant. If the task in the disparity-alone condition were done without normalizing the disparity signal, the psychometric data would not reflect errors introduced by the scaling process (which, within the framework of weighted-linear cue combination, is essential for combining disparity and texture signals), and we would underestimate the variance of disparity-based

*slant*estimates. For this reason, we looked for evidence that observers scale the disparity signal for distance in a discrimination task with our disparity-alone stimuli. We did so by having observers perform the slant-discrimination task with the disparity-alone stimulus with the comparison stimuli appearing at different distances relative to the standard stimulus.

^{TM}liquid-crystal shutter glasses. Left- and right-eye images were displayed on alternate frames so each eye’s image was drawn only when the corresponding shutter was open. The monitor refresh rate was 100 Hz, so each eye’s image was redrawn at 50 Hz. The stimuli were drawn using the red phosphor only because this minimized cross-talk through the shutter glasses. The room was otherwise dark. Precise reproduction of visual directions was achieved using the same anti-aliasing and spatial calibration techniques as in the main experiment. The same bite-bar set up was used to position and stabilize the observer’s head.

*HSR*s), but different slants (diagonal dotted line).

*did*take the change in stimulus distance into account. However, none exhibited complete compensation, which means that the distance compensation was not veridical. The data from JMH and ACD, the observers who participated in all conditions in the main experiments, are represented by filled symbols. The results of this control experiment show that most observers (importantly including observers JMH and ACD) do not perform the slant-discrimination task by only comparing the disparity gradient in the two stimulus intervals.

*HSR*s were biased. Other reports (Johnston, 1991; Johnston et al., 1993; Bradshaw, et al., 1996) and our control experiment (2) suggest that observers do not scale the disparity signal veridically for distance; that is, they tend to use a farther estimate than the actual distance for near viewing and a near estimate than the actual distance for far viewing. As we pointed out in the Discussion (Comparison to other studies), this result could be due to the influence of unmodeled cues. Here we consider the possibility that the observed changes in PSEs with distance resulted from mis-scaling of the

*HSR*signal rather than changes in cue weighting with changes in distance.

*^d*=

*C*; one fixed distance is used to scale the

*HSR*signal at all distances. (2)

*^d*=

*α*·

*d*+

*β*; distance estimates are a linear function of distance. There were three main steps to fitting the distance-scaling models: (1) calculate slant estimates from disparity,

*^S*

_{d}, from the modeled distance estimate and

*HSR*s for the cue-conflict and no-conflict PSE stimuli; (2) calculate the cue-combined slant estimate (assuming

*^S*

_{t}was unbiased) for the PSE and conflict stimuli using Equations 1 and 2 and the weights determined from fits in Figure 6; (3) find the parameters for the distance model that yield the smallest squared difference between the cue-combined slant estimates for the cue-conflict and PSE stimuli.

*S*

_{d}= 30 deg and

*S*

_{t}= 40 deg).

*HSR*s for cue-conflict stimulus at 19.1, 57.3, and 171.9 cm were 1.22, 1.07, and 1.02, respectively. If a single distance estimate were used to scale these

*HSR*s, the slant estimates derived from the disparity cue (which objectively specifies 30 deg at the three distances) would be quite different. For example, if a distance of 57.3 cm were used in all three cases (

*^d*=

*C*), the perceived slant from disparity

*S*

_{d}would be 60, 30, and 10.9 deg, respectively. To get the combined slant estimate, we used Equation 1 with the weights determined from the fits to the 57.3-cm data in Figure 6 (in other words, the weights were determined only by HSR and the texture gradient and not by distance). The perceived slants for the cue-conflict stimuli at these three distances would be

*^S*

^{c},

*^S*

^{c}=

*^S*

^{c}

_{19}, or

*^S*

^{c}

_{57}or

*^S*

^{c}

_{172}. The objective slants of the no-conflict stimuli that appeared, on average, the same as the conflict stimuli were 34.1, 37.8, and 40.6 deg; these correspond to

*HSR*s of 1.26, 1.09, and 1.03. If these

*HSR*s were scaled by the distance estimate of 57.3 cm,

*^S*

_{d}, would be 63.8, 37.8, and 15.9 deg. The cue-combined slant estimate for these no-conflict stimuli would be

*^S*

^{nc}=

*^S*

^{nc}

_{19},

*^S*

^{nc}

_{57}, or

*^S*

^{nc}

_{172}. The error, minimized across all conditions simultaneously, was .

*^d*=

*C*model from three lines of evidence. (1) In the condition in which texture was perturbed and disparity-specified slant was 0 deg (black open diamonds in center row of panels in Figures 10 and 11), the

*^d*=

*C*model predicts that distance should have no effect on PSEs. According to this model, a single distance estimate is used to scale

*HSR*s, so

*HSR*s of the PSEs should be the same at all three distances. Figure C1 plots

*HSR*as a function of conflict for the

*S*

_{d}= 0, texture-perturbed condition. There is clearly a systematic effect of distance on the

*HSR*s of the PSEs, which indicates that there was some compensation for distance. (2) For the distance that provided the best fit to the data, a stimulus that had an objective slant of 30 deg would have appeared to have a slant of 65 deg. This is inconsistent with the phenomenology. (3) In a control experiment (2), we found that observers take distance into account when performing the slant-discrimination task.

*^d*=

*α*·

*d*+

*β*). The former has no free parameters and the latter has two free parameters (

*α*and

*β*). By Occam’s Razor, the optimal weighting model is preferred, but we hasten to point out that the latter cannot be rejected without further experimentation.

*x*-axis is parallel to the interocular axis. The

*z*-axis lies in the plane of fixation and is perpendicular to the interocular axis. The

*y*-axis is perpendicular to the other two axes.

*μ*. The plane’s slant is

*S*

_{C}= 0°, and its tilt

*τ*

_{C}is undefined. From the viewpoint of the left eye (equivalent to placing the origin there), . From the right eye’s viewpoint, or, equivalently, .

*S*

_{C}= Δ and

*τ*

_{C}= 0°), the texture-specified slants and tilts in the left and right eyes vary, but the tilts do not. Technically, for rotations about a vertical axis, the tilt parameter for eye-centered viewpoints flips between 0 and 180°. This corresponds to a sign change in slant. For simplicity, we used a sign change in slant, rather than changing the tilt parameter, for vertical-axis rotations. The eye-centered slants specified by the texture gradients are so they differ by

*μ*. The eye-centered, texture-specified tilts are

*τ*

_{L}=

*τ*

_{R}= 0. The left panel of Figure A1 shows eye-centered slants and tilts as a function of Cyclopean slant for

*τ*

_{C}= 0°.

*S*

_{C}= Δ and

*τ*

_{C}= 90°). For simplicity, we now use positive slants and let the tilt flip from 0 to 180° when necessary. The texture-specified slants are equal in the two eyes: . But the tilts differ: . The right panel of Figure A1 shows these eye-centered slants and tilts.