Computational models for determining three-dimensional shape from texture based on local foreshortening or gradients of scaling are able to achieve accurate estimates of surface relief from an image when it is observed from the same visual angle with which it was photographed or rendered. These models produce conflicting predictions, however, when an image is viewed from a different visual angle. An experiment was performed to test these predictions, in which observers judged the apparent depth profiles of hyperbolic cylinders under a wide variety of conditions. The results reveal that the apparent patterns of relief from texture are systematically underestimated; convex surfaces appear to have greater depth than concave surfaces, large camera angles produce greater amounts of perceived depth than small camera angles, and the apparent depth-to-width ratio for a given image of a surface is greater for small viewing angles than for large viewing angles. Because these results are incompatible with all existing computational models, a new model is presented based on scaling contrast that can successfully account for all aspects of the data.

*σ*) at the center of each texture element can be determined by the following equation:

*λ*and

*ω*are the major and minor axes of a texture element's optical projection. We will refer to this approach as the analysis of texture foreshortening because that is the term that is typically used in the literature to describe the ratio

*ω*/

*λ*. Similar computations can also be performed for less regular isotropic textures from the distribution of edge orientations in each local image region (Aloimonos, 1988; Blake & Marinos, 1990; Blostein & Ahuja, 1989; Marinos & Blake, 1990; Witkin, 1981) or from the relative anisotropy of their local amplitude spectra (Bajcsi & Lieberman, 1976; Brown & Shvayster, 1990; Krumm & Shafer, 1992; Sakai & Finkel, 1994; Super & Bovik, 1995).

*σ*) in a given local region can be determined by the following equation

*δ*is the projected distance between neighboring optical texture elements in the direction that slant is being estimated, and

*λ*

_{1}and

*λ*

_{2}are the projected lengths of those texture elements in a perpendicular direction (see Figure 2). In the limit of an infinitesimally small

*δ,*the right side of Equation 2 equals the normalized depth gradient (Purdy, 1958, Equation 14; Gårding, 1992, Equation 33). Similar computations can also be performed on less regular textures from the affine correlations between the amplitude spectra in neighboring image regions (Clerc & Mallat, 2002; Malik & Rosenholtz, 1994, 1997) or from the systematic changes in the distributions of edges (Gårding, 1992, 1993).

*α*) of their asymptotic lines relative to the frontoparallel plane (see Figure 5). Because prior research has shown that the apparent depth of a textured surface is attenuated by reduced camera angles or negative signs of curvature (Todd, Thaler, & Dijkstra, 2005), we increased the simulated depths in those conditions in an effort to ensure that none of the stimuli would appear completely flat. Thus, for the concave surfaces with 20° camera angles, the possible values of

*α*were 55°, 60°, and 65°. For the concave surfaces with 60° camera angles and for the convex surfaces with 20° camera angles, the possible values of

*α*were 50°, 55°, and 60°. Finally, for the convex surfaces with 60° camera angles, the possible values of

*α*were 45°, 50°, and 55°. Note that all of the different combinations of camera angle and sign of curvature included a common asymptotic angle of 55° (see Figure 6).

*P*

_{1},

*P*

_{2},

*P*

_{3}, and

*P*

_{4}), which observers could manipulate by adjusting four sliders with a hand held mouse to match the apparent cross-section in depth of the depicted surface (see Figure 8). Because some of the stimuli did not appear to have hyperbolic cross-sections, the adjustment space required four degrees of freedom to adequately match the observers' perceptions in all of the different conditions. The shape of the adjustment figure was defined by the following equation:

*P*

_{1}controls the angle of the asymptotic lines and

*P*

_{2}controls the curvature at its midpoint. The third term with parameters

*P*

_{3}and

*P*

_{4}was included so that the asymptotic lines of the hyperbola could be made to bow inward or outward. The adjustment figure had a fixed width of 5.71 cm and had a height that varied from 0 to 13.2 cm.

*P*

_{1}with the appropriate slider to indicate the overall perceived relief of the depicted surface. Next, they adjusted the parameter

*P*

_{2}to indicate the apparent curvature at its midpoint. Then, if necessary, they adjusted the parameters

*P*

_{3}and

*P*

_{4}to add some curvature to the asymptotic lines. Once observers were satisfied with their settings, they could move on to the next trial by clicking on a button that was labeled “next.” All observers agreed that these response tasks were quite natural and that they had a high degree of confidence in their settings.

^{1}

*S*′) of an optical texture element in radians can be closely approximated by the following equation:

*S*is the diameter of a physical texture element and

*D*is its distance from the point of observation (see Figure 10). One way of describing the global variations of scaling within an image is to incorporate a measure that will be referred to here as

*scaling contrast,*as defined by the following equation:

*S*′

_{max}and

*S*′

_{min}are the maximum and the minimum projected texture lengths, and

*D*

_{max}and

*D*

_{min}are the maximum and the minimum distances on a surface relative to the point of observation. Note that scaling contrast provides a reliable estimate of the surface depth contrast that is invariant over the size of the physical texture elements.

^{2}

*S*′

_{ ϕ}) of an optical texture element that was centered at each visual angle (

*ϕ*) along a horizontal cross-section through the center of each display. For the displays that were viewed at the correct visual angle, the pattern of projected lengths in an image can be used to determine the veridical depth-to-width ratio and shape of a horizontal surface cross-section using the following equations:

*X*

_{ ϕ}defines the horizontal position of each point along the surface cross-section, and

*Z*

_{ ϕ}defines its position in depth. It is clear from Figure 13, however, that observers did not adopt that strategy. We have tried several different procedures to model the judged shapes from the spatial variations of

*S*′

_{ ϕ}, and the best fits to the data were obtained using the following equations:

*Z*

_{ ϕ}) of each point on a cross-section varies from 0 to the value of scaling contrast times a constant

*k*. The horizontal position of each point (

*X*

_{ ϕ}) is linearly scaled with visual angle and is normalized by the maximum visual angle (

*ϕ*

_{max}) so that the range of positions varies between −1 and 1. An alternative procedure that produces fits only slightly worse than those obtained using Equation 8 is to scale

*X*

_{ ϕ}with respect to position in the image plane (i.e.,

*X*

_{ ϕ}=

*X*′

_{ ϕ}/

*X*′

_{max}).

*k*), which was assigned a fixed value of 2.5 for the generation of each curve. The other free parameter is the restricted 45° range of visual angles that was considered for the analysis of convex surfaces (see Figure 9). Note in Figure 13 how closely the estimated shapes approximate the observers' settings in all of the different conditions. This model can account for the apparent depth-to-width ratios of the depicted surfaces, the apparent outward bowing of the surface asymptotic lines for convex surfaces, and the attenuated curvature at the near point for convex surfaces.

^{3}

^{1}In an extensive series of experiments, Li & Zaidi, (2000, 2001) have consistently shown that observers are unable to accurately perceive the sign of curvature for surfaces with isotropic textures. Because several of the textures in the present experiment were statistically isotropic, the accuracy of observers' judgments may appear at first blush to be fundamentally incompatible with those earlier studies. The solution to this conundrum is suggested by Todd et al. (2005) who showed that observers can accurately determine the sign of curvature for surfaces with isotropic texture if the visual angle of the depicted surface patch is sufficiently large (see also Saunders & Backus, 2006). The camera angles used in this study were all well above threshold for performing this type of judgment.

^{2}The values of scaling contrast shown in Figure 12 were computed from the projected lengths of idealized texture elements along a horizontal cross-section through the center of each image and did not take into account the variability of the physical texture elements in the variable size dot and the variable shape ellipse textures. To obtain a stable measure of texture scaling when there are random variations among the physical texture elements, it is necessary to average the projected sizes of the optical texture elements over an appropriately large neighborhood. A more detailed discussion of this issue is presented in Thaler et al. (2007).

^{3}Consider, for example, a planar surface at a 50° slant relative to the frontoparallel plane that is observed with a viewing angle of 60° (see Figure 1). The local optical slants across different regions of the surface in that case would vary from 20° to 80°. To account for the results of this study using a variable bias hypothesis, the systematic underestimation of slant would have to vary between 20% and 80% over that range of optical slants (see Figure 14). The change in apparent curvature that would cause is much larger than what is evident in observers' judgments.