When a surface is illuminated by an extended light source, it can create some interesting effects in the overall pattern of luminance. For some regions of the surface, a portion of the extended source will be occluded by other regions, which produces a partial shadow that is often referred to as a penumbra. Another term to describe this effect is
vignetting. As described by Koenderink and van Doorn (
2003), this is derived from the French word vignette that refers to an ornamental frame that occludes part of the framed picture. In the 19th century these frames were often composed of stylized leaves of a grapevine (
Fr. Vigne).
Vignetting is most pronounced when observing outdoor scenes on a cloudy day. The illumination in that case can be approximated by a hemispherical dome (the sky) that radiates large amounts of light, with an opposing dome (the ground) that radiates relatively little light. This is typically referred to as diffuse illumination. Langer and Zucker (
1994) were the first to perform a mathematical analysis of surface shading under diffuse illumination. They argued that vignetting is the primary cause for variations of luminance in that case, so that the overall illumination in any given region will be primarily determined by the proportion of the sky that is visible from that region, and that convex regions will in general be more strongly illuminated than concave regions. As a first approximation, they proposed a simple “darker-is-deeper” heuristic for the analysis of shape from shading under diffuse illumination. However, they also made clear that the pattern of luminance is also influenced by the widths of valleys, so that darkness and depth are only roughly correlated. Stewart and Langer (
1997) later proposed a more accurate model of diffuse illumination that also takes into account the orientation of incident light that illuminates a surface, and approximates the effects of interreflections.
Tyler (
1998) performed a similar analysis of shading for sinusoidal surfaces with both directional and diffuse illumination. As part of this analysis, he considered an image like the one shown in
Figure 1A and noted that the bright regions appear closer in depth and the darker regions appear farther away. He argued that this perception is incompatible with any form of directional illumination, and, based on this observation, he concluded that diffuse illumination must be the default assumption for the perception of 3D shape from shading. A similar argument has more recently been proposed by Chen and Tyler (
2015) based on the appearance of a linear sinusoidal intensity grating like the one shown in
Figure 1B.
This is a surprisingly broad conclusion given that it is based on the perceptual appearance of just two images. Another reason to be skeptical of this claim is that the images on which it is based were created by directly manipulating 2D image intensities, as opposed to simulating the pattern of shading on a 3D surface from diffuse illumination. In this article we will present a large number of examples to show what actually happens when surfaces are viewed under directional and diffuse illuminations. The results will demonstrate that vignetting is but one of several factors that can influence the pattern of shading on a surface, and that the darker-is-deeper heuristic is generally invalid for all types of illumination.
Most of the images presented in this article were created using the Maxwell renderer, which is a physically accurate (unbiased) renderer that produces photo-realistic results, with a space of possible materials and lights that are all parameterized by measurable properties of the physical environment. Unlike most other renderers, Maxwell does not employ approximations or tricks to speed up the rendering process. For example, it does not allow one to turn off shadows or place an arbitrary limit on the number of bounces when computing global illumination. The only way of controlling the magnitude of indirect illumination is to manipulate surface reflectance. Maxwell is free for academic use. It will run on a typical office or laboratory computer, but the rendering speed in that case can be quite slow. It is best suited for a network-rendering environment. For example, the images in the present article each took approximately 10 min to render on a 64 node cluster.
Let us begin our analysis of the darker-is-deeper heuristic by providing a demonstration of vignetting under diffuse illumination.
Figure 2A shows images of a Lambertian circular disk with a diameter of 10 cm whose height was displaced with 1.5 cycles of a radial cosine function. The peak-to-trough amplitude of this function was 3.3 cm and its wavelength was 2.2 cm. The image at the top of column A was illuminated by a hemispherical dome light whose central axis was coincident to the line of sight. The diameter of this dome was 200 cm, and both it and the disk were centered at the same location in space. The image in the bottom row shows the same scene with a viewing direction that was rotated downward by 30° in order to depict the surface geometry more clearly. The silhouette in the second row of Column A shows the depth profile of a horizontal cross-section through the center of the image in the top row, and the luminance profile of this horizontal cross-section is shown just below it. The regions inside and outside the circular ridge are identified on this graph by the red-dashed and solid lines, respectively. Note that the regions inside the circular ridge for Column A are substantially darker than those on the outside. That is the effect of vignetting. It should also be noted, however, that there is a local luminance maximum at the deepest part of the circular valley. That effect is caused by Lambert's law, which states that luminance varies as a cosine function of the angle between the local surface normal and the direction of illumination (i.e., the angle of incidence). The light rays that illuminate the deepest part of the valley have much smaller incidence angles than those that illuminate the walls of the valley, and this is why there is a local luminance maximum there.
Figure 3 is designed to demonstrate this effect more clearly. The image on the left depicts a corrugated surface for which each half cycle has a semicircular depth profile. This particular geometry was selected because it is easy to compute the visible portion of the sky at each point (see Koenderink, van Doorn, Dana, & Nayar,
1999). In an effort to reduce the complexity of the luminance pattern, this image was created using the Finalrender renderer with interreflections turned off. The top two graphs on the right show the depth profile for one cycle of the surface corrugation, and the luminance variations over one cycle. The third graph from the top shows how much of the sky is visible from each point on the surface as a function of horizontal position. Note that the extent of the visible arc of the sky drops off quite rapidly from the apex of the convex regions of the surface, but that it remains almost constant in the concave regions. The bottom graph shows the cosine of the incident angle of the average illuminant, which is completely responsible for the luminance variations in the concave regions. Another thing to note in this graph is that the effects produced by variations in illuminant direction in the convex regions more closely resemble the actual pattern of luminance than those produced by the extent of the visible arc of the sky in those regions. This suggests that Lambert's law is an important component of luminance variation over the entire surface. Some observers report that the concave regions depicted in
Figure 3 can appear convex (or multistable), and that interpretation is consistent with the darker-is-deeper hypothesis.
Langer and Bülthoff (
2000) were the first to notice the local luminance maxima in the deepest regions of surface concavities under diffuse illumination, and they attributed this phenomenon to the orientations of the incident light relative to the local surface normals. They wondered if this might produce illusory bumps if observers' perceptions of shape from shading under diffuse illumination were based on a darker-is-deeper heuristic. They employed a depth discrimination task in order to test that hypothesis, and the results revealed that observer's depth judgments did not conform to a darker-is deeper rule. That is to say, the regions of concavities that appeared farthest in depth were often located on luminance maxima.
Another important factor that can influence patterns of luminance under diffuse illumination is surface interreflections (see Koenderink & van Doorn,
1983). When light enters a surface depression, it can bounce around many times before exiting or being absorbed, and these multiple bounces increase the total illumination. Note that this has the opposite effect of vignetting because it increases the luminance of concavities relative to convexities (Langer,
1999). Consider the images in
Figure 2. The surface depicted in
Figure 2A has a reflectance of 50%, so that half of the light was absorbed on each bounce. The one depicted in
Figure 2B, in contrast, has a reflectance of 90%, which allows a much greater amount of interreflection. As is shown in the luminance profile in the third row, the interreflections completely overcome the effects of vignetting so that the regions inside the circular ridge are noticeably brighter than those on the outside. This washing out of contrast for highly reflective surfaces has been noticed previously by Gilchrist and Jacobson (
1984), who showed that this provides a source of information for distinguishing black and white rooms in a scene of one reflectance.
The relative balance between vignetting and interreflections is also influenced by the depth-to-width ratio of convex regions. For the depicted circular valley in
Figure 2A, the depth of the valley is 3.3 cm and the wavelength of the radial cosine function is 2.2 cm. The surface depicted in
Figure 2C has the same reflectance and illumination as the one in
Figure 2A, but the depth of its valley is only 1.65 cm, which reduces the effect of vignetting. As is shown in the luminance profile in the third row, the difference between the inside and the outside of the circular ridge is greatly diminished relative to
Figure 2A.
In order to examine the predictions of the darker-is-deeper heuristic for the top row images in
Figure 2, a regression analysis was performed to get a quantitative estimate of the relationship between depth and luminance. If the analysis is restricted to the convex portion of the central bump, then the correlations between depth and luminance are 0.98, 0.96, and 0.95 for
Figures 2A,
2B, and
2C, respectively. This is not surprising because the effects of vignetting and Lambert's law are consistent with one another in that region, but the connection is also quite misleading. If the analysis is restricted to the concave portion of the circular valley, then the correlations between depth and luminance are all negative (−0.46, −0.12, and −0.14). If the analysis is performed over the entire surface, then the correlations between depth and luminance for these images are 0.23, 0.15, and 0.10. This analysis demonstrates quite clearly that the darker-is-deeper heuristic is remarkably bad at estimating the pattern of relief on a surface, even when its underlying assumption of fronto-parallel diffuse illumination is satisfied.
It is important to keep in mind that the evidence to support this heuristic is not based on images of actual surfaces, either real or simulated. Rather, it is based on the appearance of image intensity gratings like the ones presented in
Figure 1. Within that context, we were curious how closely the images in
Figure 1 conform to the patterns of luminance produced by an actual linear or circular sinusoidal surface under diffuse illumination. In order to address this issue we used the images in
Figure 1 as displacement maps to deform a fronto-parallel plane with a reflectance of 50%, and we illuminated the displacement surfaces with a hemispherical dome light whose central axis was coincident to the line of sight. The results of these simulations are shown in
Figure 4. Contrary to the claims of Tyler (
1998) and others, it is clear from these results that the patterns of luminance shown in
Figure 1 are fundamentally incompatible with what would be expected for a sinusoidal surface under diffuse illumination. This is because the unoccluded light rays that enter the valleys have smaller incidence angles with the deepest regions relative to the walls of the valley. Thus, the deepest parts have local luminance maxima as a result of Lambert's law.
Langer and Bülthoff (
2000) noted that a darker-is-deeper heuristic might still be able to account for the apparent relative depth inside concavities if images are blurred prior to the computation of 3D shape from shading (see also Schofield, Rock, & Georgeson,
2011). That could potentially explain the appearance of
Figure 4B. When blurred sufficiently, that image can be transformed into something that is quite similar to the one in
Figure 1B, but with much lower contrast. However, this hypothesis cannot explain the appearance of
Figure 4A. If that image is blurred sufficiently to remove the local intensity maxima in the concave regions, it creates a large, dark crater in the center that is not perceived by human observers. As is evident from this figure, the perceptual analysis of shape from shading is able to resolve surface undulations that have relatively high spatial frequencies, an outcome which would not be possible if images were blurred prior to the computation of 3D shape.
In order to examine how luminance patterns in concavities and convexities are influenced by different types of illumination, we created a roughly cylindrical object with a prominent valley in its center (see
Figures 5 and
6). The cylindrical part of this object was modeled by extruding a spline curve, and the caps at the top and bottom were created by lathing the same curve. Thus, the occlusion boundaries of the caps are identical to the relief of a horizontal cross-section through the cylindrical part. The horizontal and vertical dimensions of the object were 8.8 cm and 9.5 cm, respectively, and its reflectance was 50%. This object was rendered with nine different illumination conditions including a hemispherical dome light, a 50 cm
2 area light, and a 1 cm
2 area light that approximated a point source. These lights were positioned at a distance of 100 cm from the object, and rotated so that the average direction of illumination could be 0°, 35°, or 70° relative to the viewing direction.
The images for each type of light source in the 0° condition are shown in
Figure 5, together with graphs of the luminance functions through the centers of each one. In that special case where the average direction of illumination is coincident with the line of sight, the points that are closest in depth are always located at local luminance maxima, as are the points that are farthest in depth, a result which violates the darker-is-deeper rule. Note that this occurs for all three of the different light sources.
Figure 6 shows the images for each type of light source in the 35° and 70° conditions, together with the luminance profiles for each one. As the average direction of illumination becomes more and more eccentric, the local luminance maxima are shifted away from the local depth extrema. The one situation for which the darkest points in a concavity were also the deepest points was when the 70° area lights cast a shadow over the concave region of the object, but, in that situation, the brightest points of the object were far from the points that were closest in depth.
The most important thing to note in these figures is that the luminance extrema on a surface are almost never located on its depth extrema, except in the special case where the average direction of illumination is coincident with the line of sight. If a darker-is-deeper heuristic were used to estimate 3D shape from shading, the resulting estimates would be systematically distorted relative to the ground truth in almost all conditions. That being said, this heuristic might still be considered as a possible psychological model if changes in the direction of illumination also produced systematic distortions in observers' perceptual judgments. As it turns out, there is some empirical evidence to show that the apparent positions of local depth extrema are shifted slightly toward the light source for oblique directions of illumination (Christou & Koenderink,
1997; Egan & Todd,
2015; Koenderink, van Doorn, Christou, & Lappin,
1996a,
1996b), but the magnitude of these shifts is much smaller than what would be expected from a darker-is-deeper heuristic. Egan and Todd (
2015) analyzed the judged patterns of relief for images of shaded surfaces with oblique directions of illumination and found that the darker-is-deeper hypothesis accounted for only 7% of the variance, as opposed to 80% that was accounted for by the simulated pattern of relief.
Is there any possible situation for which a darker-is-deeper heuristic could provide a reasonably accurate estimate of 3D shape? If the average direction of illumination is coincident with the line of sight, that will ensure that local depth minima will always correspond with local luminance maxima, but how can we make the deepest points coincide with luminance minima? The reason this did not occur for any of the simulations presented thus far is that the deepest part of the depressions had fronto-parallel orientations, so they reflect more light than the more slanted regions along the walls of a concavity because of Lambert's law. However, suppose that the deepest part of a depression ended in a cusp, which is actually quite common with malleable materials such as skin or cloth. The deepest part of a depression in that case would have the largest incidence angles, and the effects of Lambert's law would be consistent with the effects of vignetting.
In order to examine the appearance of cusp depressions, we created a new object similar to the one in
Figures 5 and
6 but with a different depth profile. The cylindrical part of this new object was modeled by extruding a spline curve that contained a cusp in its center, and the caps at the top and bottom were created by lathing the same curve.
Figure 7 shows three renderings of this object with a small area (point) light, a large area light, and a hemispherical dome light, for which the average direction of illumination in each condition was coincident with the line of sight. Note that a darker-is-deeper heuristic provides a qualitatively correct estimate of 3D shape for cusp depressions for all three types of illumination. It is important to keep in mind, however, that the success of the heuristic in this special case can only be achieved if the light field is aligned appropriately with the viewing direction. Otherwise, the luminance maxima will be shifted away from the local depth extrema.
Given the limitations of a darker-is-deeper heuristic with respect to the direction of illumination, Sun and Schofield (
2012) have proposed that the perception of shape from shading has two distinct modes: one involving a darker-is-deeper heuristic that is applied when the direction of illumination is reasonably close to the observer's line of sight; and another involving a linear shape from shading heuristic that is applied for more oblique illuminations. This latter proposal is based on an analysis by Pentland (
1989) that decomposed Lambertian shading from collimated light fields into linear and quadratic components. As the direction of illumination becomes more and more oblique, the linear component becomes more and more dominant in the pattern of surface reflections. However, one serious problem with this analysis is that it has no way of dealing with the presence of cast shadows. As is clearly demonstrated in
Figure 6, when a collimated light field becomes more and more oblique, the cast shadows on a surface become more and more prominent. Thus, Pentland's model is only valid for surfaces that have negligible variations in depth to prevent cast shadows, such as the bas-relief on a coin. This model was recently tested by Egan and Todd (
2015), who obtained local orientation judgments for randomly deformed spheres with moderately oblique directions of illumination. They found that the linear shape from shading heuristic accounted for only 8% of the variance in observers' judgments at different probe points.