Perception of light source distance from shading patterns

Abstract

Varying the distance of a light source from an object alters both the intensity and spatial distribution of surface shading patterns. We tested whether observers can use such cues to infer light source distance. Participants viewed stereoscopic renderings of rough objects with diffuse and glossy surfaces, which were illuminated by a point source at a range of distances. In one task, they adjusted the position of a small probe dot in three dimensions to report the apparent location of the light in the scene. In a second task, they adjusted the shading on one object (by moving an invisible light source) until it appeared to be illuminated from the same distance as another object. Participants' responses increased linearly with the true light source distance, suggesting that they have clear intuitions about how light source distance affects shading patterns for a variety of different surfaces. However, there were also systematic errors: Subjects overestimated light source distance in the probe adjustment task, and in both experiments, roughness and glossiness affected responses. We find the pattern of results is predicted surprisingly well by a simplistic model based only on the area of the image that exceeds a certain intensity threshold. Thus, although subjects can report light source distance, they may rely on simple—sometimes erroneous—heuristics to do so.

Introduction

Background

Although illumination is thought to play a central role in surface perception, surprisingly little is known about how we estimate properties of the illumination itself. The image of a surface depends to a large extent on the incident illumination. To infer the shape or reflectance properties of a surface from shading patterns, the visual system must somehow factor out the contribution of the illumination to the observed image. This discounting of the illuminant could be achieved in part through the use of image measurements that are sensitive to shape or reflectance, while being relatively insensitive to (or, in the limit, invariant across) changes in the illumination. However, it is also widely believed that to compute surface reflectance or shape from shading patterns, the visual system may also make use of an explicit estimate of the illuminant (von Helmholtz, 1925; Horn & Brooks, 1989; Adelson & Pentland, 1996; Mamassian & Kersten, 1996; Yang & Maloney, 2001; Boyaci, Doerschner, & Maloney, 2006; Boyaci, Doerschner, Snyder, & Maloney, 2006). Indeed, when we look at a typical scene, we usually have a clear subjective impression of at least some characteristics of the prevailing lighting conditions, including the relative intensity, direction, and diffuseness of light flow within the scene (Koenderink, Pont, van Doorn, Kappers, & Todd, 2007; Mury, Pont, & Koenderink, 2007, 2009). Photographers, architects, and lighting designers use a wide variety of terms to describe the specific qualities of light and their effects on the appearance of scenes (Russell, 2012). This suggests that illumination is certainly not fully discounted (i.e., eliminated from the percept) but rather constitutes a key aspect of our subjective impression of the scene. Thus, in addition to its central role in the perception of shape from shading and surface material properties, the perception of illumination and light sources is worthy of scientific study in its own right with applications in many domains. 

Numerous studies have investigated the successes and errors that observers make when estimating object properties under different illumination conditions. Despite the remarkable constancy that we experience in everyday life, these studies show that color (Katz, 1935; D. H. Brainard & Wandell, 1992; Hurlbert, 1998; Gegenfurtner, 2003; Maloney, Gerhard, Boyaci, & Doereschner, 2010; Foster, 2011), lightness (Gilchrist, 1977; Gilchrist, Delman, & Jacobsen, 1983; Gerhard & Maloney, 2010a), shape perception (Berbaum, Bever, & Chung, 1983; Koenderink, van Doorn, Christou, & Lappin, 1996; Caniard & Fleming, 2007), and the perception of material properties (Fleming, Dror, & Adelson, 2003; Fleming & Bülthoff, 2005; Motoyoshi & Matoba, 2012; Zhang, de Ridder, & Pont, 2015) depend on the illumination in the scene. Of particular note for the current study are the findings that surface relief and gloss interact in complex and often nonintuitive ways with changes in illumination (Ho, Landy, & Maloney, 2006, 2008; Te Pas & Pont, 2009; Wijntjes & Pont, 2010; Fleming, 2012; Marlow, Kim, & Anderson, 2012). 

In comparison to the vast literature on the perception of object properties, research on the perception of illumination sources has been relatively scarce and has focused primarily on the perception of the direction, intensity, and diffuseness of light in the scene. Koenderink, van Doorn, Kappers, te Pas, and Pont (2003) investigated whether observers were able to judge the direction of illumination of textured surface patches. They found that subjects could estimate the azimuth of the illumination direction to within approximately 15° but the elevation only just above chance level. 

Along similar lines, Pont and Koenderink (2007) measured the perception of illumination direction and directedness from images of shaded objects. As with the textures, they also found that human observers can match the azimuth of illumination quite accurately (i.e., veridically and with low uncertainty) but that elevation estimates were highly variable and that subjects performed poorly on some objects. They also found systematic interactions between the parameters of the illumination, suggesting that subjects could not fully separate direction from directedness. However, in this study, the object properties were not varied parametrically, so it remains unclear whether there are systematic interactions between object properties and the perception of illumination. Moreover, light was varied from collimated to diffuse, but the distance of the light source was not manipulated. Thus, it is unclear whether subjects can estimate light source distance from shading patterns. 

O'Shea, Agrawala, and Banks (2010) found that subjects can judge the illumination direction more accurately if they have additional cues for the shape of the illuminated object, such as surface texture or bounding contour. This indicates that subjects incorporate shape information into their lighting direction estimates. Furthermore, in line with many previous studies (Brewster, 1826; Hershberger, 1970; Berbaum et al., 1983; Kleffner & Ramachandran, 1992; Sun & Perona, 1998; Mamassian & Goutcher, 2001; Adams, 2007), subjects seem to rely on a prior that light comes from above if there are no other shape cues than shading. However, this prior is relatively weak (Morgenstern, Murray, & Harris, 2011) and is easily overridden by training (Adams, Graf, & Ernst, 2004), suggesting that the visual system does not rely solely on assumptions to infer light source direction but rather depends heavily on explicit sensory estimates of prevailing lighting conditions. It is such sensory estimates that we seek to investigate here. 

Arguing in a similar direction, Gerhard and Maloney (2010b) investigated how subjects perceive the direction of motion of a light source over random hilly surfaces. They found that all subjects could report the movement direction of the light source. Furthermore, a model based on noisy perception of shape could explain subjects' classifications well and better than a model based on noisy perception of shading. This further supports the notion that shape information could play a potential role in the perception of light source position. 

In another study, Koenderink et al. (2007) investigated the more general question of the extent to which subjects perceive the light field in scenes. They showed subjects images of scenes with various lighting conditions and superimposed a spherical probe at various points within the scene and asked subjects to alter the intensity, direction, and diffuseness of the illumination on the probe until it appeared to be consistent with the local surroundings in the scene. That is, they had to report the expected appearance of the spherical probe, given the perceived light field for that scene. The authors found that subjects were remarkably consistent and quite accurate at matching the appearance of the probe to the prevailing light field, although they performed less well when the probe was located within volume shadows. 

The fact that subjects can predict the effects of complex, spatially varying light fields on the appearance of shaded surfaces suggests the visual system may derive quite sophisticated estimates of the light field. Nevertheless, to our knowledge, no previous studies have investigated whether this enables observers to infer the distance of light sources from the surfaces they illuminate. In general, inferring the properties and positions of light sources themselves from observations of the light field is a nontrivial computational problem. However, artificial light sources that can be held close and moved at will—such as candles, lanterns, and torches—provide us with everyday experience with the consequences of light source position (including distance) on real-world scenes. Moreover, dramatic use of proximal lighting in architecture, paintings, photography, and film suggests that we are at very least sensitive to the effects of light source distance and likely have specific intuitions about how shading and shadows indicate light source distance. However, it remains unclear whether these intuitions take the form of explicit estimates of the light source position derived by sophisticated inverse optics computations or whether they result from heuristics based on simple image measurements. Therefore, we sought to (a) investigate experimentally the conditions under which observers can infer light source distance and (b) develop a simple image-based model of their performance, to test the plausibility of heuristic-based approaches. 

Physical effects of light source distance

When a point source is moved closer to a surface, both the intensity and the spatial pattern of shading on the surface change systematically. The closer the light source is to the surface, the brighter and more focused the shading pattern becomes (Figure 1). 

Figure 1

 

Physical effects of light source distance on shading patterns. (A) At large distances, rays are approximately parallel, and direct shading depends solely on the orientation of the surface. (B) At small distances, both distance and changes of angle contribute significantly to shading. Because d₂ is longer than d₁, it is correspondingly dimmer as a result of the inverse square law. Similarly, although the surface normal is constant across the surface, because θ₂ is larger than θ₁, the surface appears correspondingly dimmer. Together these effects cause a rapid falloff in shading intensity for proximal light sources, which does not occur with distant sources.

For a diffusely shaded surface of given albedo, directly illuminated by a point light source of fixed intensity, image intensity varies as a function of two parameters: (a) the angle between the surface normal and the light rays (the cosine law) and (b) the distance from the light source (the inverse square law). For distant sources, the light rays are effectively parallel, so direct shading is due almost exclusively to the three-dimensional (3D) orientation of the surface normals. Depth variations on the surface are too small compared with the distance of the light source to have any effect, and the angle between the surface normals and ray vectors depends solely on the orientation of the surface. 

By contrast, when the light source is brought very close to the surface, the contributions of both factors change. The angle between surface normals and the point source varies from location to location, even when the surface normal is constant. Whereas a plane illuminated by a distant source yields uniform intensity across the surface, when the light source is brought very close, intensity varies across the plane because the vector to the light source changes from location to location. In addition, when the light source is brought close to the surface, the inverse square law starts to play a role. The overall intensity of the shading pattern increases, and differences in distance between various points on the surface and the light source can significantly affect their intensity in the image. Points that are close to the light source are brighter than those that are farther away. Together, these effects create a bright, localized highlight on the surface that grows in intensity and shrinks in radius as the light source is brought close to the surface. Note, of course, that this highlight is different in origin from the highlights created by specular reflection. 

Thus, these simple physical effects mean that surface shading patterns provide potentially useful information about light source distance, based on the intensity and spatial fall-off of intensity on the surface. Our goal here was to measure the conditions under which observers can exploit these cues to infer light source distance. It should be noted that these effects are most pronounced for ideal (infinitesimal) point sources. As light source size increases, illumination becomes more diffuse, with a wider range of ray angles receiving light from the source for each surface normal. This would have the effect of blurring out shading transitions and shadows across the surface, which would weaken the cues to light source distance. However, for the current study, we focus on ideal point sources, where the cues are most pronounced, to test whether there are any conditions under which the visual system can estimate light source distance from shading patterns. 

The specific form of the intensity profile created by a surface also depends on the object's shape and reflectance properties. We therefore varied the glossiness and meso-scale surface roughness (texture) to measure how they affect light source position judgments. Glossy surfaces produce specular highlights in addition to the shading patterns. For an ideal mirror-like surface and an infinitesimal point source, the specular reflection of the source is a Dirac delta function. Only the intensity and position of the specular reflection on the surface vary as the light source is brought closer. However, for glossy surfaces, which are not perfectly smooth on a micro-scale, the light is spread out around the reflection direction. In technical terms, the surface then has a specular lobe with nonzero spread. Similarly, even the reflection of a small light source varies in size with the light source distance once it is not a point anymore. Thus, under more realistic conditions, the size of the specular highlight varies with light source distance. From a visual point of view, the main effects of meso-scale surface roughness are (a) to break up the boundary of the highlights, potentially making it harder to measure the spatial falloff of intensity, and (b) to provide additional information from shadowing about the local illumination direction at each point on the surface (Koenderink & Pont, 2003). 

Here, we sought to measure whether the additional effects introduced by glossiness and visible surface perturbations aid, hinder, or change the perception of light source distance. 

Experiments

To investigate the perception of light source distance, we used two different tasks. In one experiment, we asked subjects to report the perceived light source position using a probe they could move in a virtual scene. In a second experiment, we asked subjects to adjust the pattern of shading on the surface of one object until it appeared to be illuminated from the same distance as another object. Finally, we tried to find image statistics that could predict the subjects' performance, to understand better which cues subjects draw on to estimate light source distance. 

As mentioned above, to test constancy, we altered the object's surface properties. We varied roughness, as Ho et al. (2006) and others have shown that perceived roughness varies with the direction of the illumination using a matching task, and similarly, Boyaci, Maloney, and Hersh (2003) showed an effect of illumination direction on roughness estimates. Other studies (Ho et al., 2008; Marlow et al., 2012) found that surface texture perception (roughness or bumpiness) interacted with the glossiness of an object. Thus, we varied the glossiness of the illuminated object as well, to test whether the resulting specular highlights interfered with the perception of light source distance. 

Methods

Experiment 1

In the first experiment, we asked subjects to adjust the distance of a light probe to their perceived light source distance. 

Observers

Eighteen students from the Department of Psychology and Sports Science at the University of Giessen (mean age = 24.1 years, SD = 6.1 years, 16 female) participated in the experiment for course credits. All observers reported having normal or corrected-to-normal vision, reported seeing the stereoscopic 3D stimuli in depth, and were unaware to the purpose of the experiment. Written informed consent was given by all observers prior to the start of the experiment, and study procedures were conducted in accordance with the Declaration of Helsinki and approved by the local ethics commission of the University of Giessen. 

Stimuli

The stimuli were stereoscopic renderings of a virtual scene with a spherical object floating above a glowing grid, as shown in Figure 2. These images were rendered with Blender (Roosendaal, 1998) as 1,024 × 1,024 pixel images. We rendered the 3 × 2 × 2 × 5(× 2) = 60(120) images of the illuminated sphere in three levels for roughness, two for glossiness, two for illumination angles, and five for light source distance, for each eye prior to the whole experiment. 

Figure 2

 

Some example stimuli. The left two columns show the image for the left eye for the smooth and rough conditions. The right two columns are uncrossed stereo pairs for the intermediate roughness condition. All stimuli displayed here are matte/lambertian. From top to bottom, the light source distance increases. The distances of the light source are 10, 16, and 22 Blender units, the limits and the mean of the range we tested.

For stereoscopic rendering, we adjusted the camera such that the viewing angle corresponds to the image size on the screens and moved the camera by 3.25 blender units to the left and to the right in the virtual scene for the images for the two eyes. Using this procedure, blender units should be equivalent to centimeters in the real world for subjects with an interpupillary distance of 65 mm. As the interocular distance varies between subjects, perceived depth in the scene varies by a small factor as well. Thus, we report all distances in blender units of the virtual scene, although they are equivalent to centimeters for a standard observer. 

The exact positions of the objects are displayed in Figure 3. They were illuminated by three light sources: (a) a weak hemispherical light from above (with energy 1), ensuring that all visible surface points yielded nonzero image intensities; (b) a lamp light source (with energy 1), also from above to get a more natural shading pattern; and (c) a bright point light source to be estimated by the subject, placed at 10, 13, 16, 19, or 22 Blender units distance to the object's center within the x-y plane (passing through the object's center). This light had a constant energy of 23 at a falloff distance of 25 Blender units. All lights and the object's reflectance were achromatic. The hemisphere and lamp sources were constant over all stimuli. As the bright point light source moved farther away from the object, its relative contribution to the lighting decreased somewhat, leading to a shift in the average direction of the lighting toward the vertical direction (i.e., the direction of the other, constant, sources). The contribution of the bright source to the shading on the surface was visible within the entire range of distances we tested. 

Figure 3

 

The layout of the virtual scene. All lengths are given in Blender units, which were equivalent to centimeters when the observer's interpupillary distance was 65 mm. The left is a plan view. The right is a side view. The light to be estimated was always placed in the x-y plane, and the distances of the light source were measured from the center of the object.

The adjustable light source itself was not rendered, even though its location always lay within the view frustum. However, a small stereoscopic white disk indicated the current setting of the subject. This approach was necessary because rendering the true location of the light source would have made the task trivial, as observers could achieve perfect performance simply by placing the probe at the location of the visible source. In practice, observers reported no problems interpreting the white probe as the origin of the light visible in the scene. 

As an additional 3D cue, we added a ground plane with a grid texture, 10 units below the center of the sphere, which neither received nor emitted any lighting, shading, or shadows. This grid was simply to facilitate the perception of relative depth between the shaded object and the probe. 

The object was a high-resolution ico-sphere (approximately 2 million faces) with a random 3D texture (“clouds,” depth = 5) applied as a displacement map to the surface. The roughness was adjusted by setting the amplitude of the displacement map to 0.025, 0.225, and 0.425 in the arbitrary units of Blender. The matte spheres were lambertian with an albedo of 0.25. For the glossy spheres, we added an achromatic Cook-Torrance specular component with an intensity of 0.5 and a hardness of 50 (i.e., the spread of the specular lobe). Thus, the glossy objects reflected 75% of the incoming light in total. 

Procedure

In each trial, the subject was presented with the stimulus as a stereo image with a small filled white circle (eight-pixel diameter) representing the light source superimposed in stereo. This probe could be moved in 3D along a straight line within the x-y plane radiating from the center of the sphere by adjusting the mouse. Thus, only the distance of the probe could be adjusted by the subjects. The subjects were allowed to use as much time as they needed to move the probe to the perceived light position. Having set the probe to the desired location, they proceeded to the next trial by clicking the mouse button. Each trial started with the probe at a different random position. There was no feedback by the computer or the experimenter about the performance of the subject at any time. A commented picture of the subject's view can be found in Figure 4. 

Figure 4

 

Illustration of the subject's task in Experiment 1. The task was to place the light probe at the position from which the subject thought the light came. The probe could be moved radially to the object using the mouse. The line, arrows, and text were not displayed.

The experiment consisted of three sessions. It started with a training session with one repetition of each stimulus, with a random gloss level, followed by one gloss session and one matte session with five repetitions each in random order. Each session contained objects with all three different roughness levels, both illumination angles, and all five light source distances. 

In the training session, the subjects were instructed about the stereoscope and how to set the light probe using the mouse. They were also informed that the light source direction was always correct. Then they performed one repetition of the stimulus set with the instructor present in the room. Nine subjects did the training with matte stimuli, the other nine with glossy stimuli. 

Directly after the training session, all subjects performed both the matte and the glossy experimental sessions without further instructions, after the experimenter had left the room. The stimuli were presented in five blocks of one repetition of all stimuli each, presented in random order. 

Apparatus

The dual-monitor Wheatstone stereoscope consisted of a chin rest with two mirrors in front of it viewing two Dell 19-inch thin-film transistor monitors with 1,280 × 1,024 pixel resolution and 75-Hz refresh rate. The light path from the eye to the monitor was 55 cm. The stimuli were presented in the middle of the screen using Psychtoolbox 3 in Matlab (D. Brainard, 1997; Kleiner, Brainard, & Pelli, 2007) on a Windows XP PC with a NVIDIA FX550 graphics card. 

Experiment 2

In the second experiment, a different set of subjects performed an asymmetric matching task. They were asked to adjust the light source distance in a “match” scene to the illumination in a “test” scene that contained a different object. This asymmetric design means that even for the correct setting, the test and match images were not identical, ensuring that subjects could not perform the task by a simple pixel-based matching but instead had to abstract some higher-level information related to light source distance or its image correlates. 

Observers

Twenty-five students from the Department of Psychology and Sports Science at the University of Giessen (mean age = 22.24 years, SD = 0.48 years, 18 female) participated in the experiment for course credits. All subjects reported having normal or corrected-to-normal vision, reported seeing the stereoscopic 3D stimuli, were naive to the purpose of the experiment, and did not participate in Experiment 1. Written informed consent was given by all observers prior to the start of the experiment, and study procedures were conducted in accordance with the Declaration of Helsinki and approved by the local ethics commission of the University of Giessen. 

Stimuli

The test stimuli were the scenes from the first experiment down-sampled to a lower resolution of 512 × 512 pixels to fit two of them side by side on the screen. Presenting the images at a smaller size and moving them to the side somewhat distorts the stereo cues for depth. However, in this experiment, the observers did not have to perform absolute judgments of light source distance; the task simply required adjusting the shading patterns until the light source distance appeared maximally similar in the two stimuli, which requires no reference to any explicit point in space. Thus, small deviations in the stereo depth cues are unlikely to have influenced the results. 

For the matching stimuli, we rendered the same scene for 60 different distances of the point light source linearly spaced between 7 and 25 Blender units distance. This range exceeded the range of light source distances presented in the test stimuli (10–22). For the match object, a new random seed of the texture (with the intermediate roughness value) was used such that the object was different from the test in all cases. The reflectance of the object and the direction of the illumination in the matching stimuli were always the same as in the test stimulus. The light source itself was not rendered (even though its location always lay within the view frustum, as in Experiment 1). If the light source had been directly visible, it would have been unnecessary to use the shading patterns on the surface to perform the matching task: Participants could simply have equated the image positions of the visible light sources in the test and match stimuli. As we sought to test the extent to which subjects can equate the shading patterns created by sources at different distances, it was therefore necessary to exclude the source from the images. We did not include light source positions outside the view frustum for consistency with Experiment 1 and because the differences in shading patterns become very small for distant light sources. As in Experiment 1, no observers reported any difficulty with the task as a result of the invisibility of the source itself. 

Procedure

In each trial, we presented the test scene and the match scene side by side and asked the observer to adjust the distance of the (invisible) point light in the match scene until the object appeared to be illuminated from the same distance as in the test scene. The initial distance of the light source in the match scene was randomized on every trial. Using the “l” and “p” keyboard keys, the observers could move the light source along an invisible linear path radiating out of the center of the object, bringing the source closer and further to the object. Pressing the “a” key, the observer could move to the next trial. Observers were not given any feedback about their performance by the experimenter or the program. 

The structure of the experiment was the same as in Experiment 1. The experiment started with a training session with one block of one random gloss level, followed by matte and glossy experimental sessions with five blocks each. Which of the two experimental sessions was run first was chosen randomly. Within each block, each of the 30 different test stimuli (3 levels of roughness × 2 angles × 5 distances) was presented once in random order. This task is illustrated in Figure 5. 

Figure 5

 

A subject's view of the task of Experiment 2 for a rough glossy test object at 16 units distance. The task was to change the light source distance in the left image such that it matched the distance in the right image. The left image always showed an object with intermediate roughness, with the same reflectance and illumination direction as the right one but using a different random seed for the texture, preventing an exact image match.

As in Experiment 1, we blocked the trials by gloss level to allow the subjects to use the same object for matching throughout a block. The aim was to reduce task switching, which may have slowed the participants down. 

Data analysis

Analysis of variance

The dependent variable in all our analyses was the matched or set light source distance measured from the light source to the center of the sphere. We analyzed the influence of the different experimental conditions using standard analysis of variance (ANOVA) repeated measurement analyses in R. 

Image statistics

To understand the pattern of results in greater depth, we tested the extent to which the observers' settings could be predicted by some simple image statistics derived from the renderings. First, we converted all images to gray-scale in MATLAB with the function “rgb2gray.” Then we computed statistics on these images (related to the proportion of pixels above certain intensity thresholds) and built a linear regression model to combine these into a prediction for the subjects' settings. The specific image statistics are described in greater detail in the Results section. 

Results

Experiment 1

The mean distance estimates across all subjects and the most and least reliable subject's settings (with reliability defined as the total variance in the observers' responses) are displayed in Figure 6. The corresponding statistical analysis—an ANOVA on participants' distance settings—is displayed in the Appendix as a standard ANOVA table of results including all factors and interactions. 

Figure 6

 

Raw results of Experiment 1. (A) Mean position set by the subjects in Experiment 1 against the true distances, with error bars corresponding to the standard error of the mean. (B) Mean distances set by the most reliable subject. (C) Mean distances set by the least reliable subject (reliability defined as the total variance in the observer's responses).

Subjects can estimate light source distance but make systematic errors

The general pattern of all curves in Figure 6 indicates that the reported light source distance increases systematically with the physical light source distance. This suggests that, broadly speaking, subjects can infer the relative distances of light sources from patterns of shading on objects. Compared with all other experimental factors, the true distance of the light source had the strongest effect on the perceived distance—significant main effect, F(4, 1003) = 1,235.21, p < 0.001—and the average perceived light distance grows with the true light distance with a slope of approximately 1. A linear regression of the perceived light distance against the true one yields a slope of 1.014 (SEM = 0.024), t(1078) = 41.76, p < 0.001. At the same time, subjects systematically overestimate the absolute distance: The intercept of the regression is 4.45, which is significant, t(1078) = 11.06, p < 0.001, and there is substantial variance between and within subjects in their settings, as we discuss in detail below. 

Moreover, subjects seem to be unable to compensate fully for the direction of illumination when judging light source distance. Illumination direction significantly affected the probe settings, significant main effect, F(1, 1003) = 305.19, p < 0.001. The estimates in the 10° condition (M = 21.74, SD = 5.72) were farther than those in the 45° condition (M = 19.60, SD = 5.00). As this effect and its significant interaction with the distance can be explained by a multiplicative effect, it might be an artefact of the subjects' inability to perceive the true distance of the probe to the object veridically. The offset between the probe and the object had a significant component along the line of sight (i.e., in depth), which was indicated by a combination of stereoscopic cues and perspective (from the ground plane). This depth component was smaller for the 45° condition than for the 10° condition. Thus, failures to perceive the full extent of the depth component—as reported by Thompson et al. (2004), for example—would predict different settings for the different illumination directions. If the depth dimension were indeed perceived to be compressed compared with the other dimensions, the subjects would be required to set a larger distance in this direction to reach the same perceived distance to the object. Thus, depth compression predicts the 10° settings to be larger, because they have a larger component in depth, as we observed. 

Object properties influence perceived light source distance

Both roughness—significant main effect, F(2, 1003) = 80.74, p < 0.001—and reflectance—significant main effect, F(1, 1003) = 225.25, p < 0.001—of the object changed the perceived light source distance significantly. Overall, the rough, matte objects led to larger perceived light distances, whereas the order of the intermediately rough and smooth objects was not consistent over reflectance conditions, significant interaction, F(2, 1003) = 12.48, p < 0.001. For glossy objects, the intermediately rough objects led to stronger overestimation than the rough objects (M = 19.52 vs. M = 18.64), whereas the difference was small and reversed in matte objects (M = 21.02 vs. M = 21.34). 

The effect of reflectance was not the same over distances or angles, significant interactions: with distance, F(4, 1003) = 2.84, p = 0.023, and with angle: F(1, 1003) = 7.62, p = 0.006, but both of these interactions are rather small, and in all conditions, the glossy surfaces led to smaller perceived light source distances. Similarly, the size of the roughness effect changed with distance, significant interaction: F(8, 1003) = 2.09, p = 0.034, but the differences are small. 

Together, these findings suggest that the visual system does not use an accurate model of object and scene properties to work out light source distance from shading patterns. The subjects do not correctly compensate for the effects of roughness and reflectance on the appearance of the shading in the image. This suggests that they may use simpler heuristics related to the measurable properties of the shading patterns as they appear in the image, rather than fully factoring out the various physical contributions to those patterns. We explore this possibility below by comparing the pattern of results to simple image statistics that capture the proximal properties of the highlight regions. 

Differences between subjects

We found that the participants differed substantially in their estimates of light source distances. We explored the nature of these differences in more detail with a principal component analysis (PCA). We used the mean settings for each condition as features (columns in the data matrix) over the 18 subjects who participated in Experiment 1 (rows in the data matrix). The results of this analysis are displayed in Figure 7. 

Figure 7

 

Results of a principal component analysis for intersubject differences in Experiment 1. In panels A–D, we display settings that yield a high or low value on the first and second principal component, formatted the same way as the results in Figure 6. These settings were generated by moving along the principal components starting from the average setting across all subjects. The first principal component corresponds to the overall overestimation of light source distance and the second to the difference for the smooth glossy objects. In panel E, we show a scatter plot of the subjects on the first two principal components, along with markers for their mean, for the four points we used for illustration in panels A–D, for perfect performance and for the predictions of our model based on simple image statistics. In panel F, we plot the variance explained by each principal component. The dashed line indicates the average variance explained per component.

The first principal component alone accounts for 57.5% of the variance between subjects. It loads positively and roughly equally strongly on all settings. Differences in this component therefore represent differences in the extent to which subjects tended to overestimate illumination distance, as illustrated for large factor loadings in Figure 7A, B. These differences could partially be caused by nonsensory effects (e.g., by different response biases). 

The second principal component accounts for an additional 9.6% of the intersubject variance. It loads positively on the smooth glossy conditions and slightly negatively on all others. Thus, differences in this component code for the size of the differences of the smooth glossy conditions compared with the others, as illustrated in Figure 7C, D. 

In Figure 7E, we plot the distribution of subjects across these first two principal components. As can be seen, most subjects show substantially higher values of the first principal component (“tendency to overestimate”) than ideal veridical performance (open circle). By contrast, subjects are roughly evenly distributed around the correct value for the second principal component (“tendency to treat smooth glossy differently”). 

As can be seen in Figure 7F, the variance explained declines gradually from the second component to the last, indicating no clear cutoff for the number of components to interpret. There is also a gradual decline in the interpretability of the subsequent components. The third component could be interpreted as a noisy measure of the strength of the roughness effect (not shown), but from the fourth factor onward, the factors do not represent clear systematic effects and likely code random variations between subjects. 

These large differences between subjects suggest that the mapping between image properties and perceived light source distance is not fixed but varies systematically from subject to subject. This might reflect different response biases, differences in the weighting applied to different cues, or different assumptions about the prior probabilities of light source distances or shapes of light sources for example. The main differences between subjects seem to concern the absolute distance of a light source and the distance for smooth glossy objects. The latter might reflect differences in the handling of specular reflections. 

Experiment 2

The mean distance estimates and the settings of the most and least reliable subjects are displayed in Figure 8. The corresponding statistical analysis—an ANOVA on the distances set by participants—is displayed in the Appendix as a standard ANOVA table of results including all factors and interactions. 

Figure 8

 

Raw results of Experiment 2. (A) Mean distance matched by the subjects in Experiment 2 against the true distances, with error bars corresponding to the standard error of the mean. (B) Mean matches set by the most reliable subject. (C) Mean matches set by the least reliable subject.

Subjects can match apparent light source distance

As with the probe adjustments, in the matching task, the true light source distance is the strongest determinant of the matched distances—strongest main effect for distance, F(4, 1416) = 2112.96, p < 0.001. The matched distances increased with the true distance in all conditions. When the test and match object were equally rough, subjects reached almost perfect performance. 

The effects of light source direction were very small in this experiment compared with the probe adjustment task. This partly reflects the symmetrical design: The angle of incidence was always the same for both test and match, so subjects did not have to compensate for differences in lighting direction in any given trial. This finding is also consistent with the idea that errors in the probe adjustment task were caused by an underestimation of the depth separation between the probe and object. Here, when there is no probe and therefore no explicit depth comparisons are required to perform the task, subjects are able to match expected appearance for different angles of incidence. 

Nonetheless, there is still substantial variance between (16%) and within subjects (11%), indicating considerable inconsistency in the subjects' judgments. 

Surface roughness affects perceived light source distance

As in Experiment 1, the roughness had a strong effect on the matched light source distance—significant main effect: F(2, 1416) = 555.91, p < 0.001. This effect was stronger for the glossy objects—significant interaction: F(2, 1416) = 79.75, p < 0.001. This difference was even larger than expected from the settings in Experiment 1. Furthermore—as in Experiment 1—the effect of roughness and the interaction with gloss changed with distance, F(8, 1416) = 18.80, p < 0.001, and F(8, 1416) = 8.22, p < 0.001. 

Smooth glossy objects are special

We found a significant effect of gloss, F(1, 1416) = 76.46, p < 0.001; of its interaction with roughness, F(2, 1416) = 79.75, p < 0.001; and a third-order interaction of gloss, roughness, and distance, F(8, 1416) = 8.22, p < 0.001. These three effects are largely explained by the smooth glossy conditions. In these conditions, subjects on average set the distance in the match stimulus to much smaller values than presented in the test stimulus. In these conditions, the subjects agreed much less than in the other conditions, and the variance between subjects' settings for each condition was substantially larger than in the other conditions, by a factor of 2.75, as tested with a F-test of the variance of the subject means around the condition mean in the smooth glossy condition against all other conditions, F(495, 1200) = 2.75, p < 0.001. Also, the variance within subjects was larger, by a factor of 1.32, as tested with an F-test of the variance of the settings around the subjects mean in the smooth glossy condition against all others, F(1000, 5000) = 1.318, p < 0.001. 

Together, these results indicate that the smooth glossy conditions are qualitatively different from the other conditions in the experiment. This effect also showed up to a lesser extent in Experiment 1, as revealed by the PCA analysis, in which the second principal component of intersubject variations related to the effects of glossiness. 

We suggest that this finding is the result of specific stimulus characteristics. Specifically, the obvious feature of the stimuli in these conditions is a prominent specular reflection of the light source, which may interfere with the image cues the participants rely on to perform the task. For example, we suggest that subjects may approach the task in general by parsing the shading pattern into a bright highlight superimposed on the rest of the shading and using the properties of the highlight region to determine the match. Such an approach would make sense as the properties of the highlight region vary systematically with light source distance (see the Introduction). However, a highlight caused by specular reflection behaves quite differently from a highlight caused by diffuse shading. Thus, if subjects tried to apply the same heuristics to a specular highlight as they do to the diffuse highlight, this would lead to inaccurate responses. Specifically, the size of specular reflections of an infinitesimal point source on a smooth surface does not vary with light source distance, and therefore, aiming to infer light source distance from highlight size would lead to errors. 

Regression analysis and predictions of simple image heuristics

Despite a general ability to distinguish between near and far light sources, the pattern of errors in the subjects' settings suggests that they did not accurately translate the pattern of shading into a light source estimate. Instead, the pattern of results suggests that they relied on some cruder heuristics based on proximal characteristics of the shading pattern in the image. Put intuitively, subjects appear to know something about how changing light source position affects shading patterns, even if they cannot accurately compute light source position from observed shading patterns. To test this idea, we extracted some simple image statistics from the images that aim to capture the two main ways in which shading patterns change when light sources are brought close to a diffuse surface: (a) the overall intensity of the shading and (b) the spatial spread (radius) of the bright highlight region on the object (see the Introduction for physical explanations of the origins of these effects). If subjects use some measure of these properties, we would expect our image statistics to correlate with performance. 

To capture these effects, we used the areas in the image brighter than certain thresholds as predictors for subjects' settings. For all stimuli of Experiment 1, we calculated the proportion of object pixels brighter than a set of threshold values ranging from 10% to 90% maximal brightness (255) in steps of 10%. In Figure 9A, we illustrate how these image statistics correlated with the physical distance of the light source. For low thresholds, the area grows with increasing light source distance, as more of the object receives light from the source. For higher thresholds, the area shrinks with increasing distance of the light source a overall intensity of the highlight shrinks. Our intuition was that by combining a small number of such image measurements, we should be able to capture the overall size, brightness, and spatial falloff of the shading pattern. 

Figure 9

 

Evaluation of the thresholds used for predicting the subjects' settings. (A) Correlation of the true distance with the area above each of the nine thresholds from 10% to 90% maximal brightness. The overlayed numbers indicate the order of these areas to enter the optimal model for subjects' settings. (B) Variance of subjects' settings explained by the optimal model against the number of thresholds/areas used as predictors. The dashed line indicates the upper limit for an image-based model (i.e., the variance explained by a model with a single factor for each of the 60 stimuli).

To investigate how well these statistics explained the data in the different conditions in Experiment 1, we used linear regressions of the responses on the image statistics and used the proportion of variance explained (R²) as a criterion. 

To decide which of these image measurements to include in our regression model, we first tested how well they could each predict the true light source distance. To do this, we incrementally added predictors to the model and monitored how the proportion of explained variance increased. Having identified the best single cue, we then tried all others as a second predictor, selected the best one, and repeated this process until all predictors were in the model. The resulting order and the variance explained by each number of thresholds are displayed in Figure 9. Based on the variance explained, we chose a model with three predictors: the areas brighter than 50%, 40%, and 10% of the maximal value. These three cues served as the basis of the model for both experiments. 

We found that the combination of these three image statistics could predict the settings of our subjects surprisingly well: They capture 52% of the total variance in the first experiment and could also predict the main effects of roughness and gloss to some extent (see Figure 10A). This is slightly—but significantly—more than the true distance can explain, F(2, 1076) = 51.927, p < 0.001, and only 6% less than the total variance consistent over subjects (which is the maximum that any purely image-based model could predict, even in principle). 

Figure 10

 

Predictions derived from the area illuminated for four different thresholds. (A) Model predictions for the different experimental conditions in Experiment 1. (B) Scatter plot, predicted distance versus set distance per subject. The three roughness conditions are color coded as in A. (C) Variance split between different factors. The model explains only 6% less than it can in principle, as red and yellow variance slices represent variance in the settings for the exact same pictures. (D–F) As A–C derived for Experiment 2 from the model fit to Experiment 1. In E, some points in the right lower corner indicate strong underestimations by some subjects in the smooth glossy condition, which are not predicted by the model. In F, it can be observed that 24% more of the variance could be explained by an image-based model.

Indeed, the results displayed in Figure 10C, F show that 42% of the variance in Experiment 1 and 27% of the variance in Experiment 2 are within the individual conditions and thus cannot be explained by the differences in the stimulus in any way (i.e., they represent response variations that occur when the stimulus is constant). To quantify the variance within and between subjects, we computed the variance of individual settings around the mean of each subject and the variance of the subjects' means around the mean for each condition. The deviations of the individual settings from each condition's mean must sum to zero and are thus uncorrelated to any predictions of the ANOVA or the image statistics. Thus, these variance components can be extracted and cannot be predicted by the image statistics or the ANOVA's factors. 

The weights in the final model were as expected from the correlation with the true light source distance. The area from the 10% threshold entered positively and the other two negatively. Using additional threshold values led to a modest increase in performance (a few percentage points of more variance explained), but as these statistics can be understood only as a rough proxy for any true image computation made by the visual system, we omit discussions of how many different thresholds should be used and emphasize instead the observation that even a small number of very simple measurements can explain the settings of the subjects better than the true light source distance. 

To estimate the contribution of the overestimation to the intersubject differences, we included an additional factor scaling the mean setting of each individual subject to the ANOVA's generalized linear model. This factor for the strength of the overestimation explained 10% of the total variance additionally to the mean per individual experimental condition (see Figure 10C for the contributions of other variances). 

To assess the generality of the model, we tested its ability to predict the settings from the matching experiment, using the same regression model and coefficients (i.e., the model that was fit to the data from Experiment 1 was used to model the data in Experiment 2). To predict the matches, we first computed the prediction of our regression model for all stimuli and subtracted the mean prediction for the test and match stimuli separately to account for differences in object shape. Then, we selected the match stimulus with the most similar prediction from our regression model. 

Evaluated this way, our model yielded predictions that explained 49% of the total variance for Experiment 2. Although this is 12% less than the true distance could explain, it is nevertheless a substantial portion of the variance. However, it is important to remain cautious in interpreting the relative successes of veridical versus image statistics–based models. First, the image statistics model and ground truth are themselves highly correlated, so distinguishing which provides a superior model of the observer's data is difficult. Moreover, although ground truth outperforms the image statistics in Experiment 2, there are several aspects of the data that deviate in important ways from simple veridical performance, such as the large intersubject differences, the substantial effects of gloss and roughness, and the large systematic errors, especially the overestimation observed in Experiment 1. Although the image statistics model does not predict all of these aspects of the data, it does at least capture some of these qualitative effects in a way that the ground truth does not. 

In the smooth glossy condition, subjects' matches deviated strongly from the predictions of the model, as can be seen comparing Figure 8 and Figure 10D or in Figure 10E. These conditions were also especially variable between subjects. These findings suggest that subjects use other cues or approaches in these conditions. 

Discussion

Overall, we find that subjects have a reasonably good understanding of how light distance affects shading, as in practically all conditions, subjects' responses increased linearly with the physical light source distance. However, they substantially overestimated the distance in our probe adjustment task, and the extent of this overestimation varied between subjects. 

In addition, the perception of light source distance was altered by the roughness and gloss of the illuminated object. The deviation from veridical perception caused by roughness was replicated in the matching task. This shows that observers cannot fully separate properties of the object from illumination properties. This finding fits well with the research showing effects of illumination on the perception of object properties, especially with previous studies that showed effects on the two object properties we studied (Ho et al., 2006, 2008; Te Pas & Pont, 2009). 

The large intersubject differences suggest that judgments of object properties, such as lightness, roughness, gloss, or shape, are probably not based on the same explicit estimates of light source location that we measured here. Judgments of lightness, color, and shape usually exhibit smaller variability than we found here. It is important to point out that discounting the illuminant for the purpose of estimating object properties does not necessarily involve an explicit estimate of the position or properties of the physical source that provided that illuminant. We suggest that judging the properties of surfaces and judging the properties of light sources themselves are not inextricably coupled but rather complementary computations that draw on different aspects of the shading patterns in the image. 

Despite the overall similarities between the results of the two experiments (probe setting and asymmetric matching), there were also a couple of substantial differences between them. These include, most notably, (a) the tendency to overestimate distances in the probe settings, which was absent from the matching experiment, and (b) the large differences with the smooth glossy surfaces in the matching experiment, which was not observed in the probe adjustments. There are several possible explanations of these discrepancies. 

The first is that the probe adjustment task involved absolute judgments of physical distance, whereas the matching task requires relative judgments. Specifically, in the probe adjustment task, misperception of the absolute position of either the light source (as depicted by the shading) or the probe (due to weak stereoscopic cues to depth) would lead to measurable errors in the responses, as observed. By contrast, in the matching task, only differences in misperception between the test and match images would be visible in the results: Misperceptions that were common to both test and match stimuli would lead to settings along the y = x diagonal. 

Second, the nature of the tasks may have caused the observers to attend to somewhat different aspects of the shading patterns in the two experiments. For the probe task, the process of adjusting the probe did not in any way alter the pattern of shading on each stimulus. By contrast, for the matching task, observers directly controlled the shading pattern on the surface: making it brighter and more focused or dimmer and more diffuse. This may have encouraged subjects to match the properties of the highlight region (as it appeared in the image) as a proxy for matching the perceived light source distance. Taking such an approach would cause the settings for the smooth glossy to be outliers, as the brightness and spatial distribution of the specular highlight was quite different than for the matte or rough surfaces. 

Third, it is also worth noting that the size and stereo depth of the images were different in the two experiments. We believe that this had a negligible effect on the results of the matching experiment as, again, the task did not require absolute judgments of depth, only relative judgments of shading appearance. 

Together, this pattern of errors suggests that the observers were not accurately inverting the physics of image generation to recover the true position of the light source. The large but systematic differences in the extent to which subjects overestimated light source distance in the probe adjustment task suggest the existence of a free parameter in the mapping from shading patterns to light source distance estimates that cannot be uniquely inferred from the image data. The distance of the source from the surface is not the only property of the source that affects the intensity and focus of the shading pattern. Other factors, such as the intensity, size, or diffuseness of the source, interact with the distance to determine the resulting shading pattern. Thus, the visual system must explicitly estimate—or assume—values of these parameters to arrive at an estimate of light source distance. There may be cues that allow the visual system to disambiguate the distance and intensity of light sources. Although the two factors can trade off to create images with the same mean intensity, the specific spatial patterns of shading created by a dim, near light source and a bright, distant light source are different even when the mean image intensity is the same. The visual system may be able to use such cues to separate the relative contributions of the different factors. However, when there is uncertainty about the scene parameters, prior assumptions may contribute to the estimates. Indeed, the tendency to overestimate light source distance may perhaps reflect the operation of a prior for distant light sources. 

We suggest that instead of performing sophisticated inverse optics computations, observers may rely on some simple intuitions about how light source proximity affects the intensity and area of the highlights in diffuse shading patterns. We tested this by creating a very simplistic image-based model that uses the percentage of the image above certain thresholds to capture the intensity and spatial distribution of shading on surfaces. We found that these image statistics could explain certain aspects of subjects' settings surprisingly well, especially in Experiment 1, where the simple image measurements explained more variance than the true physical distance of the light source did. Proponents of inverse optics models should explain how such a model could predict these large and systematic errors. 

We did not find decisive evidence that the visual system relies on simple image statistics—in Experiment 2, ground truth actually captured a larger proportion of the variance than the simple heuristics did. However, we find it intriguing that simple properties of the shading pattern can capture most of the qualitative patterns of error in the observers' data. Moreover, this line of reasoning fits well with other findings suggesting that material properties may be also perceived using simple heuristics. For example, for gloss, Motoyoshi, Nishida, Sharan, and Adelson (2007) suggested a heuristic based on the skewness of the luminance histogram, although this view might not capture all aspects (Anderson & Kim, 2009). 

At the same time, it is clearly very unlikely that the image measurements in the model presented here are actually computed by the human visual system, not least because the front end of visual processing is relatively insensitive to absolute luminance levels, and subsequent processing does not operate on a pixel-like representation of the image. The purpose of the model was not to create a process model of computations performed by the visual system but rather to test the intuitive idea that certain simple measurements can capture the main effects of light source distance on the intensity and spatial structure of shading patterns, without having recourse to more complex inverse optics calculations. We believe that some set of biological image measurements could give the visual system access to an estimate of the size of the highlight region in the shading pattern, and thus the idea of such a heuristic does not rest on the specific image measurements presented in the model. Doubtless, there exist other statistics and models that could explain the pattern of results as well or better, and subsequent research should compare and contrast the abilities of different process models to predict subjects' responses. 

It is also worth noting that such simple image-based models cannot even in principle predict variations between or within observers, which was an important portion of the variance in our experimental data. A slightly more sophisticated model that uses a weighted combination of multiple cues to fit the data might be able to model some of this variance. 

Finally, we also noted that the smooth glossy condition seems to be perceptually different in some important ways from the other conditions. The participants' judgments of light source distance were more variable with the smooth glossy objects (both between and within subjects), and the model also failed to predict the strong underestimation of light source distance that occurred in this condition. For some reason, the large, salient glossy reflection appears to make it difficult for subjects to estimate light distance correctly. It might be useful to investigate such conditions separately in further studies. 

Conclusion

Our findings suggest that human observers certainly have some clear—and broadly accurate—intuitions about the effects of light source distance on patterns of surface shading. In all conditions we tested, the reported light source distance increased systematically (indeed, roughly linearly) with the true light source distance, although their estimates were rather unreliable and differed substantially between subjects. In addition, their settings were influenced by the roughness and gloss of the object in both a probe task and a matching task. The average settings of observers can be predicted surprisingly well by a simplistic model based on simple image measurements related to the size and intensity of the highlight created by the nearby light source. Thus, it is possible that observers may base their judgments on simple heuristic strategies rather than more sophisticated inverse optics computations. 

Acknowledgments

This research was supported by the EU Marie Curie Initial Training Network “PRISM'' (FP7-PEOPLE-2012-ITN, Grant Agreement: 316746) to R. W. F. H. H. S. was supported by DFG Grant WI 2103/4-1. 

Commercial relationships: none. 

Corresponding author: Heiko H. Schütt. 

Email: [e-mail]. 

Address: Department of Psychology, University of Gießen, Gießen, Germany. 

Appendix: Detailed statistics

Experiment 1.

 

Repeated measurements ANOVA table for the influence of the factors distance, angle, roughness, and gloss and their interactions on the perceived light source distance. *p < 0.05; **p < 0.01; ***p < 0.001.

 

Experiment 2

 

Repeated measurements ANOVA table for the influence of the factors distance, angle, roughness, and gloss and their interactions on the matched light source distance. *p < 0.05; ***p < 0.001.