Shape from shading: New perspectives from the Polo Mint stimulus

Abstract

The visual system exploits prior knowledge on the world to disambiguate a scene. In particular, the inference of shape-from-shading relies on the fact that light comes from above our heads. Recent studies have helped make that assumption more precise and suggested that the preferred light source position was further biased on the left of the vertical. We investigated the generality of this result in two experiments with a different stimulus and a novel task. The stimulus was shaped like a ring ( Polo Mint) divided in eight equal sectors. Depending on the assumed light source position, all sectors but one could be perceived as either convex or concave. In the first experiment, observers had to report the different shaped sector position (left or right side of the stimulus). In the second experiment, they had to classify the shape of the odd sector (convex or concave). Various amounts of blur were also applied on each stimulus. Results in both experiments confirm previous studies that observers prefer stimuli lit from the left rather than the right. We also demonstrate that left-lit stimuli give the observers a more sensitive perception of the shape of the objects. Finally, the second experiment confirms a preference for globally convex shapes, especially when the stimulus is severely blurred.

Introduction

Shading refers to the reflected light from a surface that varies according to its shape and is thus a potentially important source of information about the object. Shading differs from shadows that refer to parts of surfaces that do not see the light because of occlusion by an object (Mamassian, Knill, & Kersten, 1998). Cast shadows (those occluded surface parts that are remote from the occluding object) are critical for the perception of spatial layout (e.g., Kersten, Knill, Mamassian, & Bülthoff, 1996; Kersten, Mamassian, & Knill, 1997). Attached shadows (those occluded surface parts that are contiguous to the occluding object) appear to play a role more similar to shading than cast shadows in that they can help define the shape of an object (Cavanagh & Leclerc, 1989). In the stimulus we describe in the present manuscript, attached shadows distinguish convex from concave object parts. 

Understanding the illumination of a scene helps the segmentation of the scene in different objects, and each object in different parts. For instance, it has been argued that lightness perception relies on the segmentation of the image intensity in different layers (Anderson & Winawer, 2005). In a natural scene with multiple objects, the visual system has to solve the shadow correspondence problem to explain the presence of dark patches in the image (Mamassian, 2004) and to discriminate shadows from paint (Freeman & Viola, 1997). The extent to which the visual system is very efficient in solving that problem has been studied by using scenes with shadows incongruous with the casting objects (Jacobson & Werner, 2004; Mamassian, 2004; Ostrovsky, Cavanagh, & Sinha, 2005). Most studies are consistent with the idea that to solve the shadow correspondence problem, the visual system uses only a coarse representation of the scene that preserves the general location of an object but that discards all fine details about its shape. In addition to using a coarse spatial representation, the visual system can accelerate its computation by using prior knowledge on the illumination. 

Statistical regularities of our environment can be used to disambiguate a scene. One of them is that light comes from above rather than below our heads. It is thus possible to infer an object shape from this assumption. For example, if an object is illuminated from above, its shape may appear convex, but if the same object is illuminated from below, its shape will appear concave (Ramachandran, 1988). If the observer assumes that light is coming from above, the object's shape will thus be disambiguated (we will make this assumption when we describe the stimuli in the manuscript). Recently, Adams, Graf, and Ernst (2004) have shown that this light-from-above preference may be modified following active interaction with the environment. Surprisingly, light also appears to be preferentially assumed to be slightly coming from the left rather than straight above. In a visual search paradigm, Sun and Perona (1998) have shown that observers discriminate convex from concave hemispheres with different speeds depending on the light direction (Figures 1A and 1D). Response times were significantly shorter for a light source position located above-left. McManus, Buckman, and Woolley (2004) found a similar leftward bias but only for short stimulus presentations. In a series of studies, Mamassian et al. (Mamassian & Goutcher, 2001; Mamassian & Landy, 2001; Mamassian, Jentzsch, Bacon, & Schweinberger, 2003) opted for a more direct shape discrimination paradigm. For example, Mamassian and Goutcher (2001) used images of parallel bright and dark sinusoidal contours displayed on a gray background (Figures 1B and 1E). These contours formed strips in relief that were judged to be wide or narrow depending on the orientation of the image. The discrimination between wide and narrow strips was best when the stimulus was tilted to the left by about 26°, in agreement with a preferred light source position located above-left. 

Figure 1

 

(A and D) Example of a stimulus used by McManus et al. (2004): the hemi-sphere appears convex (A) or concave (D). (B and E) Example of a stimulus used by Mamassian and Goutcher (2001): the surface presents narrow strips (B) or wide strips (E) in relief. (C and F) Example of the Polo Mint stimulus used here: the object appears as a convex (C) or concave (F) ring. Images D, E, and F are 180° rotations of images A, B, and C, respectively.

Not only the light direction has a dramatic effect on the perceived shape, but some shapes are more easily perceived than others. For instance, concave shapes are detected faster among convex shapes than the other way around (Kleffner & Ramachandran, 1992). Two explanations have been offered to explain this asymmetry. First, this result might reflect a default preference for convex shapes in that concave shapes are detected faster because they stand out from the default (Deutsch & Ramachandran, 1990). Alternatively, it has been proposed that concave shapes are perceived faster because they appear more contrasted than convex ones. In particular, Chacón (2004) found that concave hemispheres were perceived to have up to 10% more contrast than the same image seen upside down. It is therefore not clear whether the advantage for concave shapes is genuinely related to the three-dimensional object or whether it is the result of an image artifact. This question is not easily answered with the hemispheric stimulus shown in Figures 1A and 1D. 

Another difficulty in using the hemispheric stimulus is the somewhat weak three-dimensional percept linked to it. Although the convex shape is usually well perceived ( Figure 1A), the upside-down image ( Figure 1D) does not produce a convincing concave percept unless the image is directly compared with surrounding convex shapes. This lack of saliency might be at the origin of interindividual and interstudy differences. For instance, McManus et al. (2004) failed to find a leftward bias for the assumed light position when the stimulus was presented for a long time to naive observers. The stimulus used by Mamassian and Goutcher (2001; Figures 1B and 1E) is arguably more balanced in that the upright and upside-down images differ only in the intercontour spacing and not in the contrast between its upper and lower parts. However, we did encounter a few naive observers who had some difficulty in perceiving a three-dimensional shape when they first looked at the figure. These observations suggest that it is worth looking for a new shape-from-shading stimulus that is more salient than the ones used in previous studies. We present below experiments based on a novel stimulus inspired from the Polo Mint candy (Figures 1C and 1F). This stimulus allows us to generalize claims for a leftward bias for the assumed light source position and to test the preference for convex shapes. 

In an informal survey on 17 naive adult observers, we presented the six patterns of Figure 1 and asked them which of the top three they thought was the most three-dimensionally salient. Thirteen observers (i.e., 76%) chose the Polo Mint ( Figure 1C), 3 observers chose the undulated stripes stimulus ( Figure 1B), and 1 observer reported not seeing any depth in any of the figures. Although these results should be replicated in a more controlled setup, they confirm our premise that the shaded hemisphere does not produce a convincing impression of depth (none of the 17 observer chose that pattern) and that the Polo Mint stimulus is worth investigating further. 

The novel stimulus will also allow us to address a new issue. Previous studies have shown that the assumed light source position affects the perceived shape of a shaded object, but it is not clear whether an object lit with this preferred light source will be better perceived. In other words, are observers more sensitive to the attributes of an object if it is lit with their preferred light? This issue of sensitivity can be addressed with the help of a task that is orthogonal to the manipulation of the light source position. 

The issues of preferred light source position and preferred shape have been addressed with two experiments. Both experiments used the same stimuli but different tasks. The stimuli were Polo Mint figures where one of the eight sectors was reversed in depth (creating a concave sector within a convex ring, or the reverse). The first task was to report the side (left or right) of this odd sector. The second task was to report the shape (concave or convex) of this odd sector. To anticipate our results, we found a consistent leftward bias for the assumed light source position but an asymmetry between convex and concave shapes only when the shape is explicitly part of the task. 

Methods

Apparatus

All experiments were conducted on a 21-in. Sony Trinitron monitor connected to an Apple Macintosh G5 computer. The monitor was calibrated in luminance (brightness setting at 50% and contrast setting at 100%). It was set to a resolution of 1024 × 768 pixels and ran at a refresh rate of 100 Hz. The experimental stimuli were created with Matlab v.704 (Mathworks, MA, USA) and displayed with the PsychToolbox for OSX (V1.05, Brainard, 1997; Pelli, 1997). 

Stimuli

Stimuli were displayed in grayscale. The stimuli consisted of a ring with bright and dark polar contours (luminance of 40 and 1 cd m ⁻², respectively) displayed on a uniform gray background (20 cd m ⁻²). The diameter of the outmost circle extended 7° (i.e., 192 pixels). The ring was divided in eight equal sectors (of 45° each). All but one of the sectors were simulated to have the same shape, either convex or concave. The light source was simulated at a position either top-left or top-right (±45° away from the vertical). The contrast of the contours (bright or dark) was determined according to the desired shape to be displayed and the simulated light source position ( Figure 2; see, e.g., Casati, 2004). 

Figure 2

 

Schematic diagram of light projected on the three-dimensional object. The projection of parallel light rays (±45°) defines bright and dark contours in the image. For the edges parallel to the light direction, the contours that should have been gray were painted in black or white in a way consistent with a light source slightly rotated in a clockwise direction.

The combination of the two shapes (convex or concave) and the two light positions (left or right) produces four types of stimuli illustrated in Figure 3. The odd sector could be placed at any of the eight locations on the ring ( Figure 3A). Note that the four stimuli shown in Figure 3 are in fact the same image rotated in steps of 90°. 

Figure 3a, 3b, 3c, 3d, 3e

 

Examples of Polo Mint stimuli used in the two experiments. The ring is divided in eight equal sectors that all have the same shape but one. Four types of stimuli are obtained by crossing a shape and a light position factor. The ring was either convex (a and b) or concave (c and d); the simulated light source was positioned either at the top-left (a and c) or top-right (b and d). In fact, all four stimuli are obtained from the same image by rotation of a multiple of 90°, as one can easily check by rotating the page. Symbols on top of each figure are used in the results figures of the manuscript.

Low-pass filters with different cut-off frequencies were also applied to the stimuli in an attempt to find a level of blur that produced a maximum of perceived depth. The filters were Gaussians with standard deviations chosen along a logarithmic scale: 2.00, 2.82, 4.00, 5.65, 8.00, 11.31, or 16.00 pixels. The image contrast was re-adjusted to cover the full range after the filter was applied. Four of the seven levels of blur are shown in Figure 4. 

Figure 4a, 4b, 4c, 4d

 

Examples of various levels of blur used on the images. The standard deviation of the Gaussian filter applied on the original image was (a) 2 pixels, (b) 4 pixels, (c) 8 pixels, and (d) 16 pixels. At the largest blur level, the odd element seems almost fused with the background and the ring is perceived with a dent.

Procedure

Fourteen normal observers were tested (aged 19–28 years old). The experiments took place in a completely dark room. Participants sat in front of the monitor at a distance of 57 cm. Head position and orientation were controlled with a chin cup and head restraint. Observers viewed the stimuli monocularly. Stimuli were shown for 100 ms and then immediately followed by a mask. 

Two experiments were run with the same set of stimuli but with different tasks. In the first experiment, observers had to report the side (left or right) of the odd-shaped sector (left–right experiment; LRE). In the second experiment, observers had to determine whether the odd sector was convex or concave (convex–concave experiment; CCE). 

Participants used two keys on the computer keyboard to respond. In total, a session was composed of 224 stimuli presented in random order (2 shapes × 2 light positions × 8 sector positions × 7 levels of blur). Each session was repeated three times for each experiment and each observer. A training set was presented before each session and no feedback was provided to the observers. 

Performance in both experiments was measured by the percentage of correct responses, where a correct response occurs when the odd sector is localized (LRE) or when the sector's shape is identified (CCE). The correct shape was taken to be the one consistent with a light source located above rather than below. A two-way ANOVA test (Factor 1: Light source position; Factor 2: Shape) was performed to test the statistical significance of the data. In addition, in the LRE, we also report the discriminability index d-prime ( d′), where chance performance is represented by d′ = 0. 

Results

We split the results across the different tasks and different factors of the experiments. 

Effect of task

Performance is overall better when observers have to report the side of the odd sector (LRE) rather than its shape (CCE): percentage of correct responses is 76.6% (±6.9%) rather than 62.9% (±10.3%). This finding is not too surprising because to correctly discriminate the shape of the odd sector, observers have first to detect its location. 

When one looks at the performance across the four stimulus types, the two experiments give different results: performance is largely above chance in the LRE for all types of stimuli but is only comparably good in the CCE for left-lit stimuli containing a concave odd sector ( Figure 5). We analyze next these findings further across the different factors of the experiment. 

Figure 5a, 5b

 

Percentage of correct responses in the left–right experiment (LRE; top plot) and in the convex–concave experiment (CCE; bottom plot) as a function of light source position. Open symbols represent stimuli lit from above-left and filled symbols stimuli lit from the right. Square symbols depict the conditions where the odd sector was concave and circles the conditions where it was convex. Examples of stimuli are shown in Figure 3. Error bars represent standard errors across observers ( N = 14).

Effect of light source position

We can compare the percentage of correct responses when the stimuli are lit from above-left (thus producing a so-called left score) with the percentage correct when stimuli are lit from the above-right (right score). Distributions across observers of right scores against left scores are shown in Figure 6. Most data points fall below the first diagonal, indicating that most observers were better in both tasks when stimuli were lit from the left. For the LRE ( Figure 6, top), the mean left and right scores are 80.2 and 73, respectively, and their difference is significant, F(1, 27) = 28.33, p < .0001. For the CCE ( Figure 6, bottom), the mean left and right scores are 70.6 and 55.1, respectively, and their difference is again significant, F(1, 27) = 6.80, p < .02. 

Figure 6a, 6b

 

Performance for stimuli lit from the right versus those lit from the left for the left–right experiment (LRE; top plot) and the convex–concave experiment (CCE; bottom plot). Left scores are percent correct for left-lit stimuli, assuming above illumination (and similarly for right scores). Each symbol represents one observer ( N = 14).

In the LRE, we can also perform a sensitivity analysis. By comparing the proportion of trials where the odd element was presented on the left and reported on the left (hits) and those trials where the odd element was presented on the right and the observer reported seeing it on the left (false alarm), we can compute the discriminability index d′. We found an increased sensitivity when stimuli are lit from above-left rather than above-right: mean d′ values are 1.28 rather than 0.70, F(1, 27) = 33.4, p < .0001. 

Effect of sector shape

To investigate the effect of sector shape, we compare the percentage of correct responses when the odd sector is concave within a convex ring (thus producing a so-called convex score) and the percentage correct when the odd sector is convex within a concave stimulus (concave score). In computing theses scores, we assume as always that the light is coming from above. Distributions across observers of concave scores against convex scores are shown in Figure 7. Most data points in the CCE fall below the first diagonal, indicating that most observers were better in this task when they had to detect a concave sector. For the CCE ( Figure 7, bottom), the mean convex and concave scores are 71.1 and 63.0, respectively, and this difference is significant, F(1, 27) = 13,6, p < .001. In contrast for the LRE ( Figure 7, top), the mean convex and concave scores are 77.7 and 75.4, respectively, and this difference is not significant, F(1, 27) = 1.48, p = .23. In summary, the shape of the odd sector had a significant influence on the performance only when the observers had to report the shape. For supplementary results on sector position, see also Supplementary Materials. 

Figure 7a, 7b

 

Comparison of performance for stimuli with different shapes for the left–right experiment (LRE; top plot) and the convex–concave experiment (CCE; bottom plot). Convex scores are percent correct for detecting a concave odd sector in a globally convex stimulus, assuming above illumination (and similarly for concave scores). Each symbol represents one observer ( N = 14).

Effect of blur

Seven blur levels were applied to the stimuli (samples are shown in Figure 4). Figure 8 displays the percentage of correct responses as a function of blur level, separately for the four stimulus types. In the LRE ( Figure 8, top), the four stimulus types produce different performances only for small amounts of blur. When the image is crisp, there is a trend for a better performance for the stimuli lit from the left (open symbols) as compared to those lit from the right (filled symbols). There is also a small trend for a better performance when the odd sector is concave within a convex ring (square symbols) as compared to the opposite (circular symbols). This difference between the four stimulus types completely disappears when the amount of blur is large, although the mean performance level stays high at about around 75% correct. 

Figure 8a, 8b

 

Percentage of correct responses in the left–right experiment (LRE; top plot) and in the convex–concave experiment (CCE; bottom plot) as a function of blur levels. Blur values are the standard deviation of the Gaussian filters applied to the image. See legend in Figure 3 for a description of symbols. Error bars represent standard errors across observers ( N = 14).

The effects of blur in the CCE are even more dramatic. Figure 8, bottom, depicts the percentage of correct responses assuming that light comes from above. When the image is crisp, there is a very large advantage for stimuli lit from the left (open symbols) rather than the right (filled symbols). More precisely, observers were about 80% of the time correct in discriminating the shape of the odd element when light came from the left and fell to chance level when light came from the right. At the other extreme when the image was very blurry, there was no difference between left and right illuminations. In fact, observers responded seeing a concave sector within a convex ring about 70% of the time, even when the stimulus depicted a convex odd sector. The bias to perceive a convex ring (and a concave odd sector) found in Figure 5 is thus probably the result of added confusion in blurred stimuli. For intermediate levels of blur, we find a gradual decrease in the advantage of left-lit stimuli and a gradual increase in the bias for ring convexity. 

Discussion

A large number of studies in the perception of shape-from-shading use spheres or shaded bubbles. However, these images do not provide very convincing perceptions of three-dimensional objects (see Figure 1). We looked for a novel stimulus that was more salient than the existing ones, and we found that presenting multiple bars at various orientations increased the subjective impression of depth. We present here a stimulus that follows this principle and that we named Polo Mint. The use of that stimulus confirmed previous results, disambiguated others, and provided new ones. 

We first confirmed previous reports that human observers interpret the shape of shaded objects as if light was coming from above their head, with a bias to the left of the vertical (Mamassian & Goutcher, 2001; Mamassian & Landy, 2001; Sun & Perona, 1998). In two experiments, one where observers had to locate the position of an odd part of the Polo Mint (LRE), the other where they had to identify the shape of that odd part (CCE), we found better performance when the stimuli were lit from the left than from the right. 

Previous reports of the above-left preference for the assumed light source position were recently put into question because of a potential artifact with the use of shaded bubbles (Chacón, 2004). Our Polo Mint stimulus does not suffer from this potential artifact, so we can be reasonably confident that the above-left bias is not a by-product of using only specific stimuli. 

Finally, the use of the Polo Mint stimulus provided new results. The main new result is a clear demonstration that lighting an object from the left improves the sensitivity to extract the shape of that object. Our LRE enabled us to apply the techniques of signal detection theory to demonstrate an improvement of 0.6 d′ units when light was simulated on the left rather than the right. This result indicates that the assumed light source position not only modulates the perceived shape of an object but also the accuracy with which this shape is perceived. 

Another result was the bias to perceive our stimuli as a convex ring with a concave hole in it. This bias was particularly obvious in the CCE, and even more so when the edge of the stimuli were blurred. This bias has previously been reported in other contexts (Langer & Bülthoff, 2001; Liu, Jacobs, & Basri, 1999; Mamassian & Landy, 1998) and probably reflects the fact that most objects found in the environment are convex. 

A surprising result was the general effect of blurring the Polo Mint stimulus. We expected that there existed a particular level of blur for which observers would be better at localizing and identifying an odd part of the stimulus. Instead, we found that increasing the level of blur removed the leftward bias for the assumed light source direction and exacerbated the convexity bias for the perceived shape. 

The final result of our study was the effect of the task given to the participants. When observers only have to localize the odd part of the stimulus, they show a better performance when stimuli are lit from the left but no significant difference between convex and concave shapes. When they are asked to explicitly report the shape of the odd part, the convexity bias starts to play a major role. The difference between the results of the two tasks suggests that the stimulus is processed at different depths depending on whether the task requires only a superficial analysis of the stimulus or a more detailed scrutiny. Moreover, different levels of processing appear to be linked with different types of biases. 

The origin of the leftward bias for the assumed light source position remains unknown. It is still not clear whether the bias is environmental or biological. Nevertheless, we hope that future studies that will address this issue will benefit from the qualities of the Polo Mint stimulus. 

Supplementary Materials

The stimuli depicted a ring composed of eight sectors of similar shape (convex or concave) except for one. Figure 1A shows the performance in both experiments as a function of the sector position (see Figure 3a for a description of each position). In the LRE, the task to decide whether the odd sector was in the left or right half of the stimulus was probably harder if that sector was adjacent to the midline. Figure 1A top suggests that observers do indeed perform a bit better when the odd sector is far from the midline (positions 2, 3, 6 and 7). The position of the odd sector was irrelevant for the CCE. Performance is not changing in a consistent pattern with sector position (Figure 1A bottom). 

Supplementary Figure - Supplementary Figure 

Supplementary Figure - Supplementary Figure 

Figure A1. Percentage of correct responses in the Left-Right Experiment (LRE, top plot) and in the Convex-Concave Experiment (CCE, bottom plot) as a function of sector position. See Legend in Figure 3 for a description of symbols and Figure 3a for a description of each sector position. Error bars represent standard errors across observers (N = 14). 

Acknowledgments

This research was supported by a Chaire d'Excellence from the French Ministry of Research awarded to Pascal Mamassian. The results were first reported at the Vision Science Society meeting in May 2006. 

Commercial relationships: none. 

Corresponding author: Peggy Gerardin. 

Email: [e-mail]. 

Address: Laboratoire Psychologie de la Perception, CNRS FRE 2929 & Université Paris Descartes, UFR Biomédicale des Saints Pères, 45 rue des Saints Pères 75270 Paris Cedex 06, France.