Search for gross illumination discrepancies in images of natural objects

Abstract

Shadows may be “discounted” in human visual perception because they do not provide stable, lighting-invariant, information about the properties of objects in the environment. Using visual search, R. A. Rensink and P. Cavanagh (2004) found that search for an upright discrepant shadow was less efficient than for an inverted one. Here we replicate and extend this work using photographs of real objects (pebbles) and their shadows. The orientation of the target shadows was varied between 30 and 180°. Stimuli were presented upright (light from above, the usual situation in the world) or inverted ( light from below, unnatural lighting). RTs for upright images were slower for shadows angled at 30°, exactly as found by Rensink and Cavanagh. However, for all other shadow angles tested, the RTs were faster for upright images. This suggests, for small discrepancies in shadow orientation, a switch of processing from a relatively coarse-scaled shadow system to other general-purpose visual routines. Manipulations of the visual heterogeneity of the pebbles that cast the shadows differentially influenced performance. For inverted images, heterogeneity had the expected influence: reducing search efficiency and increasing overall search time. This effect was greatly reduced when images were presented upright, presumably when the distractors were processed as shadows. We suggest that shadows may be processed in a functionally separate, spatially coarse, mechanism. The pattern of results suggests that human vision does not use a shadow-suppressing system in search tasks.

Introduction

A general axiom in vision science is that our visual system attempts to recover the properties of objects in the visual environment, while ignoring the properties of the illumination of the scene, since the objects are likely to be important while the illumination is subject to large and capricious changes in quantity, spectral quality, and direction. Thus, research on lightness and color constancy shows evidence of processes, which “discount” the properties of the illuminant (Gilchrist, 2006). In a sense, shadows are a property of the illumination and are therefore possible candidates for such discounting. Two distinct types of shadows occur. Attached shadows are changes in the brightness of part of the surface of an object, which reflects that surface's orientation in relation to the prevailing illuminant. Cast shadows are changes in the illumination of a surface that results from occlusion of the illuminant by an intervening object. In natural images, both types of shadow cooccur and their presence is highly correlated. However, it is well known that both attached shadows (Adams, 2007; Chacón, 2004; Champion & Adams, 2007; Enns & Rensink, 1990; Gregory, 1966; Kleffner & Ramachandran, 1992; Ramachandran, 1988; Sun & Perona, 1996, 1997) and cast shadows (Kersten, Mamassian, & Knill, 1991) provide strong cues to the geometrical structure of the objects in the image—which suggests that shadows may not be universally ignored by vision because they can be a rich source of information about object properties. However, the idea that something about the encoding of shadows is subject to suppression processes remains attractive and has been the subject of experiments using a visual search paradigm. Gross manipulations of the directions of cast shadows are often not seen by observers (Jacobson & Werner, 2004). On the other hand, visual exploration of the possible interpretation of shadow figure/ground effects that results in multistable, ambiguous figures suggests that shadows provide a strong input to form perception and are certainly not universally discounted (Leonards & Troscianko, 2004). 

Apparent evidence for the discounting of shadows comes from visual search experiments by Rensink and Cavanagh (2004). They examined search efficiency when participants searched for a rotated shadow and found that search times were inefficient when images were upright but were efficient, and faster, when images were inverted. This suggests that shadows are discounted, but only when the shadow is consistent with the normal lighting direction. Because their stimuli were artificial 2D gray rectangles, they only had cast shadows (dark quadrilaterals) and the target shadow was rotated by only 30° (see Figure 1). When stimuli were presented in the ‘upright’ condition the percept was of a plane viewed from above with a number of upright pillars casting shadows. When viewed in the ‘inverted’ condition the percept was of rectangular objects hanging from a ceiling plane. The inversion manipulation relies upon the ubiquitous assumption that light comes from above (Berbaum, Bever, & Chung, 1983; Gibson, 1950; Ramachandran, 1988) or at least within 30° of above (Mamassian & Goutcher, 2001) and consequently that shadows should be below objects. When observers view the search images in an upright orientation they were searching for an odd shadow among shadows, whereas when the images were inverted, the targets and distracters were not treated as shadows. The relatively slow search times and low efficiency for images presented ‘upright’ were interpreted as evidence that shadows were at least partially discounted, i.e., they were less directly accessible to perception than non-shadows—with similar visual characteristics, i.e., the inverted shadows. However, when the images were inverted, both the shadows and the implied ground plane were rotated, so the possibility remains that the effect of inversion was caused not by the processing of a shadow mechanism and a non-shadow mechanism, but instead by changing the ‘normal’ viewed-from-above scene into an unusual scene. Torralba, Oliva, Castelhano, and Henderson (2006) suggest that, when scenes are upright, the ‘gist’ is grasped but access to individual scene objects and their shadows is weakened, thereby delaying search processes. 

Figure 1

 

An example of the stimuli redrawn from the original Rensink and Cavanagh ( 2004) study. The target is located in the lower right-hand corner of the image.

In the present study, we examine the question of shadow detection in more detail, using a greater variety of shadow manipulations and using photographs of natural objects with or without cast shadows rather than the schematic objects used previously. We employ stimuli that share characteristics of the original concave and convex “bumps” (Gregory, 1966; Ramachandran, 1988), i.e., using a fronto-parallel presentation. However the current stimuli feature natural objects with natural shadows that have both a cast and an attached component. By using natural objects (Cunningham, Beck, & Mingolla, 1996) and their shadows in a visual search paradigm, we can manipulate the heterogeneity of the shadow casters. Where heterogeneity is increased, search becomes less efficient and search slopes increase (Duncan & Humphreys, 1989). If the mechanism underlying shadow processing is coarsely scaled and presumes overhead lighting, then we should expect a smaller influence of heterogeneity upon search slope for upright images than for inverted images, despite the fact that the contents of the scenes are identical—except for the inversion. It has been argued that shadow processing operates at a coarse visual scale (Mamassian, 2004). Put simply, the purpose of our experiments was to be able to dissociate explanations of shadow perception based on “suppression” and “coarse processing,” while using natural stimuli and controlling both cast and attached shadows. 

Experiment 1

Methods

Stimuli

In order to create stimulus materials that appear natural to observers while allowing the experimenter to easily manipulate characteristics such as appearance, shadow orientation, shadow direction, and shadow presence we decided to use individual photographs of objects with and without shadows. This enabled us to assess the visual similarity of individual objects and subsequently to assemble similar or heterogeneous objects into “chimeric natural images”. Pebbles were chosen as a suitable object for stimulus generation as they vary a great deal in detailed appearance from one another, have an overall similarity, and because of their shape, can naturally produce cast and attached shadows. Pebbles are also useful as stimuli because they do not have a cardinal orientation and therefore appear equally plausible in upright and inverted images. Sixty pebbles (Cotswold limestone) were randomly selected from the garden of one of the authors. 

In order to generate naturalistic images without noticeable joins in the background between pebbles, each pebble was photographed under carefully controlled conditions. In order to capture linearized images we used a camera (Nikon D5700) that had previously been characterized and calibrated (Lovell et al., 2005). The pebbles were photographed individually against a gray cloth background with a Macbeth color checker chart (www.gretagmacbeth.com) placed within the frame. Pebbles were photographed from above while being rotated through four orientations (90° steps), under ring-flashed (Figure 2, left) and unidirectional (Figure 2, right) illumination—ring flashes produce images without shadows by illuminating the scene from a ring of light located around the lens of the camera. Under the unidirectional illumination the light source is termed “above” but in fact it was slightly to the above right (by approximately 15°), when photographed with shadows directly below the object the stimuli tended to look quite artificial. The long axis of elongated pebbles was roughly aligned with the photograph's vertical axis in the 0° photographs. The same flash system (Elinchrom Free-Lite Ring flash with Elinchrom Free Style power unit; www.elinchrom.com) was used for both illuminants by rotating the ring flash away from the camera lens and toward a mirror. The exposure for all photographs was fixed at 1/4000 sec. Removal of the image background was facilitated by also photographing the gray cloth under both illumination conditions without a pebble present. As the path of light from the flash to the pebble was much shorter in the ring-flashed mode compared to the shadowed mode—the ring-flashed images would be much more brightly illuminated. To compensate for this the voltage supplied by the flash was varied (4.1 V for ring-flash illumination, 6.8 V for unidirectional illumination) and a neutral density filter was placed in front of the camera's lens (0.5 log units) during the ring-flashed photography. Consequently, photographs in the non-shadowed and shadowed conditions were neither over- nor underexposed. 

Figure 2

 

(Leftmost) A pebble photographed under ring-flashed illumination; the other images show the same pebble photographed with unidirectional illumination from above, left side, below, and right side, this is achieved by rotating the pebble through 360° (in 90° steps) while photographing and then rotating the photograph so that the pebbles are coaligned.

Each image was linearized and then rescaled so that the mean RGB values of a patch of the gray background proximate to the pebble matched the average of the same patch in all of the images. Subsequently the images were cropped and subsampled by a factor of 0.225 so that each pebble and its shadow were located within a 115-pixel-square image. Consequently, with a subsequent viewing distance of 2 m, the average size of the pebbles was approximately 0.5° in visual-angle units. 

In order to manipulate the heterogeneity among the pebbles present within a stimulus image, the similarity of each pebble was estimated using a visual difference predicting (VDP) model (Lovell, Párraga, Ripamonti, Troscianko, & Tolhurst, 2006; Párraga, Troscianko, & Tolhurst, 2005). For each pebble, one of the four available orientations was randomly chosen. Then, using the non-shadow images of the chosen orientations, each of the 60 images was compared with all the others using the VDP model, giving a 60 × 60 confusion matrix. The confusion matrix was then subjected to a classical multi-dimensional scaling analysis (Mathworks, 2007). The first two dimensions of the MDS space were sufficient to account for much of the variance in the original confusion matrix (r-squared = 0.59). Figure 3 shows each of the selected pebble images superimposed onto the two most significant MDS axes. The first axis corresponds essentially to orientation and the second to size (although color and lightness also seem to vary across this axis, though for clarity this axis will be referred to as the ‘size’ axis). These axes form the basis of selections of similar and heterogeneous pebbles used in Experiments 1 and 2. 

Figure 3

 

Each pebble photo superimposed upon the first two axes of the MDS space. The horizontal axis represents orientation, and the vertical axis represents size and perhaps color. The dashed areas (A, B, C, and D) and the individual pebbles (X and Y) delimit those pebbles selected in order to achieve varying levels of heterogeneity, the details of this procedure are discussed below.

To achieve varying degrees of heterogeneity in the pebbles included within a stimulus, pebbles were selected from a number of regions within the MDS space (see Figure 3). These regions delimited those pebbles that fell within the 35th and 65th percentiles of the orientation and size axes. For example, in order to achieve the most homogenous stimulus, drawn from the ‘size’ axis, only pebble ‘X’ would be included in an image. In order to achieve an intermediate level of heterogeneity pebbles were selected from region ‘B’. Finally, to achieve the most heterogeneous stimulus, distractor pebbles were chosen from regions ‘B’ and ‘D’. Selection was made from each bin in turn, in random order and without replacement. For the ‘orientation’ axis, pebbles were selected using the same procedure, from regions A, C, or solely consisted of pebble ‘Y’. Pilot studies revealed that reaction times for the detection of discrepant shadows were significantly longer when the shadow was cast by two particular pebbles. These pebbles were not included in any further stimuli; two additional, randomly selected pebbles (from the other groups) were excluded in order to balance the size of the different heterogeneity groups (A, B, C, D). 

The locations of the pebbles in the search display were determined as follows

Stimulus images were constructed to have 512 × 512 pixels (6.2° angular subtense). Five, ten, or fifteen locations were randomly selected within the 512 × 512 pixel image with the constraint that none of the locations was allowed to be closer than 88 pixels (1°), thereby ensuring that pebbles and shadows did not overlap. The left–right distribution of these points was shifted so that the centroid was in the center of the stimulus. One of these locations was randomly selected as the location for the target pebble, with the constraints that the target pebble could not be located within 0.6° of the vertical midline and that, within a stimulus condition, 50% of the targets were located to the right of the midline and 50% to the left. During the photography stage, each pebble was photographed with four different orientations. Now each shadowed-pebble image was rotated so that the pebbles were in alignment. Thus four images of each pebble were available at a given orientation, each with shadows falling in four different directions. In other words, we now have four shadowed images of each pebble, with the pebble in the same orientation, but with four different shadow directions available. For the current experiment the image featuring the pebble with the same orientation as that randomly selected during the MDS analysis was used as the base orientation, in order to avoid using pebbles that were predominantly horizontally or vertically aligned. For the inverted (i.e., light from below) condition the images were flipped top to bottom. That is: the target pebble always had its cast and attached shadow pointing in the opposite direction (180°) compared to the distractors. Altogether there were 36 stimulus conditions: 2 different target pebbles, at 3 levels of distractor homogeneity, with 4, 9, or 14 distractor pebbles, and finally each condition was presented upright and inverted ( Figure 4). 

Figure 4

 

Sample stimuli with varying levels of distractor heterogeneity. In all cases, the target's shadow is pointing 180° in the opposite direction from the distractor pebbles. These stimuli all feature from above illumination.

Procedure

The images were presented on a linearized CRT with overall size of 36.5 by 27.4 cm viewed from a distance of 200 cm. The individual images were presented one at a time in the center of the screen; the images measured 6.2° (angular subtense), while the remaining parts of the screen were held a uniform gray (39 cd m ⁻²) matching the background visible between the pebbles. Vertical white lines (length = 0.5° visual angle) extended from the top and bottom of stimulus image at the center; these were present throughout the experiment. Observers were asked to fixate between these lines at the beginning of each trial, but they were free to move their eyes once each trial had begun. They were asked to search quickly but accurately through the stimulus to find the target (the single pebble with discrepant shadow direction) and to push either a left or a right button to indicate on which side of the display, relative to the vertical white lines, the target was located. The stimulus presentation order was randomized, except for the inversion of images; upright and inverted images were presented in separate blocks of 27 trials. Within each block the number of occurrences of each condition (target pebble and heterogeneity, but excluding inversion) was randomized; this randomization was different for each observer. The order of blocks (upright/inverted) was counter-balanced across observers. At the beginning of each block, observers were shown a cue image informing them of the direction of the prevailing illumination in the following block (i.e., the direction of the cast shadows of the distractor pebbles). At the end of each block, observers were prompted to press a key to continue. They were also advised to take a rest break at the end of each block if they wished. Reaction times since the onset of the stimulus, and the observer's decision (target left or right) were recorded. During practice, observers were advised to respond quickly but to avoid giving incorrect responses. Auditory feedback was given when mistakes were made during practice but not during the experiment. Over the course of the experiment each observer viewed 12 images for each experimental condition and number of pebbles; giving a total of 432 trials. An experimental session (excluding practice) would generally last 30 minutes. 

Observers

Nine observers were tested; all were undergraduates in the Department of Experimental Psychology, University of Bristol, one was male and eight were female. All were naive to the purpose of the experiment. The experiments were carried out according to the ethics guidelines of the Department of Experimental Psychology, University of Bristol. 

Results

For each participant, the median reaction time for each experimental condition was taken, and the means (across observers) of those medians are plotted in Figure 5. For each observer, the degree of heterogeneity and each search slope (reaction times against number of distractors) and centercepts were analyzed. The centercept is a measure of overall search time that, unlike the intercept, is not confounded by search slope (Wainer, 2000). The mean error rates for each experimental condition were less than 2%. In the few instances where observers made more than a few errors, their error-free reaction times to stimuli of the same heterogeneity or numerosity tended to be large. Thus, errors were associated with long reaction times, and there was no speed-accuracy trade-off. We have discarded the erroneous trials from analysis, recording the medians of the remaining error-free trials, a complete table of error rates for all three experiments is supplied (see Supplementary materials). 

Figure 5

 

Averages across observers of the median reaction times. The leftmost plots represent the search RTs for the most homogenous stimuli, while the rightmost are for the most heterogeneous. The error bars are the standard errors for the averages from 9 observers.

Clearly, inverting the images (changing the presumed direction of illumination) has changed the relations between RT and number of distractor pebbles ( Figure 5). Observer search slopes and centercepts were analyzed using an Analysis of Variance (ANOVA); in each case (for Experiments 1– 3) the minimal adequate model was achieved by iteratively deleting non-significant terms. For search slopes, there were significant effects of heterogeneity ( F(2, 54) = 41.1, p < 0.0001) and inversion ( F(1, 54) = 34.7, p < 0.001); there was also a significant interaction between heterogeneity and inversion ( F(2, 54) = 17.8, p < 0.0001). For centercepts, there were significant effects of heterogeneity ( F(2, 54) = 54.2, p < 0.0001) and image inversion ( F(1, 54) = 30.1, p < 0.001), and once again a significant interaction between heterogeneity and inversion ( F(2, 54) = 17.8, p < 0.001). 

Figure 6 shows the averaged search slopes (left) and centercepts (right) as a function of distractor heterogeneity for all observers, averaged across the size and the orientation heterogeneity conditions. As in a classical search experiment with simple stimuli (Duncan & Humphreys, 1989), search slope and centercept increase with heterogeneity. The slopes and centercepts are clearly higher for the inverted (light from below—red symbols) condition. 

Figure 6

 

Averaged search slopes (left) and centercepts (right) plotted as a function of inversion and heterogeneity. Error bars represent ±1 SE.

Discussion

If the shadows in upright images were correctly perceived as shadows, and if those in the inverted images were actually treated as “non-shadows,” then the results of the current experiment suggest that visual search for discrepant shadows is not impaired relative to non-shadows; indeed visual search for shadows was faster and more efficient. Increasing heterogeneity reduces search efficiency when images are inverted—i.e., objects illuminated from below—and has little influence when they are upright. Although this seems to contradict Rensink and Cavanagh's ( 2004) specific proposal that shadow information is suppressed, this result is nevertheless consistent with their suggestion that a different mechanism underlies processing of the upright and inverted images. When upright, processing seems to be spatially coarse and thus insensitive to small changes in the shapes of the shadow casters and their shadows. The following experiment examines this hypothesis. 

Experiment 2

Experiment 1 demonstrated a significant effect of image inversion: search was faster and more efficient, for images with an upright presentation. This result is exactly the opposite of that reported by Rensink and Cavanagh (2004). However, the current Experiment 1 was only a partial replication of theirs. A number of key differences exist: 1) the target shadow in the Rensink and Cavanagh study had been rotated by only 30° whereas our target shadow was rotated by 180°, 2) the presentation of stimuli was fronto-parallel in the current study but on a slanted ground plane in Rensink and Cavanagh's, 3) even the least heterogeneous stimuli in our Experiment 1 were far more heterogeneous than those presented in the original study. Thus, the current experiment examines whether the degree of shadow rotation is important. It was hypothesized that a spatially coarse shadow processing mechanism—as implied by the results of Experiment 1—might perform poorly when searching for shadows with small angular deviations. The current study now manipulates the angular deviation from 30° to 180° in increments of 30°. In this new experiment, the low-heterogeneity 30° shadow deviation condition could be considered a close replication of the original Rensink and Cavanagh experiment with natural stimuli, while the 180° shadow deviation conditions are, of course, equivalent to the stimuli conditions used in our Experiment 1. 

Methods

Stimuli

The photographs of pebbles were as described above. In the current experiment, there were two levels of heterogeneity, low and high. Unlike the first experiment, distractor and target pebble images were randomly selected from the four available orientations for each pebble; consequently, the rotated orientation of the target pebble could not be used as a cue to the location of the target shadow, as it was unlikely that observers would memorize the four cardinal orientations of each pebble and then notice when one of these was rotated to create a discrepant shadow. For low heterogeneity stimuli, a single pebble was used (see X, Figure 7). For high heterogeneity stimuli, pebbles without a clear principal axis were selected (see area Y, Figure 7). 

Figure 7

 

Pebbles selected for Experiment 2: (X) the single pebble used in the low-heterogeneity condition. (Y) The pebbles included in the high-heterogeneity condition (enclosed within the dashed area); these pebbles were selected as they did not have an obvious principal axis, hence rotation would not provide an obvious cue towards the target shadow.

Example stimuli are shown below in Figure 8; apart from the selection of pebbles and the incremental rotation of targets, all other aspects of the stimuli were identical to those described in Experiment 1. Over the course of two sessions each observer undertook a total of 864 trials. 

Figure 8

 

Sample (upright) stimuli for Experiment 2. Target shadow deviations were varied from 30° to 180° in steps of 30°.

Observers

Six participants, 5 male, 1 female, were tested. All participants were undergraduates, post-graduates, or staff of Bristol University's Department of Experimental Psychology. All were naive to the purpose of the experiment. 

Results

The overall error rates for each observer were less than 5%; however, in the more difficult (30°) conditions, error rates occasionally rose as high as 25% ( n = 3) but with no particular display-size bias, reflecting the increased task difficulty. There was no evidence of any speed and accuracy trade-off, neither overall nor in any particular experimental condition (see Supplementary materials for error rates). Even the errors were made with long reaction times. The results are summarized in Figure 9. In general, the RTs for inverted images (red symbols) are higher than for upright images, as in Experiment 1, except very importantly, at the 30° shadow-rotation condition. 

Figure 9

 

Search performance (averaged across observers, for correct responses) across each experimental condition and number of distractors. Error bars represent ±1 SE. NB target orientations are presented as varying in a counterclockwise direction; in fact, they were randomly rotated clockwise and counterclockwise.

Observer search slopes and centercepts were analyzed using ANOVA. For search slopes there were significant main effects of inversion ( F(1, 85) = 11.86, p < 0.02), orientation ( F(5, 85) = 6.21, p < 0.001), and heterogeneity ( F(1, 85) = 8.43, p < 0.05). The main effects were modified by a significant interaction between orientation and heterogeneity ( F(5, 85) = 3.69, p < 0.005) and a marginal interaction between inversion and heterogeneity ( F(1, 85) = 3.41, p = 0.068). This statistical model is reflected in the averaged search slopes presented in Figure 10. Search slopes are lowest for intermediate shadow angles and rise particularly for the 30° condition. Although the differences in slope between upright and inverted images are not significant at any single value of shadow deviation angle, it does seem that the effect of image inversion increases as a function of shadow deviation. 

Figure 10

 

Search slopes plotted as a function of target-shadow deviation angle, pebble heterogeneity, and image inversion. The black lines represent the search slopes for upright images, while the red lines represent the slopes for inverted images. Individually none of the inverted/upright conditions at each orientation have significantly different slope (post-hoc Tukey–Kramer test). Error bars represent ±1 SE.

For centercepts, there were significant main effects of inversion ( F(1, 85) = 15.34, p < 0.05), orientation ( F(5, 85) = 14.81, p < 0.0001), and heterogeneity ( F(1) = 33.41, p < 0.005). These were modified by significant interactions between inversion and orientation ( F(5, 85) = 10.17, p < 0.0001) and orientation and heterogeneity ( F(5, 85) = 18.88, p < 0.0001). Furthermore, post-hoc (Tukey–Kramer) tests revealed significant differences between centercepts at specific target orientations and levels of heterogeneity; these are presented within Figure 11. The centercepts are very high for the 30 degree condition and are small for intermediate angles. Centercept is increased by image inversion only at the greatest angles. 

Figure 11

 

Mean centercepts for visual search as a function of shadow deviation angle, inversion, and heterogeneity. Those centercepts that are significantly different (post-hoc Tukey–Kramer test, corrected for multiple comparisons, p ≤ 0.05) across inversion are indicated (*). Error bars represent ±1 SE.

Discussion

For lower shadow deviations (30°), search slopes and search efficiency are slower for inverted images—though these patterns are only significant for low-heterogeneity centercepts. The reverse is the case for the largest shadow deviations (150–180°) replicating the faster and more efficient search found in Experiment 1. We consider the pattern of results at 30° as a replication of Rensink and Cavanagh's (2004) result; a visual examination of Figure 3 in their paper shows a significant change for gradients and centercept. It is pleasing that our natural image stimuli with attached as well as cast shadows have given the same result as their stylized stimuli, when we have arranged that the shadows have the same 30° deviation. In the current experiment there is a significant centercept difference at 30° but no slope difference. Our failure to find a significant change in search slope may be a function of the greater search times involved in the current experiment, caused by increased heterogeneity and the larger number of distractors. For intermediate angles (90–120°) there seems to be a weak but non-significant effect of inversion, discrepant shadows are identified more quickly in the inverted images. Furthermore, there is a ‘dip’ in the centercept curves for these angles, this dip may reflect a potential confound in these intermediate conditions; where observers were perhaps identifying a difference in the orientation of the elliptical outline drawn around each pebble and shadow pairing, these outlines will differ most where targets are rotated by 90°. However, the 2D outlines become more variable in the high-heterogeneity condition reducing the usefulness of this potential cue, but despite this reduction in the strength of this cue, the ‘dip’ is actually larger in this condition. The pattern of results for the largest shadow rotation angles, i.e., faster and more efficient search for upright images, is congruent with those reported in Rensink and Cavanagh ( 1993). However, our interpretation of this pattern differs from that of Rensink and Cavanagh, these differences will be summarized in the general discussion. 

Experiment 3

Experiments 1 and 2 have examined visual search for natural shadows, featuring both the cast and the attached components; Rensink and Cavanagh's stylized stimuli had only cast shadows. One potential criticism of Experiments 1 and 2 is that it is difficult to distinguish the results reported for cast shadows from those reported for stimuli featuring simple concave and convex bump and dimple stimuli (i.e., Adams, 2007; Chacón, 2004; Champion & Adams, 2007; Enns & Rensink, 1990; Kleffner & Ramachandran, 1992; Sun & Perona, 1996, 1997). This is an inevitable consequence of our desire to examine search for naturalistic objects and their shadows. The current experiment examines whether the results reported in Experiments 1 and 2 are due to the manipulation of the cast or of the attached components. 

Methods

The stimuli feature the main manipulations of the previous experiment: shadow rotations of 30 and 180°, and pebbles with either low or high heterogeneity. Importantly, the images were manipulated so that they either featured cast shadows alone, attached shadows alone, or more natural images with both shadow types. This was achieved by cutting and pasting the ring-flashed images in conjunction with the shadowed images to create combinations where pebbles either featured surface shading but no cast shadows or cast shadows but no surface shading. The low and high heterogeneity manipulations utilized the same pebbles as those featured in Experiment 2 ( Figure 12). 

Figure 12

 

Stimuli featuring different components of natural shadows. For the sake of brevity only the low-heterogeneity stimuli are shown.

Observers

Seven observers were tested; all were post-docs or undergraduates in the Department of Experimental Psychology, University of Bristol; 3 were male and 5 were female. All were naive to the purpose of the experiment. The experiments were carried out according to the ethics guidelines of the Department of Experimental Psychology, University of Bristol. 

Results

Where the shadow angle was 30°, the average error rates for the attached-only condition were around 40% and reaction times for the correct responses toward these stimuli were very long, ranging from 4 to 8 seconds. Where the target shadow angle was 180°, error rates were lower in all conditions (around 5%); however, reaction times for the attached-only shadows again exceeded those in the natural and cast-only conditions. This pattern of reaction times and errors, at least at 30° rotations, for attached shadow stimuli suggests that the presence of attached shadows could not have contributed significantly toward the results reported in the previous experiments. Because the RTs for the attached-only stimuli are so much longer and are inevitably contaminated by correct guesses no further statistical analysis is undertaken for the attached-shadow data (see Supplementary materials and the lower panel of Figure 13 for error rates). 

Figure 13

 

Search slopes (top), centercepts (center), and error rates (bottom) for all experimental conditions.

The minimal adequate ANOVA model for search slopes, for normal and cast shadow data, replicated the main results reported in Experiment 2. There was a significant main effect of heterogeneity ( F(1,71) = 14.22, p ≤ 0.01), i.e., search becomes less efficient as the object casting shadows and consequently the shadows become more variable. There were significant main effect of shadow rotation angle ( F(1,71) = 10.64, p ≤ 0.02): search was more efficient for larger shadow rotation angles. There is a significant interaction between shadow type and shadow rotation angle ( F(1,71) = 8.05, p ≤ 0.006), post-hoc (Tukey–Kramer) tests confirm that this is due to search being more efficient in the normal shadow condition where the target shadow is rotated by 30°. This is particularly true in the high-heterogeneity condition—a 3-way interaction between these factors (shadow rotation, shadow type, and heterogeneity) was just below the level of statistical significance ( F(1,71) = 3.68, p = 0.0644). In short, search is more efficient for normal shadows than cast-only shadows when the shadows are only rotated a little, especially when heterogeneity is high. This may reflect the fact that the surface shading of the pebble helps in the interpretation of the scene or aids in the detection of the target when other cues are at their weakest. 

There was also a significant interaction between inversion and shadow rotation angle ( F(1,71) = 5.1, p ≤ 0.027); this is as reported in Experiment 2. Finally, there was also an interaction between heterogeneity and shadow rotation angle ( F(1,71) = 11.21, p ≤ 0.002). Post-hoc tests confirm that search is even less efficient where there are smaller shadow rotations and where heterogeneity levels are high. 

For centercepts the minimal adequate model only includes shadow rotation angle, heterogeneity, and inversion (in addition to the ‘observer’ random factor). There was no significant effect, nor interaction, for the type of shadow—normal or cast-only, hence data from both types of stimuli are collapsed together; however, all analyses were repeated with the cast-only data and give the same results as those reported here. As with Experiment 2 there was a significant effect of shadow rotation angle ( F(1,81) = 10.35, p ≤ 0.02), and as with Experiments 1 and 2 heterogeneity significantly slowed the centercept ( F(1,81) = 34.71, p ≤ 0.001). As with Experiment 2 there was a significant interaction between the shadow rotation angle and the inversion of stimuli ( F(1,81) = 29.77, p ≤ 0.001). Search was slower for small shadow rotations in the upright images compared to inverted. The reverse was found for the 180° shadow rotation angle. This is illustrated in Figure 14. There was also a significant interaction between the shadow rotation angle and heterogeneity ( F(1,81) = 14.47, p ≤ 0.001); this reflects the fact that centercepts were slower in the 30° high-heterogeneity condition than would be predicted by the main effects alone. This was confirmed by a Tukey–Kramer test. 

Figure 14

 

For the 30° rotation angle there is a significant slowing of the centercept for the upright images.

Discussion

The results of Experiment 3 confirm that the results of Experiments 1 and 2 are not due to manipulations of attached shadows; rather, they are due to the processing of cast shadows. In fact, there is only one instance where there is any statistically significant influence of the attached shadows. There was greater efficiency for natural shadows (with surface shading) in the 30° condition (regardless of inversion). 

Further support for the assertion that the pattern of results reflects the processing of cast rather than attached shadows comes from the fact that, for both the normal and cast-only shadows, the pattern of centercept results was exactly the same as that reported in Experiment 2: for small angular deviations search is slower in upright images, but for larger angular deviations the search becomes slower in the inverted condition. 

General discussion

In all of our experiments, there was a significant effect of image inversion on search. In Experiment 1 search was faster and more efficient for the upright images (presumed illumination from above) compared to inverted images (presumed illumination from below); in Experiments 2 and 3, this was found only for the larger shadow angles. This suggests that separate processing streams do exist for shadows and non-shadows (Rensink & Cavanagh, 2004) and that the “light from above” assumption is important in the designation of ‘shadow’ status. Experiment 1 has shown that where shadow rotations are large, the heterogeneity of shadow casters and the concomitant variability in shadows only has an effect in inverted images, implying that processing within the shadow stream is coarse in nature, consistent with the suggestion of Mamassian ( 2004). Our experiments have encompassed a larger set of shadow angles than Rensink and Cavanagh's, and we have found that the differential processing of shadows and non-shadows is more diverse than we had previously thought. 

Let us consider how this compares to the arguments put forward by Rensink and Cavanagh ( 2004). They argue that their “results support the existence of a process that can rapidly identify regions as shadows” (p. 1352). We agree that such a mechanism, which is based on the light-from-above assumption, is strongly suggested by both their, and our, results. 

Rensink and Cavanagh (p. 1352) also argue that their “results are consistent with a difficulty in accessing the shapes of regions that correspond to cast shadows.” We agree—as we will argue below, their encoding is subject to a high degree of spatial uncertainty. 

Finally, Rensink and Cavanagh (p. 1352) argue that there is a process that identifies the shadows “and then discount(s) them to some extent.” We argue that this statement needs some further thought. Rensink and Cavanagh argue that shadows may be (partly) discounted because they find that inverted images are processed more rapidly. We replicate this effect but find that it only holds when the angular discrepancy between the target and distractor shadows is similar to the one used in their 2004 paper (30°). For larger angular discrepancies, there is no evidence that image inversion speeds things up—rather, inversion now becomes costly and upright images are processed more rapidly, this finding is briefly reported by Rensink and Cavanagh ( 1993). However, as we outline below, our interpretations of these results differ. 

Why should shadows with small discrepancies be detected more slowly in upright images than inverted images while the reverse is found for shadows with large discrepancies? We argue that the key to this discrepancy lies in the second of the three claims of Rensink and Cavanagh—namely, that there is a “difficulty in accessing the shapes of regions that correspond to cast shadows.” This statement is more important than may have been realized at first. It seems reasonable to recast it, which means that small angular deviations will not be clearly encoded by such a spatially coarse mechanism. So the mechanism still recognizes the dark areas as shadows but cannot tell which one has a discrepant angle. For small angular deviations of the target shadow, any advantage of having a fast shadow detector is eliminated by the inability of that detector to deliver a clear description of the detailed spatial properties of the shadow. The problem (of how to find the target) now needs to be solved in a different manner, and this costs extra time. We argue that this is the reason why one finds a disadvantage for identifying upright-image target shadows when the deviation angle is small (e.g., 30°). However, when the angle becomes larger, the situation changes radically. The spatially coarse shadow mechanism can now deliver a clear, and fast, result—and thus, upright images are processed quickly and accurately. This idea that a shadow processor is spatially coarse and can only deliver clear results if the conditions are right allows findings that appear discrepant to be explained in a simple and unified manner. The results of Rensink and Cavanagh with simple stimuli (and a small deviation) lead to an inability to perform the task on shadow information. Our, more complex, stimuli give exactly the same result for similar, small, angular deviations but produce no inversion advantage for larger deviations, for reasons given above. We note that Rensink and Cavanagh (1993) also report a situation in which shadows are processed but there is no inversion advantage. One can now account for all of these apparently contradictory data with one hypothesis—that shadows are detected and processed rapidly, but in a manner which is spatially uncertain or coarse (in agreement with the suggestion of Mamassian, 2004). 

Thus, our findings broadly support the ideas of Rensink and Cavanagh—that there is a special shadow detector, which is fast and spatially uncertain. However, we believe that it is possible to account for all the data without having to argue that the visual system “ignores” or “discounts” shadows in some way. Rather than “ignoring” them, the visual system cannot use shadow information for spatially fine tasks. If such fine tasks are required to be carried out (e.g., for small angular target/distractor deviations), another mechanism must be invoked. The implication is that the shadow mechanism is invoked first, and not in parallel, with the other, more fine-grained, analyzing mechanism. 

The coarse nature of the shadow mechanism may also account for the result that greater image heterogeneity has little effect on the results. A spatially coarse mechanism may not respond to subtle changes in the sizes and shapes of the constituent shadows. Figure 15 shows an outline of the functional relationship between the two mechanisms. 

Figure 15

 

A schematic model of shadow-visual search. Where it is clear that the scene is illuminated from above, shadows are processed as shadows, i.e., the shadow provides cues towards the 3-dimensional structure of the scene. Conversely, when the image is inverted, the shadows are not necessarily processed as such; search is undertaken by general-purpose algorithms, these are slightly slower but have a higher spatial resolution.

The results for Experiment 2 at 30° and those of Rensink and Cavanagh arise when the shadow mechanism has insufficient spatial precision, which has also been argued by Aks and Enns (1992); see also Adams, 2006) who argued that a “quick and dirty” shadow processing mechanism cannot discriminate subtle changes in shadow orientation. Instead, the pattern of results simply reflects a coarse-scale mechanism's response to a relatively fine-scale manipulation. 

The nature of the search that occurs in the upright images is unclear, observers may be searching for the different 2D shapes of the shadows, this search will become less efficient if the shadow has been discounted or if its representation is coarsely scaled. Alternatively, perhaps the search within the upright images is based upon the implied 3D shape of the objects within the scene. A small shift in the orientation of a shadow may imply a small, and unimportant, change in the 3D structure of objects within the scene; conversely when there is a large shift (i.e., 180°) then the implied 3D shape is very different. Consequently, it may be that the responses of observers are driven by the implied 3D shape of the objects within the scene. When images are inverted perhaps the 3-dimensional structure of objects within the scene is not so readily computed, maybe due to the overriding prior for light-from-above percepts (Sun & Perona, 1996). 

Regarding the central issue of suppression versus a coarse representation of shadows, we suggest an account that does not invoke shadow suppression and instead is based on the notion, supported elsewhere (Mamassian, 2004) that shadows are coarsely represented. It may be possible to account for some of the results of individual experiments (those reported here and the work of Rensink & Cavanagh, 1993, 2004) in terms of a suppression account, but we feel that such an account is less able to account for the full set of data reported here. Further work will need to be carried out, especially to learn more about the properties of the spatially coarse shadow representation mechanism. 

Conclusions

Shadows are not discounted or ignored by the human visual system; instead, they are processed rapidly, as distinct signals that provide information about the 3-dimensional nature of the scene being viewed (Elder, Trithart, Pintilie, & MacLean, 2004; Rensink & Cavanagh, 2004). The processing of shadows is conducted in a coarsely scaled manner (Mamassian, 2004), making subtle shadow manipulations difficult to notice. Conversely, where shadow manipulations are relatively large, these manipulations are noticed quickly and efficiently. 

Acknowledgments

This research was supported by EPSRC/Dstl Project Grants (GR/S56405/01 and GR/S56399/01). PGL was employed on GR/S56405/01. 

We would like to thank our reviewers, including Prof. Ronald Rensink (named reviewer) for their very helpful comments offered during the preparation of this manuscript. The authors would like to thank Roland Baddeley for the loan of the Elinchrom ring flash. We would also like to thank Mat Burgess for initial (but unused) photos of pebbles, and Dr. Karen Spencer for her comments on the statistical analysis. We thank Prof. Pascal Mamassian, Dr. Marilyn Gilmore, Dr. Ian Moorhead, and Dr. Michelle To for their helpful suggestions. 

Commercial relationships: none. 

Corresponding author: P. George Lovell. 

Email: [e-mail]. 

Address: Department of Experimental Psychology, University of Bristol, Clifton, Bristol, UK.