A common light-prior for visual search, shape, and reflectance judgments

Abstract

The “light-from-above” prior is invoked to simplify and expedite complex visual processing. This prior is observed in visual search and shape judgments with shaded stimuli, where perceived shape and ease of target identification are both affected by stimulus orientation. In addition, perceived surface reflectance varies with surface orientation in a manner consistent with assumed overhead lighting. Do the light-priors exhibited by these different tasks have the same underlying mechanism or even lighting direction? Some evidence has suggested that an “above-left” rather than “above” prior guides behavior in some tasks, but not others. In the current study, the “light-from-above” prior was measured using visual search, shape perception, and a novel reflectance-judgment task. There were substantial differences between observers. However, strong positive correlations were found between the light-priors measured using all three tasks. The data imply that a single mechanism is responsible for a light-from-above prior in “quick and dirty” visual search behavior, shape perception, and reflectance judgments. Furthermore, the data support the notion that perceived shape is the preattentive feature in visual search with shaded targets.

Introduction

The visual system must quickly and accurately construct a three-dimensional representation of its surroundings. To this end, it relies on prior assumptions or knowledge about statistical regularities in the environment including isotropy and homogeneity of texture (e.g., Knill, 1998), surface convexity (e.g., Langer & Bülthoff, 2001; Mamassian & Landy, 1998), and slow and smooth motion (Weiss, Simoncelli, & Adelson, 2002). Another such visual prior, the “light-from-above” prior, is used to recover shape from otherwise ambiguous shading (Brewster, 1826). Observers generally see the top-left object in Figure 1a as concave and the other three objects as convex, consistent with an assumed overhead light source. The light-from-above prior has also been measured using a visual search task (Sun & Perona, 1998); detection is more efficient when targets and distractors have vertical shading gradients. When shading gradients are horizontal (consistent with side lighting), then detection becomes difficult (Kleffner & Ramachandran, 1992). Adams, Graf, and Ernst (2004) showed that, in the absence of explicit light-source information, observers also assume that light is coming from roughly overhead when making surface reflectance judgments. 

Figure 1

 

Examples of stimuli used in the three experiments. (a) Shape task: Observers usually see the top left object as concave and the others as convex, consistent with the “light-from-above” prior. (b) Visual search: Target detection is relatively difficult until the figure (or observer) is rotated by 90°. (c) Reflectance task: The left object is perceived as a homogenous object lit from above, whereas the right object appears to have a darker upper surface, consistent with overhead lighting.

The question addressed here is, do all these varied tasks reflect a common light-prior? Some evidence suggests that they might: Adams et al. (2004) exposed observers to an environment in which objects (like those in Figure 1a) were consistently lit from the side. The new shape interpretations were reinforced by haptic (touch) cues. Observers learned to assume a shifted light source when making shape judgments, and their reflectance judgments were also affected in line with the new prior. However, it is not clear whether the observed learning in this study was specific to the trained environment, also visual search behavior was not assessed. Here light-priors are measured across three different tasks, without specific training, to investigate whether a common prior for lighting is observed. 

There is evidence that visual search performance with shape-from-shading (SFS) stimuli depends on perceived shape or reflectance rather than orientation per se (Enns & Rensink, 1990; Sun & Perona 1996), although some debate exists as to whether shape is a preattentive feature (see Wolfe & Horowitz, 2004). If shape is the driving feature in visual search with the type of stimuli seen in Figure 1b, then similar light-priors should be measured when observers are performing shape judgments and search. 

Alternatively, the different tasks may exhibit different light-priors; there has been some debate on whether humans exhibit a light-from-above prior or a light-from-above-left prior (Mamassian & Goutcher, 2001; McManus, Buckman, & Woolley, 2004; Sun & Perona, 1998). Differences between the measured light-priors across and within these studies have been attributed to task differences (visual search vs. shape judgments) by the authors of the latter two papers. In addition, Adams (2007) found that in shape perception, but not visual search, the assumed light direction is affected by orientation relative to gravity. Therefore, from these studies, we might not expect a close relationship between the light-priors measured with different tasks. Although behavior in all three tasks considered here (shape judgments, reflectance judgments, and target detection) might be roughly consistent with an assumption of above lighting, this could reflect a general predominance of overhead lighting in our visual experience rather than demonstrating a common mechanism. 

We can look for evidence of a single mechanism by investigating whether individual variations in light-prior are common to multiple visual behaviors. The light-prior varies substantially across observers, as reported in this study (standard deviation = 28.3°) and previous studies ( σ = 25.7°, McManus et al., 2004; σ = 15.4°, Mamassian & Goutcher, 2001; σ = 15.3°, Jenkin, Jenkin, Dyde, & Harris, 2004). Although this variation is a puzzle to explain, it allows a neat study of whether individual variations in light-prior are reflected in visual performance across quite different tasks and stimuli. 

Methods

Observers

Twenty-seven observers, including the author and two other experienced observers, completed three separate experiments (see Figure 1). All had normal or corrected-to-normal vision. Observers gave informed consent, and approval was given by the University of Southampton ethics committee. 

Stimuli and procedure

Shape experiment

Observers viewed four shaded disks (see Figure 1a) in various configurations (either three with the same orientation and one with the opposite orientation, or two the same and two opposite) on a CRT display. The stimuli were consistent with squashed hemispheres illuminated by a single, distant light source. Importantly, however, there was no actual source of illumination in the room; it was completely dark, aside from the light emitted by the monitor. The (simulated) illumination direction is defined relative to a coordinate frame centered at the screen's center, where the X- and Y-axes are in the horizontal and the vertical directions in the plane of the screen. The Z-axis is positive toward the observer. The slant of the light source (the angle between the lighting vector and the Z-axis) was held constant at 55°. The tilt of the light source (the angle between the projection of the lighting vector and the Y-axis), or equivalently the orientation of the stimulus, varied across trials. Each shaded disk subtended 1.5° and was 2.4° from the central fixation cross at the viewing distance of 75 cm. On each trial, either one or two of the disks was rotated by 180° relative to the remaining disks. Halfway through the presentation time of 1,200 ms, a star appeared, indicating which of the four disks should be judged. After the stimulus disappeared, the observer had an unlimited time to respond “convex” or “concave” via key-presses. Stimuli were presented 16 times at each of 16 equally spaced orientations in random order. All 256 trials were presented in a single session lasting approximately 20 min. The observer's head was fixed for the duration of the experiment by an individually molded bite-bar and one eye was covered by an eye-patch. 

Visual search experiment

Sixteen disks of diameter 1.2° were presented in two slightly jittered rings (radii 1.9° and 4.6°) around a central cross. The observer was informed before the experiment that on 50% of trials, one disk (the target) would be rotated by 180° relative to the distractors. The observer's task was to detect, as quickly and as accurately as possible, target presence or absence. The stimulus remained visible until the subject responded. Incorrect responses were followed by a flashed red screen. Each observer completed 1,152 trials (48 repetitions of present and absent trials at 12 orientations) in a single session lasting approximately 35 min. Other details were similar to the “shape” experiment. 

Reflectance experiment

On each trial, two objects appeared side by side for 2,500 ms. Each subtended 3.4° and was displaced 2.4° laterally from the center of the display. The objects were consistent with tetrahedrons squashed in depth and each illuminated by a separate distant light source, with a constant slant of 68°. The tilt of the two light sources always differed from each other by 120°. For example, in Figure 1c, the left object is illuminated from the above right and the right object is illuminated roughly from below. Each object was always consistent with a tetrahedron whose sides all had equal reflectance. However, this was not the subject's perception. By assuming a single light source (see, e.g., Ramachandran, 1988), the observer assumed that one object had all three sides painted the same shade, whereas the other differed in pigment across the three sides. The observer's task, on each trial, was to identify the object with three sides of equal reflectance, taking as much time to respond as required. The orientation of the two objects and the colour of the objects varied randomly across trials. The mean illumination tilt direction (θ) also varied randomly, as did the position (left or right) of the object illuminated by a light source at θ + 60° (the other illuminated from θ − 60°). The solid rings piercing each tetrahedron ensured that the two objects were always seen as convex. The rings' shading was consistent with the mean illumination direction, such that observers' responses were not biased. Each observer completed 384 trials (16 repetitions at 16 orientations) in a single session of around 30 min. 

Results

Figure 2a shows data for the shape experiment for one naive, representative observer (B.C.). The perceived shape of the stimulus changes as a function of its orientation: disks bright at the top appear convex, and those bright at the bottom appear concave. The observer's fitted light-prior (see legend) is shown by the arrow. 

Figure 2

 

(a) “Shape” data: The proportion of stimuli perceived as convex, as a function of orientation. Zero degree corresponds to a stimulus that is lighter at the top. The fit (dashed red) is a simple function comprising two cumulative Gaussians (see Equation A1). The fitted light-prior is the center of the range seen as convex. (b) Visual search data in terms of “perf” (Santhi & Reeves, 2004) where perf = d′² / (RT_correct − RT_motor). RT_correct is the average RT of correct responses (calculated from reciprocal RTs to correct for skew). RT_motor is the motor response component of the RTs, estimated from the tenth percentile of correct RTs. d′ indicates d prime. The model fit (dashed blue) is a simple combination of two sine waves plus a constant (see Equation A2). The phase gives the observer's light-prior (arrow). (c) Reflectance data: The proportion of trials that objects were perceived as having homogenous reflectance, as a function of the average illumination of the two objects (&z.sbrhr;). Data for the object illuminated from direction &z.sbrhr;-60° are plotted in green and those illuminated from direction &z.sbrhr;+60° are given in red. Data are fitted by sine waves (dashed lines) and the average phase is the light-prior.

The same observer's data for the visual search task are plotted in Figure 2b, in terms of search efficiency as a function of target orientation. The observer is more efficient (smaller reaction times and/or lower error rates) when the stimulus contains only vertical gradients (i.e., can be interpreted as top-lit, rather than illuminated from the left or the right). Figure 2c shows the observer's judgments of equal reflectance, as a function of illumination direction. Objects were perceived as having all sides with equal reflectance when they were consistent with illumination from above. The observer's light-prior estimated from these data is given by a green arrow. 

Not all of the subjects' data for the reflectance experiment varied as coherently across lighting directions. Data from seven of the 27 observers were excluded from this experiment, as a reliable light-prior estimate could not be calculated (see Figure 3 legend). This in itself suggests that the prior for light-from-above is not as strong for making reflectance judgments as it is for recovering shape from shading, although the two processes are clearly linked (Sun & Perona, 1996). One of these seven subjects was also excluded from the shape experiment because he perceived convexity with equal probability across all orientations. 

Figure 3

 

Scatterplots showing the relationship between the light-priors from the three tasks. Errors give ±1 SE from bootstrapping. The diagonal (dotted line) gives the prediction for identical light-priors across tasks. The dashed line gives the best-fit for the data, treating errors in X and Y equally. Observers were excluded from the correlation calculations if a reliable light-prior estimate could not be recovered ( SE > 25°, n = 7). Under a conservative approach of replacing these missing data points by the mean, all correlations remained significant (shape/search: r = .72, p < .001; shape/reflectance: r = .49, p = .005; search/reflectance: r = .4, p = .019).

For each of the three experiments, The light-prior was extracted for each observer. The mean and standard deviation of light-priors across observers were as follows: shape: μ = −13.9°, σ = 37.9°; visual search: μ = −0.8°, σ = 28.5°; and reflectance: μ = −2.6°, σ = 24.7°. Although the average light-prior was to the left of vertical in all three experiments, none deviated significantly from 0 (shape: t ₂₅ = −1.8, p = .07; search: t ₂₆ = −0.14, p = .89; reflectance: t ₁₉ = −0.47, p = .64). A single light-prior was calculated for each observer by averaging across the three tasks ( μ = −5.1°, σ = 28.3°), again providing little support for a leftward bias in the population ( t ₂₆ = −0.93, p = .36) especially of the size previously reported (−26° and −16° by Sun & Perona, 1998, and Mamassian & Goutcher, 2001). 

The primary aim of this study was to discover whether behavior in the three distinct tasks reflects a common light-prior. Significant correlations were found between the measured light-priors for the three tasks ( Figure 3). These were shape/search: r = .74, p < .001; shape/reflectance: r = .55, p = .006; and search/reflectance: r = .53, p = .008. 

Discussion

A strong relationship was found between shape perception and visual search. We know that three-dimensional shape is, in general, more readily perceived from vertical shading gradients than horizontal gradients (Adams et al., 2004; Curran & Johnston, 1996). Similarly, visual search is more efficient with vertically shaded objects. However, what the current study demonstrates is that individuals deviate substantially from these rules, and that such deviations reflect a single lighting prior affecting both shape perception and visual search behavior. In other words, observers that interpret shading patterns as though lit from the side when estimating shape also detect targets more efficiently in a scene that is consistent with side lighting. This finding strongly supports the notion that shape (or reflectance derived from shape) is a preattentive feature and is inconsistent with the notion that visual search is based on orientation per se, in SFS displays. This confirms experimental findings from Enns and Rensink (1990) who found pop-out in displays with a three-dimensional interpretation but serial search with stimuli containing similar spatial changes in luminance yet giving rise to two-dimensional interpretations. Hanazawa and Komatsu (2001) showed that this preference for overhead lighting can be seen in the tuning of V4 cell responses to shaded surfaces under varying illumination directions. Lee, Yang, Romero, and Mumford (2002) found that weak neural correlates of pop-out with SFS stimuli in V1 and stronger correlates in V2 were affected by trained behavioral relevance. Early processing of cast shadows has also been observed: Rensink and Cavanagh (2004) demonstrated that cast shadows can be processed (and discounted) preattentively, but only in scenes consistent with overhead lighting. 

A novel stimulus (with unambiguous shape) was developed to measure the effect of assumed light position on reflectance judgments. Most observers judged reflectance in a way roughly consistent with overhead lighting; however, a quarter did not appear to use any assumed directional light source to interpret these stimuli. The light-from-above prior appears to have a weaker role when making reflectance rather than shape judgments. The data may also reflect a greater willingness to abandon the assumption of surface homogeneity when abrupt rather than gradual luminance changes are present (compare Figure 1c with Figure 1a, and for a discussion of abrupt vs. gradual luminance changes, see Adelson, 2000). 

Significant correlations were found between reflectance and shape judgments and between reflectance and search behavior, suggesting that all three of the tasks used here reflect a single light-prior. These latter two correlations were weaker than the relationship between the search prior and the shape prior. However, the difference between the correlation coefficients was not significant and to some extent reflects the difference in precision with which the priors could be measured from the data for the three tasks. To gain a more accurate measure of the relationships between the three tasks, the correlations were disattenuated to account for unreliability, using the standard formula. For each task, the data were bootstrapped to create 500 sets of observers' light-prior estimates. Averaging the correlations between all possible pairs of sets gave estimates of test–retest reliabilities for the three tasks of shape: .99; search: .92; and reflectance: .82. The resultant disattenuated correlations were shape/search: r = .78; shape/reflectance: r = .61; and search/reflectance: r = .61. 

Two possible explanations exist for the observed correlations. The three different judgments used here might involve common processing, in effect a single light-prior directing all three tasks. Alternatively, the observed correlations could be the result of independent learning in three separate mechanisms exposed to common environmental illumination distributions. The former is more plausible for two reasons: Firstly, it is unlikely that an observer with an extreme light-prior (e.g., 90°) has experienced significantly more right-illuminated environments while getting feedback (e.g., from other shape cues) to facilitate learning in all three presented tasks. Secondly, the crossover to a lightness task shown by Adams et al. (2004) from training on a shape task suggests a single mechanism. Similarly, models of lighting and surface perception now emphasize the importance of three-dimensional scene structure in reflectance and brightness judgments (e.g., Adelson, 1993; Knill & Kersten, 1991). However, it should be noted that although strong and significant correlations were observed between all tasks in the current study, they were all less than 1. This leaves open the possibility that in addition to the apparent commonalities, some task-specific learning may also occur that can affect individual light-prior measures. An example of this has recently been demonstrated where only some of the changes induced by training on a shape task transferred to performance in a visual search task (Champion & Adams, in press). 

The huge variability in light-priors across observers is in itself very interesting. For example, in the shape task, five observers had light-priors deviating from overhead by more than 50°. It has been suggested that some interobserver variance might be accounted for by handedness (Sun & Perona, 1998) whereby observers tend to orient themselves or the light source such that the dominant hand does not cast an obtrusive shadow. This would lead to a more leftward bias in right than left handers. In fact, the opposite (but nonsignificant) pattern was found in all three of the current tasks, with left handers displaying a stronger leftward bias. Mamassian and Goutcher (2001) also failed to find any relationship between handedness and light-prior. 

McManus et al. (2004) suggested that spontaneous head tilt may at least partially account for individual differences and reported a significant relationship between head tilt and measured light-prior. However, in their study, subjects performed visual search and shape tasks without any head fixation. Their apparent correlation between head tilt and light-prior was therefore likely to have simply reflected the predominance of the retinal frame of reference in assumed light direction (e.g., Adams (2007); Howard, Bergström, & Ohmi, 1990; Jenkin et al., 2004; Kleffner & Ramachandran, 1992; Yonas, Kuskowski, & Sternfels, 1979). To test the possibility that spontaneous head tilt would produce an aftereffect when the head is horizontal, I measured the light-priors (using the shape task, with head fixed) and the spontaneous head tilt of an additional 22 naive observers. Head tilt was measured as the orientation of a line joining the pupils from photographs of the observers standing against a wall. Head tilt showed far less variation than the light-priors (head tilt: μ = 0.93°, σ = 3.0°; light-prior: μ = −6.6°, σ = 66°) and could not account for a significant proportion of the variance in observers' light-priors (r = −.094, p = .68). 

In summary, the current data show that the light-prior varies substantially across observers. The reason for such large interobserver differences remains unclear. It seems unlikely that particular observers have been immersed in environments lit from one side over long time frames. The large differences between observers are more likely to reflect short-term environmental experience. This is in line with the fast adaptation of the light-prior (10° in 1.5 hr) reported by Adams et al. (2004). Critically, correlated deviations from above lighting were present across divergent tasks, suggesting a unitary light-prior. The current data support the notion that shape is a driving feature in visual search. 

Appendix A

The shape data ( Figure 2a) were fit by a simple function comprising two cumulative Gaussians, such that the mean of each Gaussian was the transition from convex to concave (or concave to convex) judgments:  

$p(\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{v}\mathrm{e}\mathrm{x}(\theta \left)\right)=1+\mathrm{C}\mathrm{u}\mathrm{m}\mathrm{u}\mathrm{l}\mathrm{G}\mathrm{a}\mathrm{u}\mathrm{s}\mathrm{s}(\theta ,[{\mu }_2,\sigma \left]\right)-\mathrm{C}\mathrm{u}\mathrm{m}\mathrm{u}\mathrm{l}\mathrm{G}\mathrm{a}\mathrm{u}\mathrm{s}\mathrm{s}(\theta ,[{\mu }_1,\sigma \left]\right),$

(A1)

where CumulGauss( θ, [ μ, σ]) is the cumulative function (integral from −∞ to θ) of the Gaussian distribution with mean μ and standard deviation σ. The standard deviation (common to both Gaussians) and the two means ( μ ₁ and μ ₂) were fit as free parameters separately for each observer. The function gives the probability of perceiving a stimulus as convex, as a function of its orientation, θ. Each observer's light-prior is the average of the two means (or equivalently the center of the range of orientations seen as convex). It should be noted that the function is not continuous at θ = 0°. For participants not at ceiling at θ = 0° (i.e., those with extreme light-priors), concave responses were fit. The light-prior was then given by ( μ ₂ − μ ₁) / 2 − π. 

As described above, visual search performance ( Figure 2b) was quantified in terms of “perf” (Santhi & Reeves, 2004). Observers' variation in performance, as a function of target orientation, θ, was modeled using the function 

$\mathrm{p}\mathrm{e}\mathrm{r}\mathrm{f}\left(\theta \right)=a\cos \left(2\right(\theta -\alpha \left)\right)+b\cos (\theta -\alpha )+c.$

(A2)

The two amplitudes (a and b), the constant (c), and the phase (α) were fit as free parameters, separately for each observer. The first component of the function (with two peaks) reflects the ease of detecting clearly convex or concave targets, relative to target detection in displays with roughly horizontal shading gradients and ambiguous depth. The second component (with a single peak) corresponds to the asymmetry between detection of convex and concave targets (generally, concave targets are more easily detected than convex ones, e.g., Kleffner & Ramachandran, 1992). The phase, α, gives the observer's light-prior, that is, the peak of performance. 

Acknowledgments

Thanks to Helen Page and Katie Gray for assistance with data collection, and thanks to Erich Graf and Rebecca Champion for comments on the manuscript. This work was supported by EPSRC grant EP/D039916/1. 

Commercial relationships: none. 

Corresponding author: Wendy J. Adams. 

Email: [email protected]. 

Address: School of Psychology, University of Southampton, SO17 1BJ, UK.