Abstract
We examined the effect of perceived orientation on the perceived color of matte surfaces in rendered threedimensional scenes illuminated by a blue diffuse light and a yellow punctate light. On each trial, observers first adjusted the color of a matte test patch, placed near the center of the scene, until it appeared achromatic, and then estimated its orientation by adjusting a monocular gradient probe. The orientation of the test patch was varied from trial to trial by the experimental program, effectively varying the chromaticity of the light mixture from the two light sources that would be absorbed and reemitted by a neutral test patch. We found that observers’ achromatic settings varied with perceived orientation but that observers only partially discounted orientation in making achromatic settings. We developed an equivalent illuminant model for our task in which we assumed that observers discount orientation using possibly erroneous estimates of the chromaticities of the light sources and/or their spatial distribution. We found that the observers’ failures could be explained by two factors: errors in estimating the direction to the punctate light source and errors in estimating the chromaticities of the two light sources. We discuss the pattern of errors in estimating these factors across observers.
Introduction
In a field, on an ordinary sunny day, the light impinging on each object is a mixture of direct sunlight, light from sky and clouds, and light absorbed and reemitted by other objects in the scene. Even when the contribution from other objects is neglected, the light absorbed and reemitted by any surface is composite, a mixture that changes with each passing cloud.
In
Figure 1, we illustrate how light is absorbed and reemitted by a Lambertian (matte) surface illuminated by a punctate source and a diffuse source. The spectral power distribution of the punctate source at each wavelength
λ is denoted by
E_{P} (
λ) and the spectral power distribution of the diffuse light by
E_{D} (
λ). The angle between the punctate light direction and the surface normal
n is denoted by
θ, and the angle between the surface normal and the direction to the viewer is denoted by
v. In the Lambertian model, the intensity of emitted light does not depend on the direction to the viewer, so long as the viewer and the light source are on the same side of the surface. When this condition is satisfied, the intensity of the light reflected from an achromatic Lambertian surface at any wavelength
λ. is given
where
α is the wavelength independent albedo (reflectance) of the achromatic surface. The spectral power distribution of the effective illuminant
E(
λ) is a weighted mixture of the spectral power distributions of the diffuse and punctate sources and the mixture changes as the orientation of the test patch changes. If the diffuse and punctate sources differ in chromaticity, then the chromaticity of the mixture will also change as a function of angle.
Our goal is to examine how human observers perceive surface colors in scenes with composite lighting. In particular, we will test whether they correctly discount surface orientation in estimating the surface color of matte surface patches in simulated scenes illuminated by a yellow punctate source and a blue diffuse source.
Previous research
A number of researchers have investigated how the spatial arrangement of lights and surfaces in a threedimensional (3D) scene affects the perception of lightness or color of a particular matte surface in the scene.
Gilchrist (
1977,
1980; Gilchrist et al.,
1999) examined scenes where the intensity of illumination varied with depth and found that perceived depth affected the perceived lightness of achromatic surfaces. The experimental setup was composed of two rooms that differed in illumination, and were connected by a doorway. Observers viewed this construction through a pinhole. In the two conditions of this experiment, the target patch, whose lightness was to be matched, would be perceived to be located as coplanar with either a brightly illuminated far wall or a dimly illuminated near wall. The actual position and brightness of the test patch were not altered. The test patch was perceived to be white in the near wall condition and almost black in the far wall condition. Gilchrist argued that perceived lightness depends on the relationship between the target and regions with which it is seen as coplanar (the
coplanar ratio hypothesis).
Boyaci, Maloney, and Hersh (
2003) set out to determine whether perceived orientation affects perceived lightness. Their stimuli were binocularly viewed, computerrendered scenes illuminated by a neutral punctate light source and a neutral diffuse light source. Observers first estimated the orientation of an achromatic test patch with Lambertian reflectance properties and then matched its perceived albedo (‘lightness’) to a reference scale. Their results clearly showed that observers systematically discounted the perceived orientation of a surface when estimating its albedo. In fact, observers’ performances closely matched the predictions of the Lambertian model.
How dramatically the perception of 3D shape can influence perceived surface reflectance was demonstrated by Bloj et al. (
1999). They used a chromatic version of the Mach card; one side of the card was painted magenta and the other white, the magenta side casting a pinkish gradient on the white areas. With binocular disparity as the only cue to shape, observers viewed the card (1) in its actual concave shape and (2) through a pseudoscope, which reversed the disparities in left and right eye so that the card appeared to be convex. Bloj and colleagues found significant evidence that observers incorporate information about the shape of the object into their estimates of surface color: The color of the white side of the card was judged by observers to be more pinkish in the apparently convex condition than in the actual concave condition.
Bloj and colleagues studied the effect of 3D shape on color perception for only two viewing conditions. What happens for different angles between the mutually illuminated surfaces? According to the physics of light, the strength of the mutual illumination will depend on the angle between the emitting and receiving surface. Doerschner, Boyaci, and Maloney (
2004) showed that observers systematically discount the angle between a brightly colored surface and an adjacent light gray surface, when setting the color of the latter to be achromatic.
Yang and Shevell (
2003) investigated surface color appearance in simulated scenes illuminated by two punctate light sources differing in chromaticity. The scenes were rendered and presented binocularly, and each consisted of two sidebyside rooms separated by an opaque partition oriented along the line of sight. The back wall of each room was covered with lozengeshaped specular objects. The light sources were placed so that each primarily illuminated one of the rooms and only secondarily the other (thus the primary light source for each room was the secondary for the other). Yang and Shevell (
2003) varied the contribution of the secondary light source to a room by varying the height of the partition dividing the two rooms. When the partition was at its maximum height, each room was illuminated only by its primary light source.
Color constancy was greatest when each room was illuminated exclusively by its primary light source and decreased with increasing admixture of the secondary. In these scenes, each light source created a distinct specular highlight on each of the specular lozenges that it illuminated. Yang and Shevell (
2003) perturbed the chromaticity of these highlights to show that they were effective as cues to the chromaticity of the light reaching each of the room, confirming earlier results (Yang & Maloney,
2001; Yang & Shevell,
2002; Maloney & Yang,
2003).
The results of Yang and Shevell (
2003) suggest that the human visual system can partially discount the relative contributions of two light sources in a scene, at least when there is sufficient information in the scene to permit estimation of the chromaticities and spatial distribution of the light sources. In the experiment we describe next, we will provide considerable visual information about the chromaticity of the light sources and the direction to the punctate light source. Before describing the experiment, we consider its possible outcomes and how we might interpret each.
Possible outcomes
One possibility is that the observer may simply make settings that do not vary with the perceived orientation of the test patch. We would conclude that the observer’s visual system does not take orientation into account in estimating color, at least not in the scenes we employ. This outcome would be consistent with the previous literature just described, although we might wonder why the visual system discounts the effect of orientation in estimating lightness (Boyaci et al.,
2003), but not in estimating color in general.
Alternatively, the observer could discount the effect of perceived orientation perfectly or nearly so. In the next section, we develop a model that allows us to predict the pattern of settings that correspond to this outcome. We will compare the observer’s settings to these predictions. To achieve such perfect constancy despite changing orientation, the observer must effectively take into account the spatial distribution of the light sources, the direction to the punctate light source, the relative intensities of the two sources, and the chromaticities of the two sources. We refer to these quantities together as a lighting model. It is evidently implausible that the observer estimates the lighting model accurately in scenes such as ours with one of the light sources not even visible.
The results of our experiment will allow us to reject both the hypothesis of no constancy and the hypothesis of perfect constancy just described. A third and more realistic possibility lies between these two extremes. It is that the observer makes achromatic settings that are incorrect, but that would be correct for a different lighting model than the one illuminating the scene. A pattern of discounting consistent with incorrect estimates of the lighting model is referred to by Brainard (
1998) as an equivalent illuminant model. Brainard’s equivalent illuminant model consisted of an estimate of the chromaticity of a single light source. He finds that observers’ deviations from color constancy can be parsimoniously explained by the assumption that they have misestimated the chromaticity of the illuminant. We will consider a more complex equivalent illumination model appropriate for a combination of punctate and diffuse sources. We fit this model to observers’ data and determine whether we can reject it. We will discover that it provides a parsimonious summary of observers’ performance in the experiment described next, and we will discuss the pattern of estimates of the lighting model parameters across observers.
Experiment
Introduction
We asked observers to carry out two tasks on each trial. They first set a Lambertian test patch to be achromatic (Helson & Michels,
1948), and then set a gradient probe to indicate its orientation. The test patch was embedded in a scene illuminated by a mixture of a blue diffuse light source and a yellow punctate light source. We varied the orientation of the test patch with respect to the punctate light source from trial to trial, thereby varying the chromaticity of the composite illuminant striking the test patch. We sought to determine whether observers compensated for changes in perceived orientation (and illuminant chromaticity) in their achromatic settings. Note that the orientation task is only included as a way of estimating the observer’s perceived orientation. The focus of the experiment is on the achromatic setting task.
Methods
Stimuli
The stimuli were computerrendered 3Dcomplex scenes composed of simple objects with different shapes (such as spheres and boxes), and with various reflectance properties (such as shiny, matte, and transparent). Each scene contained a matte test patch at the center. The scenes were rendered with the Radiance software package (Larson & Shakespeare,
1996). Each scene was rendered twice, from slightly different viewpoints corresponding to the positions of the observer’s eyes. A stereo image pair for a typical scene is shown in
Figure 2. The other objects in the scene were varied randomly from trial to trial and were included as possible cues to the spatial distribution and chromaticities of the light sources (see Yang & Maloney,
2001).
Apparatus
The experimental apparatus was a Wheatstone stereoscope (
Figure 3). The left and right images of each stereo pair were presented to the corresponding eye of the observer on two 21″ Sony Trinitron Multiscan GDMF500 monitors placed to the left and right of the observer. The screens on these monitors are close to physically flat, with less than 1 mm of deviation across the surface of each monitor.
The stereoscope was contained in a box 124 cm on a side. The front face of the box was removed, and the observer sat there in a chin/head rest. Two small mirrors were placed directly in front of the observer’s eyes. These mirrors reflected the images displayed on the left and right monitor to the corresponding eye of the observer. The interior of the box was coated with blackflocked paper (Edmund Scientific) to absorb stray light. Only the stimuli on the screens of the monitors were visible to the observer. The casings of the monitors and any other features of the room were hidden behind the nonreflective walls of the enclosing box. Additional light baffles were placed near the observer’s face to prevent light from the screens reaching the observer’s eyes directly. The optical distance from each of the observer’s eyes to the corresponding computer screen was 70 cm. To minimize any conflict between binocular disparity and accommodation depth cues, the center of the test patch was rendered to be exactly 70 cm in front of the observer. The monocular fields of view were 55 × 55 deg. The observer’s eyes were approximately at the same height as the center of the scene being viewed, which was also the position of the center of the test patch.
Spatial coordinate system and spatial arrangement
We used a spherical coordinate system (
ψ,
φ,
r) to specify a simulated scene (
Figure 4). This coordinate system has the origin at the center of the test patch. The spherical coordinate system (
ψ,
φ,
r) and the Cartesian coordinate system underlying it are explained in the figure legend.
Color coordinate system
We will describe all light sources and the light radiating from surfaces as weighted mixtures of three abstract primary lights referred to as red, green, and blue (RGB). For convenience, the spectra of these lights coincide with those of the corresponding guns of the monitors, and the three primaries can be thought of as linearized versions of the guns, for that is what they are. We measure the intensities of these three primaries in arbitrary units proportional to their luminance, chosen so that a mixture of the three lights with equal intensities appears roughly achromatic to most observers. We denote the intensities by
E^{R},
E^{G}, and
E^{B}, respectively, and refer to the tristimulus coordinates (
E^{R},
E^{G},
E^{B} that describe the light at a particular location on the monitor as an RGB code.
1 In making an achromatic setting, the observer in effect selects the RGB code for the test patch that makes it appear to be an achromatic surface, as described in more detail below. We report the u’v’ chromaticities (Wyszecki & Stiles,
1982, p. 165) of the guns (and, therefore, of the primaries) in the “
Calibration” section below.
Realism and rendering
Typical rendering packages used in computer graphics represent spectral information about surfaces and light sources by 3D vectors, often referred to as RGB codes. When light with RGB code (
E^{R},
E^{G},
E^{B}) strikes a Lambertian surface with RGB code
S^{R},
S^{G},
S^{B}), the light emitted from the surface is assigned the RGB code (
E^{R} ×
S^{R},
E^{G}, ×
S^{G},
E^{B} ×
S^{B}, scaled by a factor that takes into account the orientation of the surface with respect to the light (see the discussion leading up to
Equation 1). Yang and Maloney
2001; Maloney,
1999) point out that this rendering interpretation (“the RGB heuristic” in Maloney,
1999) does not always lead to accurate simulation of lightsurface interactions.
However, the scenes that we use are designed to avoid the limitations of typical rendering packages. First, we define an achromatic Lambertian surface to be one that multiplies the RGB code of the light that it absorbs and reemits by a constant factor that depends on the albedo of the surface and the direction from which the light arrives. If we assign this neutral surface the RGB code (α, α, α), then typical rendering packages will simulate lightsurface interaction correctly. So long as our chromatic lights interact with only neutral surfaces, the resulting RGB codes assigned to the light reemitted will be accurate. There are other surfaces in our scenes that are rendered, but the RGB codes of these surfaces are assigned at random and change from trial to trial. Consequently, errors in rendering, due to using the RGB heuristic, are of no consequence. The intended random color assigned to a surface is just replaced by a different random color.
We can derive three equations that break
Equation 1 into three RGBcode components. For B, we have,
and there are two analogous equations for R and G, respectively. When the test patch is achromatic, each of the components of the RGBcode of the light emitted by the test patch is the same weighted mixture of the corresponding components of the two light sources.
Calibration
Lookup tables were used to correct the nonlinearities in the gun responses and to equalize the display values on the two monitors. The tables were prepared after direct measurements of the luminance values of each gun on each monitor with a Pritchard PR650 spectrometer. The maximum total luminance achievable on either screen was 114
cd/
m^{2}. To test the linear additivity for a monitor, first we measured the isolated spectrum of each gun alone, set to about half of its maximum intensity. Then we measured the spectra of each pair of guns simultaneously set to half of their maximum intensities and compared it to the sum of the isolated spectra for each gun in the pair. Last, we measured the spectrum with all three guns set to half of their maximum intensity and compared it to the sum of the isolated spectra for all three guns. We plot the results of this last test in
Figure 5 for both monitors. The red, green, and blue solid lines are the isolated spectra, the gray solid line is the sum of the three isolated spectra, and the black dashed line is the measured spectra when all three guns were simultaneously set to half of their maximum intensities. The curves agree to within 7% or better at each point in the spectrum, for both monitors. The test of additivity for pairs of guns also agreed within 7% or less. The u’v’ chromaticity coordinates (Wyszecki & Stiles,
1982, p. 165) for the three primaries are: red (.409,.519), green (.117,.565) and blue (.157,.196) for the left monitor, and red (.430,.528), green (.115,.564), and blue (.160,.189) for the right monitor. The u’v’ chromaticity coordinate for the mixture of all three guns at half intensity was (.176,.460) for the left monitor and (.172,.455) for the right monitor.
Light sources
A yellow punctate light source and a mostly blue diffuse light source illuminated the scenes. The punctate light source was not directly visible to the observer. The RGB code of the punctate light source is denoted (
) and that of the diffuse light source is denoted (
). For convenience, let
and
. To specify the chromaticities of the two light sources and their relative strengths, we define the following set of parameters:
, the bchromaticity
2 of the punctate source,
, the bchromaticity of the diffuse source, and δ =
E_{D}/
E_{P}, the diffusepunctate ratio. The r and gchromaticities of the yellow punctate source were always equal, as were those of the diffuse source. The values used in rendering were
π_{B} = 0,
δ_{B} = 0.66, and and δ = 0.37. In other words, the punctate source had no blue component (
π^{B} = 0), the diffuse source was mostly blue (
δ^{B} = 0.66), and the ratio of the intensity of the diffuse source to the intensity of the punctate source was 0.37 (δ = 0.37). The punctate source was always behind and above the observer, and either to his RIGHT or to his LEFT at (
ψ_{P},
φ_{P},
r_{P}) = (±15°, 30°, 670
cm) (
ψ_{P} = +15° for RIGHT,
ψ_{P} = −15° for LEFT;
Figure 4). The position of the punctate source was varied only from session to session, but in a single session, its position was kept constant. The punctate source was sufficiently far from the test patch so as to treat its light rays collimated. The vector
p = (cos
φ_{P} sin
ψ_{P}, sin
φ_{P}, cos
φ_{P} cos
ψ_{P} is a unit vector pointing from the test patch toward the punctate light source (
Figure 4).
Test patch
Each scene contained a test patch at the center, which was rendered as an achromatic Lambertian surface with an albedo of 0.55 (the observer never saw this patch and we used it only to verify that the output of the Radiance program agreed with the predictions of
Equation 2 and
Equation 3; the parts of the left and the right images corresponding to this patch were replaced by a uniform test patch before the images were shown to the observer). The initial RGB code of the substituted test patch was chosen at random before each trial. The test patch could be displayed with either a rotation in only the
ψ direction (
ψ — rotation) or in only the
φ direction (
φ — rotation). The test patch measured 4.8 cm by 3.6 cm; its center was always 70 cm away from the observer along the observer’s line of sight. The orientation of the test patch was specified by (
ψ_{T},
φ_{T}), and its surface normal was
n = (cos
φ_{T} sin
ψ_{T}, sin
φ_{T}, cos
φ_{T} cos
ψ_{T}). After a
ψ — rotation(
φ_{T} = 0),
φ_{T} could take any of the values {−60°, −45°, −15°, 15°, 45°} when the punctate source was on the LEFT, and any of the values {−45°, −15°, 15°, 45°, 60°} when the punctate source was on the RIGHT. After a
φ — rotation(
ψ_{T} = 0),
φ_{T} could take any of the values {0°, 15°, 30°, 45°, 60°}.
Figure 6 shows a schematic drawing of the two kinds of rotations.
The test patch floated in space in the middle of the scene. It was closer to the observer than all other objects and sufficiently high above the floor so that we could eliminate possible mutual illumination effects.
Angle of incidence
The cosine of the angle between the surface normal of the test patch and the direction to the punctate source is found by employing the law of cosines
or
because
φ_{T} = 0 for a
ψ — rotation and
ψ_{T} = 0 for a
φ — rotation.
Tasks
The observer carried out two tasks on each trial.
Achromatic setting task
The observer first adjusted the chromaticity of the test patch such that it looked achromatic. He or she did this by varying the b, r and gchromaticities without altering the total intensity of the test patch (
Figure 7a). The observer used the arrow keys on the keyboard to perform this task. Pressing the up arrow key increased the bchromaticity while decreasing the r and gchromaticities by the same amount; the down arrow had the opposite effect. Pressing the right arrow key increased the rchromaticity while decreasing the gchromaticity. The left arrow had the opposite effect.
Orientation task
The second task was to estimate the orientation of the test patch (the independent variable) by adjusting a stickandcircle gradient probe superimposed at the center of the test patch (
Figure 7b). The orientation of the probe was controlled by moving a computer mouse. The probe was presented monocularly to the right eye, and the observer had only one degree of freedom on each trial: if it was a
ψ — rotation trial, the probe could rotate only in the
ψ direction; if it was a
ψ — rotation, it could rotate only in the
φ direction. Observers reported no difficulty with setting the probe and were unaware that it was visible in only the right eye. Once the observer was satisfied with the setting, he or she clicked the left button on the computer mouse to finalize the task. The purpose of this task was to control for the possibility that observers’ perceptions of orientation of the test patch were so different from its actual orientation that it would affect the interpretation of the results. We assume that the observer is using the same cues to test patch orientation during the achromatic setting task as in the orientation task. In these scenes, these cues include binocular disparity and linear perspective. See Landy, Maloney, Johnston, and Young (
1995) for a review of cuecombination models for depth and slant.
Software
The experimental software was written by us in the C language. We used the X Window System, Version 11R6 (Scheifler & Gettys,
1996) running under Red Hat Linux 6.1 for graphical display. The computer was a Dell 410 Workstation with a Matrox G450 dual head graphics card and a special purpose graphics driver from Xi Graphics that permitted a single computer to control both monitors. We use the open source physicsbased rendering package Radiance (Larson & Shakespeare,
1996) to render the left and right images that comprised the stereo pair for a given virtual scene. The output of the rendering described above was a stereo image pair with floating point RGB codes for each pixel. We translated the output relative intensity values to 24bit RGB codes, correcting for nonlinearities in the monitors’ responses as described above.
Procedure
The observers repeated each of the 20 conditions (10 test patch orientations, 2 punctate source positions) of the experiment 20 times for a total of 400 trials. The experiment was split into four sessions, each with 100 trials. In a single session the position of the punctate source (LEFT or RIGHT) was kept constant. The order of presentation was randomized. The observers were allowed to perform a few trials before the actual experiment started, until they were completely comfortable with both tasks. The experiment was paced by the observer. Usually observers completed different sessions on different days and each session took less than an hour.
Observers
Four observers completed the experiment. All were experienced psychophysical observers who were unaware of the purpose of the experiment. One other observer was excused after the first session. She had difficulty doing the task and spent more than 3 hours to finish a single session (which usually took other observers under an hour).
Instructions to the observer
For the color task, we asked the observer to adjust the color of the test patch such that it looked as if it were cut out of an achromatic or gray piece of paper. For the orientation task, the observers were simply instructed to move the mouse until the probe’s circles were in the plane of the test patch and the stick was perpendicular to it.
Geometric chromaticity functions
To quantify the observers’ perception and compare it with the model predictions, we define
as the
geometric bchromaticity function. In
Equation 5,
E(
θ) is the total intensity of the light emitted from the test patch.
E^{B} (
θ) is the blue component of the RGB code of the light emitted from the test patch, as defined earlier. The last term in
Equation 5 is gotten by substituting
Equation 1 and
Equation 2 into the middle term.
Equation 5 is the relative intensity of the blue primary in the light emitted from the test patch. We define a geometric gchromaticity function Λ
^{G}(
θ) and a geometric rchromaticity function Λ
^{R}(
θ), analogously, and refer to them collectively as geometric chromaticity functions. Note that when the light sources have the same b, r and gchromaticities, the geometric chromaticity functions are all constant, independent of
θ.
Achromatic setting
Suppose that the observer views a matte test patch in a scene illuminated by a combination of blue diffuse and yellow punctate sources. The angle between the normal to the test patch and the punctate light direction is
θ. The observer is asked to adjust the chromaticity of the test patch without changing the total intensity until it looks achromatic. We denote this achromatic setting as a function of
θ, by (
). Note that his setting is always constrained so that
, and it is convenient to express the achromatic setting in terms of chromaticities.
We define the observer’s geometric blue bchromaticity function
The observer’s geometric r and gchromaticity functions are defined similarly. If the observer were perfectly color constant, then
would coincide with
would be identical to
identical to
and
indentical to
.
By means of the color adjustment task just described, we can measure the observer’s geometric chromaticity functions and compare them to the theoretical ones for an achromatic Lambertian surface in the scenes we employ.
Analysis and results
Orientation settings
We first tested whether changes in the orientation of the test patch in ψ and φ direction had an effect on observers’ orientation settings by separate ANOVAs for each observer. We rejected the hypothesis that the mean orientation setting did not vary with orientation for all observers, for both directions (p < .0001 in both ψ and φ directions). With the exception of subject MM in the ψ direction, we found no significant interaction between perceived test patch orientation and punctate light source position (LEFT or RIGHT) for both directions (ψ direction: p = .206, .637, and .304 for BH, MD, and RG, respectively, p = .01 for MM; φ direction: p = .852, .39, .07, and .928 for BH, MD, MM, and RG, respectively). This implies that for all but one observer the position of the light source (LEFT or RIGHT) had no significant effect on how observers made their orientation settings.
Figure 8 shows one observer’s (BH) mean settings when the punctate source was positioned on the left. For each observer, we regressed the observer’s mean orientation settings on the true orientation settings separately in both the
ψ and
φ (
vs.
ψ_{T} and
vs.
φ_{T}) directions. We have plotted the bestfitting regression lines to BH’s settings in
Figure 8. We report the regression coefficients for similar fits for all observers in
Table 1. The estimated regression coefficient
a (intercept) is in units of degrees; the regression coefficient
b (slope) is unitless. We report
p values for hypothesis tests against the corresponding veridical value (0 for
a, 1 for
b). In the
φ direction, slopes were significantly different from 1 for ject MD (punctate on RIGHT:
p < .001) and ject MM (RIGHT:
p < .001). All other subjects’ slopes in this direction were not significantly different from 1 for both punctate source positions (LEFT:
p = .934, .037, and .148 for BH, MM, and RG, respectively; RIGHT:
p = .834, .01, and .125 for BH, MD, and RG, respectively). The intercepts in the
φ direction were significantly different from 0 for all observers (
p < .001) except for observer RG (LEFT:
p = .782, RIGHT:
p = .042).
Table 1
Results: orientation setting regression coefficients. We fit a linear model to the orientation settings (
vs.
ψ_{T} and
vs.
φ_{T}) separately for each observer. The estimated regression coefficient
a (intercept) is in units of degrees, the regression coefficient
b (slope) is unitless. We report
p values for hypothesis tests against the corresponding veridical value (0 for
a, 1 for
b). We report exact
p values when the values are larger than .001. With a Bonferroni correction for 16 tests per observer, the significant level corresponding to an overall Type I Error rate of 0.05 for each subject is .0031. Values whose corresponding
p values fall below this cutoff are marked with an asterisk.
Table 1
Results: orientation setting regression coefficients. We fit a linear model to the orientation settings (
vs.
ψ_{T} and
vs.
φ_{T}) separately for each observer. The estimated regression coefficient
a (intercept) is in units of degrees, the regression coefficient
b (slope) is unitless. We report
p values for hypothesis tests against the corresponding veridical value (0 for
a, 1 for
b). We report exact
p values when the values are larger than .001. With a Bonferroni correction for 16 tests per observer, the significant level corresponding to an overall Type I Error rate of 0.05 for each subject is .0031. Values whose corresponding
p values fall below this cutoff are marked with an asterisk.

Punctate source position: LEFT 
Punctate source position: RIGHT 






A

b

A

b

a

b

a

b

Veridical 
0

1

0

1

0

1

0

1

BH

3.33* 
1.06

4.83* 
0.98

3.18* 
1.06

7.42* 
0.96

p < .001

p = .009

p < .001

p = .934

p < .001

p = .005

p < .001

p = .834

MD

−2.33* 
1.14* 
4.72* 
1.14* 
2.54* 
1.1* 
3.55* 
1.08

p = .002

p < .001

p < .001

p < .001

p = .002

p < .001

p = .003

p = .001

MM

−0.55

0.83* 
−6.59* 
0.95

−1.76

0.76* 
−4.6* 
0.87* 
p = .398

p < .001

p < .001

p = .037

p = .038

p < .001

p < .001

p < .001

RG

−4.79* 
0.71* 
0.38

0.95

−5.7* 
0.66* 
−1.68

0.97

p < .001

p < .001

p = .782

p = .148

p < .001

p < .042

p < .125

In the ψ direction slopes were significantly different from 1 for all observers (p < .001), except observer BH (LEFT: p = .009, RIGHT: p = .005). The intercepts in the ψ direction were significantly different from 0 for all observers (p < .002) except MM (LEFT: p = .398, RIGHT: p = .038).
We conclude that observers’ perceived orientation changes with orientation but that observers’ perception of orientation is not veridical. We will use the observers’ own estimates of orientation in analyzing the effect of orientation on achromatic settings. However, we note that it would not alter our conclusions in any important respect if we used the true orientations instead.
Achromatic settings
Observers’ achromatic settings are the key dependent variable of our experiment. Consider a model observer who is effectively using
Equation 5 to arrive at estimates of surface color appearance but whose estimates of some or all of the parameters in the equation were in error. The achromatic setting of the model observer would not match the geometric chromaticity function of
Equation 5 and would instead exhibit characteristic failures due to errors in the parameter estimates. In this section, we examine the effect of errors in each parameter on the model observer’s predicted performance.
An equivalent illuminant model
The chromaticities of the light sources in the rendered scenes differ only in the blueyellow balance; therefore, we are primarily interested in the blue and yellow components of the observers’ achromatic settings, and we first consider the geometric bchromaticity function.
Equation 5 can be rewritten as
where the variables
π_{b},
δ_{B} and δ were defined above. An observer’s visual system can compute what the bchromaticity of a gray surface should be if estimates of the parameters in
Equation 7 are available. However, if the observer’s estimates of the parameters are in error, the achromatic settings would differ from the predicted ones. Let
, and
denote the observer’s estimates of the parameters in
Equation 5, then
The observer’s estimate of the angle of incidence
depends on his or her estimates of the orientation of the test patch (
) and the direction to the punctate source (
) through
Equation 4. Note that the observers explicitly estimated the orientation of the test patch
by performing the orientation task.
Suppose that we hold the lighting conditions constant, in particular the parameters
and vary the orientation of the surface by varying
φ_{T} as we do in the experiment.
Figure 9a shows the geometric bchromaticity function Λ
^{B} plotted with respect to the angles
ψ_{T} and
φ_{T} assuming the veridical values of the lighting parameters, Θ. Now suppose that the estimates of the lighting parameters
are in error, what kind of distortions would those errors introduce? Misestimating the direction to the punctate source shifts both curves without much effect on their curvatures (
Figure 9b). When the test patch is oriented such that it faces the punctate source as directly as possible, that is
ψ_{T} =
ψ_{P} (after a
ψ rotation) or
(after a
φ rotation), it receives the maximum possible amount of light from the yellow punctate source. However the blue content of the mixture of light falling on it remains fixed, hence the bchromaticity, λ
^{B}, assumes its minimum.
More rigorously, the extrema of the function Λ
^{B} are found by taking its derivative with respect to ψ
^{B} and
φ_{T} and then equating it to zero, which
(For
ψ_{P} = ±15,
φ*
_{T} = 30.87°, only slightly different from
φ_{P} = 30°.) Note that
ψ*
_{T} and
φ*
_{T} correspond to minima for
π^{B} <
δ^{B}, and to maxima for
π^{B} >
δ^{B} (see
Equation 13 below). If the model observer misestimated the punctate source direction, his or her achromatic settings would reveal this because the corresponding geometric bchromaticity function
would shift and have its minimum at roughly the estimated direction to the punctate source (
).
Errors in estimating the parameters
π^{B},
δ^{B}, and δ would shift the curves up or down and increase or decrease their curvatures (
Figure 10). If the observer’s estimates of the bchromaticity of the punctate source and diffuse source were the same (
), then the geometric bchromaticity function would be a constant, because changing the orientation of the test patch would not affect the overall chromatic balance of the light reaching the patch.
would be constant also when
or
, that is, if the observer estimates that the scene is illuminated either by only a punctate source or by only a diffuse source. However, because veridical values are such that
π^{B} ≠
δ^{B} and Δ is not 0 or infinity, should we find that the observer’s geometric bchromaticity function is constant, then the implication is that the observer does not discount the perceived orientation of the test patch for its color.
Figure 11 shows the empirical geometric bchromaticity functions for all four observers. As mentioned above, if an observer were perfectly color constant, then his or her data would fall on the theoretical curve of the geometric bchromaticity function Λ
^{B}. On the other hand, if the observer were completely ignoring the orientation of the test patch in his or her achromatic judgment, then the ratio would be constant.
It is clear that observers take the orientation of the test patch into account and have some degree of color constancy, although the constancy is not perfect. A comparison of the patterns of data to the family of Λ
^{B} curves in
Figure 9 and
Figure 10 suggests that observers make settings that are indistinguishable from those of a Lambertian color constant observer who discounts the perceived orientation for estimating color, but who does so using incorrect estimates of the lighting parameters Θ.
We use a maximum likelihood fitting procedure to estimate values of the lighting parameters
that best accounted for each observer’s data separately (recall that we explicitly measured observers estimates of
ψ^{T} and
φ^{T}.) These estimates are reported in
Table 2.
Table 2
Achromatic setting: maximum likelihood estimates of lighting parameters. We report the maximum likelihood estimations of the punctate source direction (
) and the lighting parameters (
). The parameter
π^{B} is the bchromaticity of the punctate source,
δ^{T}is bchromaticity of the diffuse source, and δ are equal to is the ratio of the intensity of the diffuse source to the intensity of the punctate. For each observer, we tested the hypotheses that
and
are equal to the veridical values and report exact
p values for the tests when the values are larger than .001. With a Bonferroni correction for 40 tests (four observers, five parameters, two punctate source positions), the significant level corresponding to an overall Type I Error rate of .05 is .00125. Values whose corresponding
p values fall below this cutoff are marked with an asterisk. All observers’ punctate source direction estimates were not significantly different than the veridical values. However, in contrast with their success in estimating the punctate source direction, observers misestimated the other lighting parameters (
). The deviations from the veridical values were significant (except for observer BH when the punctate source was on the right). Those deviations in the lighting parameter estimates result in failure to discount perceived orientation of the test patch for its perceived color exactly as the model predicts.
Table 2
Achromatic setting: maximum likelihood estimates of lighting parameters. We report the maximum likelihood estimations of the punctate source direction (
) and the lighting parameters (
). The parameter
π^{B} is the bchromaticity of the punctate source,
δ^{T}is bchromaticity of the diffuse source, and δ are equal to is the ratio of the intensity of the diffuse source to the intensity of the punctate. For each observer, we tested the hypotheses that
and
are equal to the veridical values and report exact
p values for the tests when the values are larger than .001. With a Bonferroni correction for 40 tests (four observers, five parameters, two punctate source positions), the significant level corresponding to an overall Type I Error rate of .05 is .00125. Values whose corresponding
p values fall below this cutoff are marked with an asterisk. All observers’ punctate source direction estimates were not significantly different than the veridical values. However, in contrast with their success in estimating the punctate source direction, observers misestimated the other lighting parameters (
). The deviations from the veridical values were significant (except for observer BH when the punctate source was on the right). Those deviations in the lighting parameter estimates result in failure to discount perceived orientation of the test patch for its perceived color exactly as the model predicts.

Punctate source position: LEFT 
Punctate source position: RIGHT 





Veridical 
(−15,30)

(0, 0.66,0.37)

(15, 30)

(0, 0.66,0.37)

BH

(−23.1, 36.6)

(0, 0.33, 1.39)* 
(18.4, 36.1)

(0.1, 1, 0.12)

p = .011

p < .001

p = .007

p = .036

MD

(−22.4, 28.8)

(0.1, 0.38, 1.87)* 
(15.2, 38.8)

(0, 0.4, 1.6)* 
p = .63

p < .001

p = .04

p < .001

MM

(−15.8, 39.3)

(0.2, 1, 0.03* 
(20.3, 37.9)

(0.2, 1, 0.03)* 
p = .734

p < .001

p = .426

p < .001

RG

(−54.4, 39.5)

(0.2, 0.31, 2.15)* 
(5.6, 37.4)

(0.2, 1, 0.05)* 
p = .029

p < .001

p = .016

p < .001

We tested the hypothesis that the observer’s estimate of the punctate source direction
was equal to the veridical values by means of a nested hypothesis test (Mood, Graybill, & Boes,
1974, p. 440). We nested the hypothesis that
and
(their true values) within a model in which they were free to vary. We fit both models to the data by the method of maximum likelihood with other parameters allowed to vary freely. The log likelihood of the constraint model (denoted by
λ_{0}) must be less than or equal to that of the unconstraint model (denoted by
λ_{1}). Under the null hypothesis, twice the difference in log likelihoods is asymptotically distributed as a
X^{2}_{2}variable
if We use this result to test whether observers’ estimates
were significantly different from the true values. We summarize the values of the observers’ estimates
in
Table 2 along with the corresponding
p values. None of the observers’ punctate source direction estimates were significantly different from their true values under all conditions. Observers’ azimuth and elevation estimates of the punctate source direction were within 11 deg of the true direction with one exception (the azimuth estimate for RG with the light on the LEFT; See
Table 2). We replot the azimuth and elevation estimates in
Table 2 as
Figure 12. It is then readily seen that the estimates
are clustered around the true values (with one exception, RG, LEFT) and that the estimates
are typically too large. We have, in effect, derived a crude estimate of the position of the light from the observer’s performance.
We next examine the three outcomes discussed in the “Introduction.” We first considered the hypothesis that observers’ achromatic settings were not affected by changes in test patch orientation. If this were so, then we would find that the geometric bchromaticity function was constant. We rejected this hypothesis for all observers in all conditions (p < .0001 in all cases). We conclude that the observers’ achromatic settings are affected by changes in test patch orientation.
We next tested the hypothesis that observers correctly discounted the effect of orientation in making achromatic settings (i.e., were observers accurately estimating the bchromaticities of the punctate and diffuse sources,
and
, and the diffusepunctate balance
and using these estimates to discount the effect of rotations?). We tested whether observers’ estimates were equal to true values
π^{B} = 0,
π^{B} = 0.66 and δ = 0.37. We nested the hypothesis that the parameters were equal to the true values within a model in which they were free to vary. We rejected the hypothesis that
for all observers except observer BH for the punctateontheright condition (
p = .036; all other
p values are reported in
Table 2).
The analysis of the results suggest that one possible reason for the failure of the perfect discounting is that the observers’ estimates of the chromaticity balances of the punctate and diffuse sources, and of the diffusepunctate balance, are in error. All observers slightly overestimated the bchromaticity of the punctate source
(veridical value is
π^{B} = 0), and misestimated the bchromaticity of the diffuse source
δ^{B} and the diffusepunctate balance Δ. The values are reported in
Table 2. It is as if observers are discounting the orientation of the test patch for the achromatic task consistent with the physically correct model but using incorrect estimates of the lighting parameters.
Redgreen balance
We also examined the redgreen balances of the achromatic settings. We define
as the redgreen balance. We present only one observer’s (MM) settings in
Figure 13. As expected, the redgreen balance of achromatic settings did not change systematically with changes in the orientation of the test patch. All other observers’ results were similar.
A neutral light source control
As the test patch rotates away from the yellow punctate light source not only the relative blue content of the patch increases but also the overall intensity of the light from the test patch decreases (as described for Lambertian surfaces). What if these two events are confounded, that is, what if subjects simply assume that darker objects appear more blue? We verified that this is not the case by letting one observer (MD) run an extra session in the experiment where the punctate and diffuse light sources were neutral (
and
;
). We found no effect of orientation on his achromatic settings, neither for bluetotal balance (
Figure 14) nor for redgreen balance. This result is consistent with
Equation 7, and indicates, for example, that observers do not simply increase the blue content of a gray surface as its total intensity decreases.
Geometric discounting index
We can quantify the observer’s performance in discounting the perceived orientation for perceived color by, first of all, noting how close his or her estimates of light source direction are to the true direction. This measurement, though, does not reflect errors in the observer’s estimates of the remaining lighting parameters. We define a geometric discounting index that effectively compares the curvature of the observer’s bgeometric chromaticity function at its minima,
where
and
and
and
are corresponding curvatures of the observers’ bgeometric chromaticity function calculated in the same way.
DI is a measure of how much the observer’s geometric bchromaticity function is curved compared to the theoretical one. A complete lack of curvature (
) corresponds to the case where the observer’s achromatic settings are unaffected by angle. Then
DI = 0, and we would conclude that the observer’s color estimates are not affected by perceived surface orientation. If, in contrast,
then
DI = 1. Note that
DI is a composite measure of all the parameters in the parameter space
. However, because the estimated parameters
were close to veridical, the errors are effectively due only to the misestimated chromaticity balances of the punctate source and diffuse source and the overall diffusepunctate balance. The values of
DI varied between 0.29 and 0.82 (
DI = 0: no discount;
DI = 1: perfect discount). The discounting indices are reported in
Table 3 and in the legends of
Figure 11.
Table 3 Discounting Indices. We define a discounting index DI to quantify how well observers discounted the effective illumination in the experimental scenes. DI is a comparison of how the observers’ achromatic settings vary with perceived test patch orientation, and how they should vary if the observer is correctly discounting the effective illumination on the test patch. The exact form of DI is given in the text. A zero value of DI means no discounting (the achromatic setting is unaffected by perceived orientation). A value of 1 corresponds to perfect discounting. All observers partially compensated for changes in effective illumination due to changes in test patch orientation.
Table 3 Discounting Indices. We define a discounting index DI to quantify how well observers discounted the effective illumination in the experimental scenes. DI is a comparison of how the observers’ achromatic settings vary with perceived test patch orientation, and how they should vary if the observer is correctly discounting the effective illumination on the test patch. The exact form of DI is given in the text. A zero value of DI means no discounting (the achromatic setting is unaffected by perceived orientation). A value of 1 corresponds to perfect discounting. All observers partially compensated for changes in effective illumination due to changes in test patch orientation.
 Discounting index, DI 
 Punctate source position: LEFT  Punctate source position: RIGHT 
BH  0.78  0.8 
MD  0.69  0.82 
MM  0.29  0.27 
RG  0.2  0.43 
Conclusion
When a scene is illuminated by punctate and diffuse light sources differing in chromaticity, the chromaticity of the light absorbed by any patch or surface depends on its orientation. We report an experiment in which observers were asked to view rendered 3D scenes binocularly. The lighting in these scenes was composite, a mixture of a yellow punctate light source and a blue diffuse. On each trial, observers adjusted a test patch to be achromatic (achromatic setting task) and then adjusted a gradient probe to match the orientation of the patch (orientation task). We varied the orientation of the test patch randomly across trials. The location of the punctate source could be either LEFT or RIGHT, and was fixed for a given session.
We used the results of the gradient probe task to estimate the observer’s perceived orientation. We showed that observers systematically take the perceived orientation of the test patch into account in making achromatic settings. However, their settings do not match the settings of an ideal Lambertian observer who has knowledge of the parameters of the composite lighting model (the location of the punctate light source and the chromaticities and intensities of the diffuse and punctate light sources).
We refit the observers’ data on the assumption that the observer correctly discounted the illumination arriving at the test patch in making achromatic settings, but that, in doing so, he or she made use of estimates of parameters of the composite lighting model that were in error. We found very good agreement between this equivalent illuminant model of the observer’s performance and our data. All of the observers’ estimates of the direction to the punctate source were close to veridical. The deviations from the model were for the most part consistent with the hypothesis that observers failed to use correct estimates of the chromaticities of the punctate source and of the diffuse source, and the ratio of intensity of the diffuse source to the intensity of the punctate source.
It has been shown that the perceived geometry of a scene influences the lightness (perceived albedo) of surfaces (Gilchrist,
1977,
1980; Gilchrist et al.,
1999; Boyaci et al.,
2003). There are also studies that show that the perceived color of surfaces is influenced by the spatial arrangements permitting mutual illumination (Bloj et al.,
1999; Doerschner et al.,
2004). Because we used several different orientations of the Lambertian test patch, we were able to parametrically determine how observers’ surface color estimates are influenced by surface orientation in scenes with composite light models. Observers do take into account the 3D structure of scenes and the lighting model of the scene in arriving at estimates of surface color. Their shortcomings are predominantly consistent with failures to estimate the parameters of the lighting model correctly.
Our results suggest that the observer, in effect, develops a model or estimate of the spatial distribution and chromaticities of light sources in a scene as part of color visual processing. Maloney and Yang (
2003; Maloney,
1999,
2002) review previous work related to the hypothesis that the visual system develops an estimate of illuminant chromaticity, and it is natural to extend this “illuminant estimation hypothesis” to include explicit estimation of the spatial distribution of light sources as well.
In retrospect, it is perhaps not surprising that the human observer, living in a world that often is illuminated by sky and sun differing in chromaticity, would be able to compensate for the effect of orientation in arriving at estimates of perceived color. However, this study is, to our knowledge, the first to show that the human visual system does so, even partially. These results for color together with previous results for lightness (Boyaci et al.,
2003) and for mutual illumination (Bloj et al.,
1999; Doerschner et al.,
2004) support the claim that the visual system effectively estimates the spatial and chromatic properties of the illuminant.
How it does so raises a new and exciting set of questions about surface color perception (Maloney,
1999,
2002). How, for example, does the visual system arrive at the admittedly imperfect estimates of punctate source direction that we derive from the observers’ own performance? Recall that the punctate light source in our scenes is not even directly visible to the observer. There are several candidate cues that might provide this information as well as information about the chromaticities and relative intensities of the light sources: specular highlights, cast shadows, and attached shadows (shading). It will be of interest to determine which of these cues are actually used by the visual system. Our results also indicate that the observers’ estimates of light source chromaticity and spatial distribution of light sources can be markedly in error.
We do not claim that observers are aware of their lighting models or of the cues that signal it (cf. Rutherford & Brainard,
2002). Kafka (
1911/1988, p. 63) described the lighting in his room as, “The lights and shadows thrown on the walls and the ceiling by the electric lights in the street and the bridge down below are distorted, partly spoiled, overlapping, and hard to follow. When they installed the electric arclamps down below and when they furnished this room, there was simply no housewifely consideration given to how my room would look from the sofa at this hour without any lights of its own.” Unlike his room, our scenes were designed carefully and with “housewifely consideration.” Yet, looking at the scenes of this experiment, the observers did not have a better understanding of the purposes of the “creator”: When asked after the experiment, observers were not even aware that the punctate source changed its position from session to session, or that there was a blue background. Their visual system simply took care of the “overlapping, and hard to follow details” for them.
Acknowledgments
This research was funded in part by National Institute of Health Grant EY08266. HB and LTM were also supported by grant RG0109/1999B from the Human Frontiers Science Program. We thank Michael Landy for comments on earlier drafts and David Brainard for comments on this work in poster form.
Commercial relationships: none.
Corresponding author: Huseyin Boyaci.
Address: New York University, Department of Psychology & Center for Neural Science, 6 Washington Place, 877 New York, NY 10003.
Footnote
We use the term RGB code as a convenient synonym for the tristimulus coordinates based on the three linearized sources (‘guns’) of the monitor (Wyszecki & Stiles,
1982).
The RGB code is the tristimulus coordinates with respect to the three linearized sources of the monitor and we define the r, g and bchromaticities e.g. b = B/R + G + B) in the usual manner (Wyszecki & Stiles,
1982).
References
Bloj, M. G.
Kersten, D.
Hurlbert, A. C.
(1999). Perception of threedimensional shape influences colour perception through mutual illumination.
Nature, 402, 877–879. [
PubMed]
[PubMed]
Brainard, D. H.
(1998). Color constancy in the nearly natural image. 2. Achromatic loci.
Journal of the Optical Society of America A, 15, 307–325. [
PubMed]
[CrossRef]
Gilchrist, A. L.
(1977). Perceived lightness depends on spatial arrangement.
Science, 195, 185–187. [
PubMed]
[CrossRef] [PubMed]
Gilchrist, A. L.
(1980), When does perceived lightness depend on perceived spatial arrangement?
Perception & Psychophysics, 28, 527–538. [
PubMed]
[CrossRef] [PubMed]
Gilchrist, A.
Kossyfidis, C.
Bonato, F.
Agostini, T.
Cataliotti, J.
Li, X. J.
1999. (An anchoring theory of lightness perception).
Psychological Review. 106, 795–834,
PubMed
[CrossRef] [PubMed]
Helson, H.
Michels, W. C.
(1948). The effect of chromatic adaptation on achromaticity.
Journal of the. Optical Society of America, 38, 1025–1032.
[CrossRef] [PubMed]
Kafka, F.
(1988). The diaries, 1910\21923 (Brod, M., Ed., & J. Kresh, Trans.) New York: Schocken Books. (Original work published in 1911)
Landy, M. S.
Maloney, L. T.
Johnston, E. B.
Young, M. J.
(1995). Measurement and modeling of depth cue combination: In defense of weak fusion.
Vision Research, 35, 389–412. [
PubMed]
[CrossRef] [PubMed]
Larson, G. W.
Shakespeare, R.
(1996). Rendering with radiance; The art and science of lighting and visualization. San Francisco: Morgan Kaufmann Publishers, Inc.
Maloney, L. T.
Gegenfurtner, K. R.
Sharpe, L. T.
(1999). Color vision: From genes to perception 387–422 (pp. Cambridge, UK: Cambridge University Press).
Maloney, L. T.
Yang, J. N.
Mausfeld, R.
D., Heyer
(2003). The illumination estimation hypothesis and surface color perception. Colour: Connecting the mind to the physical world (pp. 335–358). Oxford: Oxford University Press.
Mood, A. M.
Graybill, F. A.
Boes, D. C.
(1974). Introduction to the theory of statistics (3rd ed.). New York: McGrawHill.
Rutherford, M. D.
Brainard, D. H.
(2002). Lightness constancy: A direct test of the illumination estimation hypothesis.
Psychological Science, 13, 142–149. [
PubMed]
[CrossRef] [PubMed]
Scheifler, R. W.
Gettys, J.
(1996). X Window System; Core library and standards. Boston: Digital Press.
Yang, J. N.
Maloney, L.T.
(2001). Illuminant cues in surface color perception: Tests of three candidate cues.
Vision Research, 41, 2581–2600. [
PubMed]
[CrossRef] [PubMed]
Yang, J. N.
Shevell, S. K.
(2002). Stereo disparity improves color constancy.
Vision Research, 42, 1979–1989. [
PubMed]
[CrossRef] [PubMed]
Wyszecki, G.
Stiles, W. S.
(1982). Color science; concepts and methods, quantitative data and formulae (2nd ed.). New York: Wiley.