Color constancy improves for real 3D objects

Abstract

In this study human color constancy was tested for two-dimensional (2D) and three-dimensional (3D) setups with real objects and lights. Four different illuminant changes, a natural selection task and a wide choice of target colors were used. We found that color constancy was better when the target color was learned as a 3D object in a cue-rich 3D scene than in a 2D setup. This improvement was independent of the target color and the illuminant change. We were not able to find any evidence that frequently experienced illuminant changes are better compensated for than unusual ones. Normalizing individual color constancy hit rates by the corresponding color memory hit rates yields a color constancy index, which is indicative of observers' true ability to compensate for illuminant changes.

Introduction

In our everyday life, we refer to color as a constant property of an object. However, the light reflected from an observed object not only depends on the reflectance properties of the object's surface but also on the illumination, other objects, their location and position with respect to the illuminant (or illuminants) and each other. Thus, the light signal reaching our eyes varies considerably, but our perception of the object's surface color stays constant because the visual system compensates for these various changes. This ability is known as color constancy (e.g., Jameson & Hurvich, 1989; Kaiser & Boynton, 1996). 

Despite intensive research, it is still unclear how color constancy is achieved by the visual system. The challenge for the visual system is to recover information about the illumination and the object reflectance in a scene from a single signal. If it is possible to estimate the illuminant correctly, then the visual system could accurately assess surface reflectance. Therefore, the more visual cues there are regarding the illuminant in a scene, the more accurate the visual system's estimate of the surface reflectance properties of the object should be. In the following part of the Introduction section, we will assess what information can be obtained from illuminant cues, discuss the role of color memory for color constancy, and review the different stimuli that researchers have used to study color constancy, before we come to the motivation for our study. 

Illuminant cues

When light illuminates a scene, a series of interactions take place, which provide cues about the illuminant. Objects cast shadows giving information about the position and the number of light sources. Light reflected between surfaces gives rise to mutual illumination, which provides information about the surface reflectance of the objects involved, chromaticity of the local incident light (Funt & Drew, 1993), and the scene geometry (Nayar, Ikeuchi, & Kanade, 1991). Specular highlights arise from shiny surfaces providing details about location and chromaticity of the light source or sources (Yang & Maloney, 2001; Yang & Shevell, 2003). However, a single visual cue on its own does not allow the visual system to be color constant. It is the variety and combination of several visual cues that support a more accurate estimate of an object's color (e.g., Kraft & Brainard, 1999; Kraft, Maloney, & Brainard, 2002). 

Color memory

A crucial issue in color constancy research is color memory. When color constancy is tested with a successive matching task, an observer compares a stimulus he or she remembers with a present one. Thus, without color memory an observer will be unable to achieve constancy. 

Studies of color memory draw an inconclusive picture of how well color is remembered and to what extent prior experience with a stimulus affects color memory. Hamwi and Landis (1955) studied color memory for different time delays (15 min, 24 h, and 65 h) and found that the remembered colors varied very little from the original and colors were well remembered independent of the time delay. Subsequent studies investigated the influence of time, texture, shape, and context on color memory (e.g., Bartleson, 1960; Hunt, 1989; Loftus, 1977; Newhall, Burnham, & Clark, 1957; Olkkonen, Hansen, & Gegenfurtner, 2008; Perez-Carpinell, Baldovi, De Fez, & Castro, 1998; Ratner & McCarthy, 1990; Siple & Springer, 1983). These studies varied considerably in their methods but all reported that color could not be remembered accurately and that the remembered color shifted with respect to the original. However, none of the studies could identify a consistent shift pattern that would have allowed the prediction of color memory. 

An achromatic setting task avoids relying on color memory because it uses the individuals' internal representation of gray. Observers are asked to manipulate a test patch until it appears achromatic (e.g., Bäuml, 1999; Boyaci, Doerschner, & Maloney, 2004; Brainard, 1998; Delahunt & Brainard, 2004a; Doerschner, Boyaci, & Maloney, 2004; Rinner & Gegenfurtner, 2000; Yang & Maloney, 2001). If the illuminant of the scene is estimated accurately and therefore is taken into account by observers, their settings will correspond to the achromatic point under the prevailing illuminant. Note that this setting is not physically achromatic but reflects the chromaticity of the incident illumination. Thus, the level of color constancy can be directly estimated from observers' settings, i.e., the deviations from the true achromatic point (Speigle & Brainard, 1999). 

Stimuli used in color constancy research

Stimuli and setups for color constancy research vary considerably. In the following, we will review the stimuli and setups used so far with an emphasis on their dimensionality, namely whether two- (2D) or three-dimensional (3D) setups were used. 

Research on color constancy has often been carried out with 2D stimuli in 2D environments, i.e., with computer-generated stimuli consisting of simple geometric forms, which were simulated as flat matte surfaces and presented under spatially uniform illumination on monitors (e.g., Arend & Reeves, 1986; Bäuml, 1999; Chichilnisky & Wandell, 1995; Jin & Shevell, 1996; Murray, Daugirdiene, Vaitkevicius, Kulikowski, & Stanikunas, 2006). Of these studies only Bäuml (1999) provides color constancy indices (between 0.79 and 0.84) whereas the others either express their results differently or refer to figures. Many different layouts were created by varying the complexity of the surrounding area of a test patch. Nevertheless, as complex as such backgrounds may be, they can only provide a limited range of visual cues, significantly less than are usually found in a natural scene. 

A more sophisticated method was developed by Amano, Foster, and Nascimento (2006), who used 2D hyperspectral images of natural scenes. Calibrated RGB images were generated from the hyperspectral images, presented on a monitor and manipulated such that the same scene appeared to be under different illuminations. Based on a test surface included in the photograph (a gray sphere), the observers judged the kind of illumination change that occurred between two images presented consecutively. Here, color constancy indices varied from 0.56 to 0.88. 

A further step toward three-dimensionality is to present images stereoscopically. A number of studies have used this approach using computer-rendered complex scenes (Boyaci et al., 2004; Doerschner et al., 2004; Schultz, Doerschner, & Maloney, 2006; Yang & Shevell, 2003). In the study by Boyaci et al. (2004) for example, the scene consisted of simple geometrical objects with different reflectance properties and was illuminated by a blue diffuse and a yellow point-like light source. The test surface in the middle of the scene was set to an arbitrary color before each trial and observers had to perform an achromatic setting by varying the chromaticity of the test surface. Boyaci et al. studied the effect of surface orientation on achromatic settings. They found that observers took the orientation of the test surface into account and that the achieved level of color constancy was good but not perfect. 

Experiments using real 3D objects are rare in color constancy research because they are difficult to control and to manipulate. However, Brainard (1998) introduced an experimental setup that consisted of real surfaces and objects. Observers could see the walls, floor, and ceiling of the experimental room, two objects (a white table and a brown metal bookcase), and the test patch, which was a gray Munsell paper mounted on the back wall of the room. The immediate surround of the test patch could be varied by displaying other Munsell papers next to it. The appearance of the test patch was controlled by illumination. Observers performed a series of achromatic settings under a variety of illuminants and conditions. Brainard reported that overall observers showed high levels of color constancy (between 0.76 and 0.82). Another real-world 3D setup was used by Kraft and Brainard (1999). Their setup included a chamber with several geometric volumes, a tin foil covered tube, and an array of different colored papers. All objects could be removed from the scene. The back wall was replaceable and the scene illumination was computer-controlled. The test patch was a gray piece of paper and even though the observer experienced its changed color appearance as a result of a surface change its appearance was entirely manipulated by illumination. In a subsequent series of experiments, the authors investigated three mechanisms for color constancy (Kraft & Brainard, 1999):

1.  
   adaptation to the local surround,
2.  
   adaptation to the spatial mean of a scene, and
3.  
   adaptation to the most intense image region.

The color constancy index varied between 0.11 and 0.83 and although all three mechanisms were silenced in one condition, the level of constancy did not drop to zero. The results revealed that none of the three mechanisms could account for color constancy completely and the authors suggested that there may be other cues and mechanisms contributing to color constancy apart from the ones studied.

Real objects and surfaces were also used by de Almeida, Fiadeiro, and Nascimento (2004) who presented two almost identical scenes concurrently to their observers. All objects and surfaces in the left and right scenes were identical, except for a cube in the middle of each scene. In the left scene, the 3D cube was made of real paper representing a test color, whereas in the right scene it was a virtual image of a cube whose color appearance could be manipulated via a computer. Both scenes were presented under different illuminations and observers adjusted the appearance of the virtual cube, by varying its chromaticity and luminance, until it appeared to be made of the same paper as the real cube in the other scene. The resulting color matches were fairly accurate throughout all surface colors and illuminants tested, revealing a high level of color constancy (the color constancy index varied from 0.81 to 0.93). The authors argued that the good results were a consequence of the 3D setup and the natural-looking stimuli. 

Ling and Hurlbert (2006) studied the effect of color memory on color constancy using a 3D dome that always represented the test color. After memorizing the test color, observers selected the matching color patch from a selection of 2D patches. Throughout the experiment, the changes in appearance of the dome and the patches did not arise from an actual surface reflectance change but were generated by computer-controlled lighting illuminating white surfaces. The achieved level of color constancy lies between 0.61 and 0.84. The authors reported that the used paradigm was not ideal because the sudden apparent change of the surface color of the dome and the patches was interpreted by the observers as artificial and not as a real surface change. Therefore, the observers might have made an appearance match instead of a surface match. 

Zaidi and Bostic (2008) also used a 3D setup to study object identification across illumination changes. Observers were presented with four real objects, of which three had the same reflectance properties. Two objects were viewed under illuminant 1, two under illuminant 2. Observers had to identify the one object under illuminant 2, which had different reflectance properties from the one shown under illuminant 1. The authors argue that their results could not be explained by color constancy, contrast constancy, inverse optics, or neural signal matching algorithms, but rather by a similarity-based suboptimal strategy that saves on the computational costs. 

Illuminant changes

Previous studies comparing color constancy across diverse illuminant changes have drawn an inconclusive picture. It seems plausible to hypothesize that human color constancy is best for illuminant changes that are experienced naturally; e.g., variations in natural daylight, as this is the light in which the visual system has evolved (see Shepard, 1992). The chromaticities of daylight and also the likes of some artificial light sources, for instance, candlelight and tungsten light, are located on or near the blackbody locus (Planckian locus). Changes between those are frequently experienced in our daily routine (e.g., change between daylight and tungsten light). We will therefore refer to illuminants on the blackbody locus and changes between them as typical. Illuminants with chromaticities that do not lie on the blackbody locus will be referred to as atypical as well as changes between an illuminant on and one off the blackbody locus (e.g., change between daylight and a purple illuminant), because they are hardly ever experienced. 

Brainard (1998) used two illuminants close to and a further nine off the blackbody locus and concluded from his results that the visual system compensates equally well for illumination changes on and off the blackbody locus. However, Ruttiger, Mayser, Serey, and Sharpe (2001) found actually higher color constancy for red–green illuminant changes than for daylight changes. Delahunt and Brainard (2004b) could not report a clear advantage of daylight illuminant changes over other illuminant changes. In their study, the highest color constancy was found indeed for one of the two illuminants chosen from the blackbody locus, but the second highest constancy was achieved for a green illuminant change. Daugirdiene, Murray, Vaitkevicius, and Kulikowski (2006) also compared color constancy levels for on- and off-blackbody locus illuminants and did not find superior constancy for the on-blackbody locus illuminants. To summarize, there is no evidence that the visual system compensates more effectively for typical than for atypical illuminant changes. 

Present study

While Brainard and colleagues (Brainard, 1998; Kraft & Brainard, 1999) and de Almeida et al. (2004) have shown and argued that the naturalness of their stimuli was instrumental in achieving good color constancy performance, nobody has actually compared 2D with 3D real-world setups, in which appearance changes correspond to true surface reflectance changes. As Kraft and Brainard's (1999) work has indicated, providing a wide range of visual cues improves color constancy, therefore we propose that learning a color in a cue-rich environment (3D), where the target color is represented by a 3D object, will lead to better color constancy performance than in a cue-poor environment (2D), where the target color is a flat swatch. 

Using real surface changes also requires abandoning an achromatic setting task and employing a task, which resembles an everyday life situation, namely the selection of a color swatch from alternatives relying on color memory. In the present study, all objects (2D patches and 3D objects) were real. In the learning phase, stimuli were either 2D paper swatches or geometrical 3D objects, whereas we always used 2D paper swatches in the test phase. In all cases, surface appearance was directly related to real surfaces rather than simulating surface changes through illuminant changes. While our selection task seems more natural, it requires establishing a baseline condition, i.e., an assessment of observers' color memory performance because their color constancy performance will be limited by their color memory performance. To ensure the generality of our findings, we wanted to study more than one illuminant change. For this, we investigated whether color constancy in our natural setup under typical illumination changes (shifts on blackbody locus) is different to that observed under atypical illumination changes (shifts away and toward blackbody locus). We argue that the previously reported absence of benefit for natural/typical lighting conditions could be due to the kind of task and unnatural setups that have been used up to now. 

Methods

Experimental setup

The experiments took place in a lighting booth sized 230 × 230 × 230 cm. The two sidewalls and the back wall were covered with black wool cloth to provide minimally reflective surfaces and the floor was covered with dark blue carpet. An additional cloth divided the booth into two compartments. One compartment was used as a palette showroom and the other one contained the 3D scene (see Figure 1). 

Figure 1

 

The lighting booth showing the (left) palette and (right) scene compartments. The palette light source is visible on the top left.

Illuminants

Two low-voltage spotlights (Altman MR16 Micro Ellipses, 75 W, 36° reflectors) were used in the experiments: one to illuminate the palette and the other one to illuminate the 3D scene. Filters were placed in front of the spotlights to adjust their chromaticities, whereas a dimmer box (Betapack 2 by zero88) controlled the intensity. 

A LEE Filter (Lee Filters, 2008) No. 201 Full C.T. Blue was used to generate the illuminant D1. For D1, the intensity of the spotlight was set to 100%. Illuminant D2 was generated by using a LEE Filter No. 202 Half C.T. Blue; the intensity of the spotlight was set to 60%. The illuminants, referred to as Tun (abbreviation for tungsten light) and Lily, were generated by a LEE Filter No. 204 Full C.T. Orange and a LEE Filter No. 704 Lily, respectively. For Tun, the spotlights ran at 100% intensity, whereas for Lily the intensity was set to 60%. Table 1 provides the CIE xy chromaticities, luminances, and correlated color temperatures of the illuminants, and Figure 2 shows the four illuminants in a CIE xy chromaticity diagram. D1, D2, and Tun were chosen to lie on the blackbody locus, while Lily was clearly off the blackbody locus. The typical illuminant change was from D1 to Tun and Tun to D1; the atypical illuminant change was from D2 to Lily and Lily to D2. Measurements were taken with a spectroradiometer (PR650) using a certified white reflectance standard (Labsphere®) and following the standard illuminating and viewing conditions recommended by the CIE (Commision Internationale de L'Eclairage—International Commission on Illumination). The (0/45) condition was used, in which the incident beam of the spotlight coincided with the normal of the reflectance standard (0 deg) and the spectroradiometer measured the reflected light at 45 deg (Wyszecki & Stiles, 2000). 

Table 1

 

CIE chromaticity and luminance values of all four illuminants; in the last column, the correlated color temperatures for the illuminants on the blackbody locus are shown.

Illuminant	CIE x	CIE y	Lum (cd/m 2)	Color temperature (K)
D1	0.370	0.377	84.52	4290
Tun	0.517	0.425	152.9	2160
D2	0.418	0.405	60.40	3350
Lily	0.462	0.352	44.61	–

Figure 2

 

Location of the four experimental illuminants in the CIE xy chromaticity diagram.

Stimuli

Colored paper samples from the NCS color collection (Scandinavian Colour Institute: Natural Colour System, 2004) were used to create stimuli for the experiment. Forty-eight colors from the entire hue circle were chosen and divided into three color groups: blue, red, and yellow (see Figure A2). Each color group consisted of 16 different colors. From the 16 colors of a group, two were selected as target colors and identified as B9 and B16 for the blue, R8 and R10 for the red, and Y8 and Y10 for the yellow color group. The NCS notation of all 48 colors is listed in Table A1 and target colors are highlighted. Further information on the NCS notation and our patch selection can be found in 1. 

The CIE xyY values of all swatches under illuminants D1, D2, Tun, and Lily were also measured as described in the Illuminants section. Figure 3 shows the distribution of the 48 colors under the four experimental illuminants in the approximately perceptually uniform color space CIE L*a*b*. 

Figure 3

 

Circles represent the positions of all 48 color swatches under the four different experimental illuminants in the CIE a*b* chromaticity diagram. Filled circles indicate target colors. (A) Illuminant D1. (B) Illuminant Tun. (C) Illuminant D2. (D) Illuminant Lily. See Table A1 for NCS notation of color swatches.

Learning and test palettes

All palettes were 30 × 30 cm in size and painted with matte blackboard paint, except one palette, which was painted with matte white paint and was used during the adaptation periods of the experiments. All black palettes contained 16 color swatches (4 × 4 layout), which were cut from the NCS papers. Each swatch was 5 × 5 cm and subtended approximately 1.8 deg of visual angle. The outer swatches were placed 3.5 cm away from the rim of the palette and all swatches were separated from each other by a 1-cm gap (see Figure 4B). The swatches were arranged pseudo-randomly on the grid. The target colors were placed in different locations but never in a corner. The swatches surrounding the target colors also varied their positions between palettes. 

Figure 4

 

(A) Schematic of a learning palette. (B) Schematic of a test palette from the blue color group. All dimensions are in centimeters. All palettes, learning as well as test palettes, had the same dimensions. See text for details.

For the learning phase, three different learning palettes were created, which contained the six target colors together with ten other NCS papers selected randomly from all three color groups (see Figure 4A). 

For the selection phase, four test palettes for each color group were created (12 palettes altogether). A test palette was composed of the two target colors and 14 colors from the same color group. This variety of test palettes was produced to present the 16 colors of a group in different arrangements and therefore minimize the possibility of memorizing positions of color swatches. All palettes were used in any of the four rotational positions. 

Two-dimensional scene

The 2D scene was set up in the left compartment of the lighting booth (palette showroom, Figure 1). A palette was placed on a table covered with black wool cloth. A stand on the table tilted the palette at 45 deg, so that the spotlight mounted above illuminated the palette at 0 deg (i.e., parallel to the surface normal). Observers were seated 1.5 m away from the lower edge of the palette and the whole palette subtended a visual angle of approximately 11 × 8 deg (W × H). 

Three-dimensional scene

The 3D scene was set up in the right compartment of the lighting booth in a small chamber (height 35 cm, width 40 cm, and depth 40 cm); the walls and floor were painted with matte white paint. Two sides of the chamber were open, the front side to allow observation of the scene and the top for illumination purposes. There were three objects inside the chamber; on the left side in the foreground there was a white sphere (diameter 7 cm), in the center a 20.2-cm-tall cone (base diameter 10 cm) and on the right side in the middle a 12 × 12 × 6 cm sized box. The back of the chamber contained a black palette containing a mixture of colored swatches from all three color groups but none of the target colors. Figure 5 shows a schematic of the chamber and a close-up photograph. The cone and the box were made from the paper of the target color and extended over a visual angle of approximately 3 to 7 deg. Observers were seated 1.4 m away from the front of the chamber, hence the 3D scene subtended a visual angle of approximately 16 × 14 deg (W × H). 

Figure 5

 

(A) Schematic of the 3D scene setup, top view. (B) Front view of the layout. All dimensions are in centimeters. (C) A close-up photograph of the 3D scene.

Experiment 1: Color memory screening

As outlined in the Introduction section, color memory is essential for color constancy when a successive matching paradigm is used. Therefore, all observers had to pass a color memory task first to ensure that they had adequate and roughly equal color memory. Otherwise, poor color constancy performance could be entirely due to bad color memory and large variance in the data could produce a Type II error. 

Observers

A total of 47 observers (29 females and 18 males, who were between 19 and 47 years old) took part in the color memory test. All had normal visual acuity or were corrected to normal and were color-normal as assessed by the Farnsworth-Munsell 100 Hue test (an error score of less than 51). 

Seven observers learned the target colors in a 2D as well as a 3D setup. The other 40 observers were split into two groups where one group learned the target colors in the 2D, the other group in the 3D setup. 

Procedure

Observers adapted initially for 2 min to illuminant D1 by looking at the white palette. Then, a target color was presented either on a learning palette (the experimenter pointed briefly with a finger in a white glove to the target) in the palette compartment or in the 3D scene by the cone and box. Observers had 20 s to learn the color. After the 20 s the learning palette or 3D scene was taken from view and a test palette was presented under the same illuminant D1. Observers were instructed to select the swatch they thought had been cut from the same piece of paper as the swatch (or the objects) they had focused on before. No time limit was set for selection, but all observers made their choice within 2 to 15 s; no feedback was given. 

The task consisted of 18 trials in which each target color was presented three times and the minimum number of correct selections needed to take part in further experiments was 8 out of 18. 

Results

Nineteen observers were not able to correctly match the set minimum of 8 out of 18, leaving 28 participants to continue with the study. These matched on average 9 to 10 color swatches correctly. The group that learned the target colors in the 2D setup selected the correct swatch in 52.4% ( SD ± 0.1%) of the cases and the 3D group in 52.0% ( SD ± 0.1%). 

Summary

The color memory test was conducted to test observers color memory and to select those with a satisfactory memory to perform subsequent color constancy tasks. Therefore, the 19 observers who did not fulfill the set target were excluded from any further tasks. 

Approximately 40% of the observers failed to reach the set target, which indicates that the color memory test was by no means easy. The task was designed to be challenging but not impossible for the observers. The task was easy enough to achieve results that would enable us to detect any deterioration for more difficult tasks while avoiding ceiling effects. 

The performance in the color memory test may seem low, but it has to be considered that the theoretical chance level was 6.25% (also see Experiment 2, Results). 

We found no difference between participants' ability to remember colors learnt in a 2D or 3D environment. 

Experiment 2: Color constancy in 2D and 3D under typical and atypical illumination changes

Two factors were investigated: (a) learning environment, i.e., whether learning a color as part of a 3D scenario, instead of 2D one, would lead to higher color constancy performance, and (b) illuminant change, i.e., whether the nature of an illuminant change may influence performance. The factor learning environment was a within subject factor, thus, each observer learned a target color in the 3D and 2D setups, while illuminant change was a between-subject factor. Seven observers were assigned to each of the four illuminant change conditions (D1 to Tun, Tun to D1, D2 to Lily, and Lily to D2). D1 to Tun and Tun to D1 were classified as a typical illuminant change, whereas D2 to Lily and Lily to D2 were classified as atypical (for definition, see Introduction section). Each of the 28 observers completed two tasks (2D and 3D) for the assigned illuminant change condition for all six target colors (repeated three times), resulting in 36 trials per observer (approximately 4 sessions of 45 min each). 

Procedure

The procedure for the color constancy experiment was identical to the one described in the color memory test. A trial started with a 2-min adaptation period to the prevailing illuminant by viewing the white palette. Observers learned the target color either as a 2D swatch on a palette or as objects made of the target colors and embedded into the 3D setup, depending on the learning environment (counterbalanced across observers). After a 20-s learning period, the illuminant changed and observers adapted for 2 min to the new illuminant by viewing the white palette. Then, a test palette was shown and observers made their selection. Figure 6 shows a trial sequence (2D and 3D). 

Figure 6

 

Trial sequence and timings for the color constancy task. The target color was presented as a swatch on a learning palette (2D) or as the cone and box in the scene (3D).

Under everyday conditions, observers are generally well adapted to the prevailing illumination when they attempt a color constancy task. That adaptation plays an important role not only for color constancy but for color appearance in general has already been established (e.g., Jameson & Hurvich, 1989; Webster, 1996; Wyszecki, 1986). Several studies have shown that chromatic adaptation to the spatial mean of a scene is almost complete (up to approximately 90–95%) after about 1 to 2 min and that color appearance remains stable after this (Fairchild & Lennie, 1992; Fairchild & Reniff, 1995; Hunt, 1950). 

In a pilot study, two adaptation periods for the color constancy task had been compared (1 s and 2 min). Performance for the 1-s adaptation period was 18.6%, which increased to 31% for the 2-min adaptation period. Observers described the task with 1-s adaptation as impossible to do. Therefore, a 2-min adaptation period was chosen. 

Results

The results are presented as hit rates. By a hit we understand the selection of the correct swatch and the hit rate is how often the correct swatch was selected, expressed as a percentage. 

Table 2 provides a summary of the hit rates for the two learning environments and four illumination change conditions. The data have also been separated for the three color groups (blue, red, and yellow). 

Table 2

 

Summary of hit rates for each color group and illuminant change condition, separated whether a target color was learned in 2D or in 3D. Highlighted columns indicate average hit rates across all color groups.

Illuminant change condition	2D				3D
	Blue	Red	Yellow	∅	Blue	Red	Yellow	∅
D1 to Tun	9.50	28.4	28.5	22.1	16.5	52.0	47.5	38.7
Tun to D1	42.9	11.9	23.8	26.2	23.8	19.0	45.2	29.3
D2 to Lily	28.6	59.5	33.3	40.5	45.2	40.5	28.6	38.1
Lily to D2	11.9	26.2	21.4	19.8	19.0	42.9	33.3	31.7
Mean				27.2				34.5

A three-way mixed ANOVA with two within-subject factors and one between-subject factor was conducted to analyze the data. The within-subject factor learning environment had two levels, 2D and 3D. The other factor was color group and had three levels (blue, red, and yellow). The between-subject factor illuminant change had four levels: D1 to Tun, Tun to D1, D2 to Lily, and Lily to D2. 

There was a significant main effect of learning environment, F(1,24) = 5.71, p = 0.025. Observers' performance improved significantly when target colors were learned as part of a 3D setup (2D, 27.2% and 3D, 34.5%; see Figure 7). Another significant main effect was found for color group, F(2,48) = 3.19, p = 0.05, with 24.7% for the blue, 35.1% for the red, and 32.7% for the yellow color group. However, there was no significant effect for illuminant change. There was neither a significant interaction between learning environment and illuminant change, nor learning environment and color group. However, there was a significant interaction between color group and illuminant change, F(6,48) = 3.69, p = 0.004. 

Figure 7

 

Significant main effect of learning environment ( α = 5%). Error bars indicate ±1 SE.

A further three-way mixed ANOVA with two within-subject factors and one between-subject factor was conducted to explicitly study whether the nature of an illuminant change may influence the results, i.e., typical vs. atypical illuminant changes. The within-subject factors were the same as in the mixed ANOVA reported above: learning environment and color group. For the between-subject factor, the data was regrouped into two illuminant changes (typical and atypical). The data set of the typical illuminant change consisted of D1 to Tun and Tun to D1, whereas the atypical illuminant change contained the data of D2 to Lily and Lily to D2. 

Neither color group nor illuminant change were significant factors. As before a main effect of learning environment, F(1,26) = 5.10, p = 0.033, was found and none of the interactions were significant. 

Strictly speaking, the chance level of selecting the correct swatch was 1/16 (6.25%), but in practice, numerous alternative swatches on the test palettes were never selected. The selected alternatives varied across observers, target colors, and illuminant change conditions. For example, for target color R10 in the illuminant change condition D2 to Lily (2D) observers considered only two alternatives besides the correct swatch. Thus, the selection effectively took place between three swatches and not 16. In illuminant change condition Tun to D1 (3D) observers made their selection out of eight swatches instead of out of 16. A chance level of 6.25% was solely a theoretical value. However, for all experimental conditions observers performed well above this level. 

Complete individual swatch selections for all observers and illuminant change conditions are available as supplementary material. 

Control experiment

The accuracy with which colors can be remembered is controversial. Earlier studies have investigated color memory for different delay periods. Nilsson and Nelson (1981) studied color memory of monochromatic stimuli for relatively short time delays, ranging from 100 ms up to 24 s. They reported that color had been remembered rather accurately over the delay periods that they had tested, whereas Francis and Irwin (1998) found a deterioration of memory for delays of 1 s and 10 s. Studies reporting considerably longer delay periods are rare. Perez-Carpinell et al. (1998) tested observers' color memory for delays of 15 s, 15 min, and 24 h. They suggested that color memory deteriorates over time and that colors are not remembered equally. 

The accuracy of color memory cannot be predicted as it varies considerably between observers and colors and as it is task dependent. Therefore, a control experiment was conducted to establish the degree of deterioration of color memory that could happen over the 2-min adaptation phase in the color constancy task. 

Procedure

The procedure in the control experiment was identical to the one described in the color memory test with the only difference that selection was performed 2 min after the learning instead of immediately after. 

Seven observers (of the 28) took part in the control experiment. All of them completed two tasks; learning the target color in 2D as well as in 3D (counterbalanced across observers). 

Results

A two-way repeated-measures ANOVA was conducted to analyze the data of the control experiment together with the color memory test. The two factors were delay, which had two levels (1 s and 2 min), and color group, which had three. 

There was no significant main effect of delay. The mean performance of the color memory test (delay 1 s) was 54.5% ( SD ± 0.2%) and dropped to 52.8% ( SD ± 0.3%) when the delay was 2 min. Thus an increased delay of 2 min before selecting a match did not produce a deterioration of color memory in our task. A significant main effect of color group again, F(2,26) = 15.43, p < 0.01, was found. Pairwise comparison (Bonferroni corrected) revealed a significant difference between the blue and red color groups as well as between the red and yellow groups ( p < 0.01 and p = 0.023, respectively). The interaction between delay and color group was not significant. 

Discussion

Studies of color constancy have often investigated the contribution of specific cues and mechanisms. Whether these studies have been carried out with 2D or 3D stimuli, they have shown that illuminant cues in a scene are essential for the visual system to be color constant. 

An important source of illuminant cues is three-dimensionality and several studies have incorporated this source in different ways. One attempt to present cue-rich stimuli was to present photographs of the real world on a monitor (Amano et al., 2006; Amano, Uchikawa, & Kuriki, 2002; Foster, Amano, & Nascimento, 2006; Nascimento, Ferreira, & Foster, 2002). The scenes were easily recognized by observers and looked familiar; however, the initial three-dimensionality was reduced when reproducing the scene on a flat monitor. A different approach was the presentation of real 3D scenarios (Brainard, 1998; de Almeida et al., 2004; Ling & Hurlbert, 2006). Although these setups were more abstract in content (they consisted mainly of geometrical volumes and plain surfaces), the actual three-dimensionality could be experienced by the observer. 

If illuminant cues play a major role, then color constancy would be expected to improve when colors are learned in a richer environment. It is generally accepted that 3D scenes and objects provide a wider range of illuminant cues than a 2D setup, but no one has explicitly tested if color constancy improves when the color is learned in a 3D scene. 

In this study, observers' color constancy performance was directly tested for 2D and 3D scenes and objects. It was found that learning a color in 3D led to a higher level of color constancy than when learning took place in the 2D setup. Individual contributions of specific cues (highlights, mutual illumination, etc.) were not explicitly tested, although it was evident that the 3D scene contained more illuminant cues than the 2D palette setup. 

The target color was presented either by the cone and the box in the 3D scene or by a swatch on a 2D palette. Thus for the 3D learning environment not only the scene is 3D but also the object from which the test color is learned. We did not want to simply use a 2D test patch embedded in a 3D environment (this would have been similar to the setup used in Kraft & Brainard, 1999) but rather to display the test color as a 3D object. In previous work, de Almeida, Fiadeiro, Nascimento, and Foster (2002) showed that observers were equally good at detecting a material change in real 3D scenes as in their 2D planar projections. The authors go on to interpret this as evidence that 3D cues play a limited role in surface color perception. In our study, we ensured that the test object provided cues that were fully consistent with a 3D object in a 3D scene including shading, shadows, and mutual illumination. If we had not found a difference between our two experimental setups, then we would be sure that dimensionality (2D vs. 3D) had no effect on color constancy. However, that was not the case. We have established that, for our experimental conditions, color constancy improves when the target color is learned as part of a 3D object. 

In total, the cone and the box subtended an area that was approximately nine times larger than the area of a swatch. It could be argued that the larger area of the target color in the 3D scene might have led to higher levels of color constancy. According to the available literature, the accuracy of color memory stabilizes for stimulus sizes equal or larger than 1 deg (Abramov & Gordon, 2005; Nerger, Volbrecht, & Ayde, 1995). Considering also that the rod-free area of the fovea extends over an angle of 1.7 deg (Wandell, 1995), it can be concluded that the difference in stimulus size was not responsible for our results because the swatches, the smallest stimuli, already subtended a visual angle of 1.8 deg. On a similar note, it has been shown that color constancy increases with size of adaptation field (Hansen, Walter, & Gegenfurtner, 2007). Hansen et al. (2007) compared a large adapting field size (64 × 45 deg) with a smaller one (10 × 8 deg), which differed in area by a factor of 36. Our scenes (2D: 11 × 8 deg and 3D: 16 × 14 deg) were similar in size to the smaller adapting field size and the difference in area between our two scenes (2D and 3D) corresponds to a factor of 2.54. In our experiments, adaptation always took place by viewing the white palette (11 × 8 deg), we believe that this combined with the negligible change in field size between 2D and 3D rules out the possibility that the 3D advantage was due to an increase in visual field size. 

It has to be emphasized that both our 3D and 2D setups consisted exclusively of real surfaces. Consequently, only a limited range of surface colors could be presented for matching and the matching had to be made by selection. Achromatic and chromatic settings are popular whenever observers are asked to judge surface appearance, because settings are continuously adjustable over a wide range of possibilities. The main disadvantage is that this kind of matching is not natural. In everyday life, we see a particular surface color and we have to decide whether it is the same color that we remember. In such a situation, a decision has to be made and we cannot adjust a surface color until it matches our memory. In this study, a more natural setting was replicated by providing a fixed set of alternatives, including the correct answer. Observers had merely to recognize the correct swatch. 

In the studies by Brainard (1998) and Kraft and Brainard (1999), observers adjusted the appearance of the test patch by varying its illumination, which was independent of the scene illumination. In the experiment described by de Almeida et al. (2004), observers did not match two surfaces, but rather they adjusted the appearance of a test object that was projected into the scenario. Furthermore, in the study by Ling and Hurlbert (2006), all so-called surface colors of the 3D dome and the flat color patches were produced entirely by illuminating white surfaces. This potentially means that observers find these settings artificial and might make an appearance match instead of a surface match. In our setup, this confusion was not possible as we exclusively used real objects and swatches. 

Color groups

Throughout our analyses the factor color group was almost always significant and had in several occasions interactions with other factors. This indicates that the three color groups had different levels of difficulty. Each color group consisted of 16 color swatches, but the perceptual difference between the swatches across the three color groups was not identical. This can be seen in Figure 3 where the chromaticity of all 48 swatches under each of the four illuminants is shown in an approximately perceptually uniform color space. Generally, the color swatches of the blue group lay perceptually closest together and the red color swatches lay furthest apart from each other, while the yellow group was the intermediate group. Although under each illuminant, the colors shifted with respect to their absolute position in the color space they maintained more or less their relative distance to each other under all four illuminants. This is also borne out by Δ E calculations (Wyszecki & Stiles, 2000). The perceptually closest color swatches to the red target colors were clearly different and therefore easily distinguishable. Hence, we expected that observers would find the detection of the red target colors easiest and the blue hardest. The results of the color memory test confirmed these predictions. In both tasks the red color group had the highest hit rates and the blue color group the lowest (see Table 2). The results for the yellow color group always lay between these two. Surprisingly, this same pattern of results was not found in the color constancy experiment. While the proportions of hit rates for the color groups followed this pattern in the illuminant change condition D1 to Tun and Lily to D2, they did not for Tun to D1 and D2 to Lily. For the illuminant change condition Tun to D1, the performance for the blue and yellow color groups was similar, whereas the red color group was much more difficult. For the illuminant change condition D2 to Lily, the red color group was again easiest to match followed by the blue and yellow groups. Albeit the differences within the three color groups in the test palettes, we were able to show that, overall, real 3D environments produced a significant improvement in color constancy performance. 

Typical vs. atypical illuminant changes

The question of whether the visual system compensates more effectively for illuminant changes that are part of daily life (typical changes) than for rare (atypical) changes has already been addressed in earlier studies (e.g., Brainard, 1998; Daugirdiene et al., 2006; Delahunt & Brainard, 2004b; Hansen et al., 2007; Ruttiger et al., 2001; Schultz et al., 2006). In none of these studies have researches found an improved level of color constancy when testing under a typical illuminant change in comparison to an atypical one. Our results are in line with these previous findings. No significant effect of illuminant change was found in the color constancy experiment, nor was there an effect when the four illuminant change conditions were grouped into typical and atypical illuminant change conditions. The visual system does not seem to compensate more effectively for frequently experienced illuminant changes. 

Color constancy index

The level of color constancy is commonly expressed by a color constancy index. To compute a color constancy index numerous studies have applied the Brunswik ratio or modified versions of it (see Arend, Reeves, Schirillo, & Goldstein, 1991; Brainard, Brunt, & Speigle, 1997; Daugirdiene et al., 2006; Murray, Daugirdiene, Stanikunas, Vaitkevicius, & Kulikowski, 2006; Troost & de Weert, 1991). These ratios consider the perceptual shift of a surface color as well as the physical shift that occurs due to an illuminant change. If a visual system is not color constant at all, the perceptual shift is therefore identical to the physical shift. In this case, an observer would perform a chromaticity match. If a visual system is perfectly color constant, it will compensate perfectly for the illuminant shift. There would be no perceptual shift and the observers would have performed a surface match. 

Under the present experimental conditions, the Brunswik ratio or any modified versions were not applicable for different reasons:

1.  
   under no circumstances could the perceptual shift equal the physical shift, i.e., it was impossible to perform a chromaticity match, indicative of a complete lack of color constancy, and
2.  
   the color constancy index calculated for any patch in the test palette would have a high value.

A target color was learned under one illuminant and when it was presented under a different one for matching, the alternative swatches on the test palettes were perceptually and colorimetrically close to the target color. Hence, it was impossible to choose a swatch that was very different from the target color or even to select a swatch with the same chromaticity as the target color under the first illuminant, as such patches were not available on the palette. Regardless which alternative swatches had been selected, the color constancy ratio would have indicated almost perfect color constancy. A Brunswik ratio was also not applied because it does not take into account the effect of color memory. Recently, Ling and Hurlbert ( 2008) introduced a new color constancy index, which incorporates the effect of color memory as well as the physical shift of a test stimulus. They separated the perceptual shift into two components, a memory and a pure constancy shift. To compute the color constancy index, the memory shift is subtracted from the perceptual shift. Therefore, the index accounts only for the perceptual shift caused by the illuminant change and the physical shift of the test stimulus.

Usually, researchers limit the matches in a color constancy task to a line in color space, which is equivalent to the illuminant change direction. By doing this, they avoid dealing with different directions in a formula that is entirely based on distances (Brunswick ratio). Because our alternatives are organized as a cloud, distances alone are not indicative enough. The results of our study have been presented in the form of hit rates, where only the correct instances are taken into account. The reported hit rates, especially of the color constancy tasks, may appear low (on average about 30%), but it must be remembered that the overall color memory hit rate was approximately 50%; thus, performance dropped by only 20%. Therefore, the color constancy performance must be set into context with the memory performance. In order to compute a color constancy index ( CI _i), the color constancy performance of each observer ( i) was normalized by their individual color memory performance from the color memory test, i.e.,  

${CI}_i=\frac{H{R}_{Ci}}{H{R}_{Mi}}.$

(1)

HR _Ci is the color constancy hit rate and HR _Mi is the color memory hit rate of an observer. This index is 1 if the color constancy hit rate is equal to the memory hit rate, indicating that the color constancy performance was not compromised by the observers' memory. Such index drops to zero if the hit rate for a color group was zero irrespective of the level of color memory that was achieved in the color memory test. Applying this normalization for each observer and for each color group yields an overall mean level of color constancy of 0.58 and 0.79 for 2D and 3D, respectively. Note that this index never assumes the upper limit of color constancy to be at 100% unlike the Brunswick ratio. Here, color constancy is normalized only by color memory. This normalization allows comparing the results of this study with those from other studies that used 3D stimuli. The levels of color constancy reported in studies using real surfaces were between 0.11 and 0.83 (Kraft & Brainard, 1999), 0.81 and 0.93 (de Almeida et al., 2004), and 0.61 and 0.84 (Ling & Hurlbert, 2006). Therefore, the level of color constancy achieved in this study lies in the same range and is comparable to these earlier studies despite a different experimental approach and the computation of an alternative color constancy index. 

Summary

In this study, we have worked exclusively with real surfaces and illuminants and used a natural selection task to explore human color constancy under various illuminant changes. Our experimental design has allowed us to take into consideration our observers' individual color memory and to test explicitly if there is an advantage in learning the color in 3D over 2D. We found that the additional cues available when the color was represented by an object in a rich 3D scene lead to higher color constancy levels than in the 2D case and that this improvement was independent of the nature of the illuminant change. 

Appendix A

Natural color system

The Natural Color System (NCS) was introduced by the Scandinavian Color Institute AB (Scandinavian Colour Institute: Natural Colour System, 2004) as a perceptual uniform color system. The NCS System is based on the six elementary colors: red, green, blue, yellow, black, and white. Their positions constitute a three-dimensional volume that has the shape of a double cone and in which red, green, blue, and yellow lie in the same plane. Figure A1 shows a diagram of the NCS color space. 

Figure A1

 

The NCS color space is set up by six elementary colors. Modified from Scandinavian Colour Institute: Natural Colour System (2004).

Color samples of this space are defined by three characteristics: hue, blackness, and chromaticness. Hue refers to the actual color. Furthermore, each color sample is defined by the perceived quantity of black in the color in comparison with pure black; this proportion is expressed by the blackness value. The term chromaticness is used to describe the saturation of a color sample. 

All possible hues of the NCS System lie on a color circle, which results from a horizontal cut through the middle of the color space (see Figure A2). 

Figure A2

 

In the NCS color circle colors change progressively from yellow to red, from red to blue, and so on in steps of 10 perceptual units. The yellow color group covered the range from G20Y to Y50R, the red color group covered the range from Y60R to R90B, and the blue color group covered the range from R90B to G20Y. Note that there was an overlap in hue between the blue and yellow, and the blue and red color groups. Modified from Scandinavian Colour Institute: Natural Colour System (2004).

Along the central vertical axis of the NCS color space lie the gray shades, reaching from white at the top to black at the bottom. A vertical cut through the color space produces a so-called color triangle (see Figure A3). There is a color triangle for each hue. All color samples of a triangle vary in blackness and chromaticness. 

Figure A3

 

NCS color triangle for the single hue Y90R. W stands for white, S for black, and C for chromaticness. The chromaticness increases from the left to the right. The least saturated color samples are next to the achromatic line. Blackness increases starting from the line between W and C and moving toward S (Scandinavian Colour Institute: Natural Colour System, 2004).

The NCS color notation is based on the scheme that every given color can be described as a mixture of two or more of the six elementary colors. The exact notation will be explained by the following example: S2030–R70B. S indicates that this is a standardized NCS color sample of the second edition. The number 2030 is called nuance and is a combination of blackness and chromaticness. Twenty indicates that the degree of resemblance to black is 20% and that the chromaticness is 30%. The hue is described by the last term of the notation. R70B means that it is a red (30%) with 70% blue. In other words, this color appears as a light bluish purple. 

Hues from all over the NCS color circle were chosen. The overriding requirement was a balance between similarly colored alternatives but at the same time discriminable from each other. This limited the number of usable papers. As a compromise between making swatches similarly saturated but having enough alternatives available all color swatches had the same blackness value of 10 and three different chromaticness values of 30, 40, or 50. Furthermore, we wanted to offer enough alternatives (thereby not making the task too easy), allow a rotationally invariant layout of the palettes (which required a square number of samples), and show reasonably large swatches. The chosen color swatches were divided into three color groups called “Blue”, “Red”, and “Yellow”, each containing equal number of samples. In Figure A2, we have indicated the three color groups into which the hue circle was divided. In the blue color group, colors range from chartreuse to blue, in the red group from blue via purple to orange and in the yellow group from orange to chartreuse. The exact notations of the experimental colors used are listed in Table A1. 

Table A1

 

NCS color notation of the color papers that were used, target colors are highlighted.

Label	Blue color group	Red color group	Yellow color group
1	S1040 B	S1030 Y60R	S1030 G50Y
2	S1040 G	S1030 Y90R	S1030 G70Y
3	S1040 B30G	S1030 R10B	S1030 Y
4	S1030 B40G	S1030 R50B	S1030 Y20R
5	S1040 B70G	S1030 R70B	S1030 Y40R
6	S1040 B90G	S1040 Y60R	S1040 G20Y
7	S1050 G20Y	S1040 Y80R	S1040 G40Y
8	S1040 G10Y	S1040 R	S1040 G60Y
9	S1040 B40G	S1040 R20B	S1040 G80Y
10	S1030 B	S1040 R40B	S1040 Y
11	S1050 G	S1040 R60B	S1040 Y20R
12	S1050 B	S1040 R80B	S1040 Y30R
13	S1040 R90B	S1040 R90B	S1040 Y50R
14	S1040 B20G	S1050 Y70R	S1050 G30Y
15	S1050 R90B	S1050 R	S1050 G90Y
16	S1040 B10G	S1050 R30B	S1050 Y

Acknowledgments

We wish to thank our summer students Diane Booth (supported by the Nuffield Foundation) and Elvira Supuk for their contributions and help with the experiments. MH is supported by a studentship from the University of Bradford (Optometry) and the Universidad Católica de Valparaíso/Chile. 

Commercial relationships: none. 

Corresponding author: Monika Hedrich. 

Email: [e-mail]. 

Address: Bradford School of Optometry and Vision Science, University of Bradford, Bradford BD7 1DP, UK.