Effects of spatial and temporal context on color categories and color constancy

Abstract

Color constancy is the ability to assign a constant color to an object independent of changes in illumination. Color constancy is achieved by taking context information into account. Previous approaches that have used matching paradigms to quantify color constancy found degrees of constancy between 20% and 80%. Here, we studied color constancy in a color-naming task under different conditions of surround illumination and patch size. Observers categorized more than 400 patches for each illumination condition. This allows one to overcome inherent limitations in color naming and to study the changes in color categories under illumination changes. When small central test patches with a full context illumination were categorized, observers followed the illumination shift almost completely, showing a high degree of constancy (99%). Reducing the available context information or increasing the patch size decreased the degree of constancy to about 50%. Moderate degrees of constancy (66%) occurred even when the test patches were never viewed simultaneously but only in temporal alternation with the illumination. Boundaries between color categories were largely stable within and across observers under neutral illumination. Under changing illumination, there were small but systematic variations in the color category boundaries. Color category boundaries tended to rotate away from the illumination color. This variation was largest under full context conditions where highest degrees of color constancy were obtained.

Introduction

Color constancy is the ability to assign a constant color to objects independent of changes in illumination. If we look at a blue object under a blue sky, the object reflects mainly short wavelengths. The same object, when illuminated by tungsten light, reflects more light of longer wavelengths. Despite such gross reflection changes, the object consistently appears blue (Jameson, 1985). The remarkable feat of the visual system is to somehow “discount the illuminant” (von Helmholtz, 1867) and to estimate the surface reflectance as an invariant object property. Color, as a largely illumination-invariant internal estimate of surface reflectance, then helps to recognize and distinguish objects. 

How can this ability of the visual system be achieved? The light reflected from an object depends both on the spectral reflectance properties of the surface and on the spectral distribution of the illumination. To disentangle the effects of illumination and surface reflectance, the visual system needs more than one source of information. Many potential cues can be used, such as local comparison between different surfaces (Foster & Nascimento, 1994; Land & McCann, 1971), global average of a scene (Buchsbaum, 1980), knowledge about the three-dimensional arrangement of a scene (Bloj, Kersten, & Hurlbert, 1999; Maloney, 1999), or knowledge about the typical color of an object (Bruner, Postman, & Rodrigues, 1951; Duncker, 1939; Hansen, Olkkonen, Walter, & Gegenfurtner, 2006). 

In natural scenes, many potential processes interact to achieve color constancy. However, these processes are not perfect, and information about the illumination is not completely discarded. We would always name the wallpaper as white, yet we are able to notice that it is a reddish white at sunset, a neutral white at noon, and a yellowish white under tungsten illumination. When referring to color tones and shades, constancy seems to work imperfectly. Measuring color constancy with a matching task reached high degrees of color constancy, but never complete constancy, even with nearly natural scenes (Kraft & Brainard, 1999). 

In scrutinizing color constancy, it may be helpful to consider the major processing stages in color vision. Four stages can be identified. Color vision starts with the absorption of electromagnetic radiation by three types of cone photoreceptors. At the second stage, retinal ganglion cells combine the cone signals into three cone-opponent channels, one achromatic channel and two chromatic channels. In the cortex, multiple mechanisms exist, with tuning curves that narrow as the processing proceeds from early visual cortex to higher cortical areas (Gegenfurtner, 2003). At the highest stage, only a few basic color categories exist (Komatsu, 1998). We find it interesting that we have perceptual access to multiple levels—we can make fine discrimination judgments within a category as well as categorical judgments. 

Previous studies of color constancy that have used a matching paradigm focused on the ability to distinguish subtle variations in chromaticity (Bäuml, 2001; Brainard, 1998; Kraft & Brainard, 1999; Yang & Maloney, 2001). Another important ability is to make categorical color judgments, for example, to decide whether a fruit is ripe or unripe or whether it is red or green. Color constancy can therefore help to robustly distinguish different color categories. Measuring color constancy based on color categories complements matching studies to characterize human color constancy performance. Here, we study color constancy under different illuminations and surround conditions using a color-naming task. 

Color naming is perhaps the most direct approach to measure color constancy (Foster, 2003; Smithson, 2005). The principal drawback of color naming is that there are many more distinguishable colors than color names in the common vocabulary. Therefore, color naming seems to be an inherently coarse method. This limitation can be overcome by using a large number of patches to be characterized and to infer color constancy from the boundaries between color categories. 

Smithson and Zaidi (2004) used a naming task (red vs. green and blue vs. yellow) to determine the red–green and yellow–blue classification boundaries for patches presented on different backgrounds and under two illumination conditions. Observers demonstrated high levels of constancy, even when the mean chromaticity of the variegated background was biased. The present study extends the said method by using more color categories (red, orange, yellow, green, turquoise, blue, purple, and gray). Compared to a coarse, color-opponent red–green/blue–yellow categorization, this allows one to investigate the stability of basic color categories at a nonopponent level under changing illumination, involving higher order chromatic processing. 

We asked observers to categorize more than 400 color patches whose colors were drawn from an isoluminant plane in color space. The resulting categories radially divide the color plane into arc segments from which the center of color space can be determined. The position of this center, relative to the illumination, indicates the degree of color constancy present in the observer: Under 100% constancy, the center follows the illumination and coincides with the chromaticity coordinates of the illuminating color, whereas with 0% constancy, the center would not shift at all. We varied the amount of information about the illumination available to the observers. When full information was available, we found that observers follow the illumination shift almost completely, showing almost perfect constancy. We conclude that color constancy can be almost perfect in human observers. 

Methods

Observers viewed a CRT monitor in a small experimental chamber. The illumination of the chamber and the monitor image could be independently varied. Observers had to name the colors of homogeneous colored patches presented on the monitor. In different experiments, we varied the surround illumination and the size of the patches. From the naming of 417 patches in each condition, we computed the neutral point of the color space. The shift of this point in the different illumination conditions characterizes color constancy performance. The methods were similar to those described in detail elsewhere (Rinner & Gegenfurtner, 2000, 2002). 

Experimental chamber

The experiments were run in a small experimental chamber (2.2 m high × 2.4 m wide × 1.3 m deep), as sketched in Figure 1a. Subjects looked into the chamber through a 2.2-m-high and 1-m-wide opening in the back wall. Computer-controlled fluorescent lamps (Osram L36W/66 in red, green, and blue) illuminated about 45° × 64° of the visual field. Through a 10° × 8° opening in the front wall, the observer looked at a CRT monitor on which the stimuli and the background were displayed. The monitor was placed in a black felt tunnel behind the wall such that no light from the fluorescent lamps could reach it. In some experiments, another black tunnel was placed between the observer and the monitor to limit the observer's field of view to the monitor. In all experiments where the wall was visible, the monitor background had the same chromaticity as the light reflected from the wall. In the following, the term illumination refers to the wall and to the simulated illumination of the monitor background. 

Figure 1

 

Apparatus and experiments. (a) Top view of the experimental chamber used in all experiments. (b) Sketch of the six different experiments. The surround illumination I, which was kept constant during the experiment, is sketched by the large rectangle. The monitor between and during presentation of the test patch T is sketched by the two inner rectangles with curved corners. The experiments differed in the size of the test patch T that had to be categorized and the part of the scene that was illuminated by the illumination color I, resulting in different combinations of local spatial contrast and temporal contrast between the test patch and the illumination. In Experiments 1, 2, and 6, a small test patch was used; in Experiments 3–5, the test patch subtended the whole monitor. In Experiments 2 and 5, the observers viewed only the monitor through a tunnel, whereas in all other experiments, the wall was illuminated with illumination color I during the whole experiment.

Stimuli and illumination

All stimuli and illuminations were specified in the isoluminant plane of the DKL color space (Derrington, Krauskopf, & Lennie, 1984; Krauskopf, 1999; Krauskopf, Williams, & Heeley, 1982; MacLeod & Boynton, 1979). Coordinates are defined relative to a neutral gray point with a luminance of 32 cd/m² and chromaticity coordinates x = 0.32, y = 0.36 in a revised CIE color space (Judd, 1951). The DKL isoluminant plane is spanned by two axes: an L–M axis along which only L and M cone modulations vary at a constant sum and an orthogonal (L + M)–S axis along which only S cone excitation varies. Lights along the L–M axis vary between reddish and bluish green; lights along the (L + M)–S axis vary between greenish yellow and purple. All lights in the isoluminant plane have the same luminance as defined by the V(λ) photopic luminosity function (Judd, 1951; Wyszecki & Stiles, 1982). The maximum available contrast along each half-axis was normalized to unity. In each experiment, five different illuminations were used: neutral gray and four chromatic illuminations at half of the maximum (0.5) along each of the four isoluminant DKL half-axes. The chromaticities of the colored patches to be categorized in each illumination condition were specified on a 21 × 21 grid. The grid covered the whole isoluminant plane with a spacing of 0.1 units. Due to limitations imposed by the monitor gamut, only 417 colors (instead of 441 [21 × 21] colors) were presented. The colored patch either was a circular disk of 2° (Experiments 1, 2, and 6) or covered the whole monitor (10° × 8°, Experiments 3–5). 

Procedure

In each condition, observers had to categorize 417 colored patches presented on the monitor in random sequence as belonging to one of eight categories (red, orange, yellow, green, turquoise, blue, purple, and gray) by pressing a corresponding button. The color names were given in German (rot, orange, gelb, grün, türkis, blau, violett, and grau). Subjects were familiarized with the experiment and the available colors in a training session. During the experiments, the head was fixated with a chin rest. Each run started with a 1-min adaptation period to the illumination, after which about 90% of adaptation is complete (Rinner & Gegenfurtner, 2000). After pressing a key, a colored patch appeared on the screen for 500 ms. Between presentations, only the surround illumination was present. Subjects were asked to assign the patch to the most adequate color category. No time limit was imposed. Categorizing the patch by pressing the corresponding button initiated the presentation of the next patch. A total of 417 different patches were presented in random order in each run, which was usually completed in about 15 min. The different illumination conditions were run in random order in separate sessions with a 5-min break between sessions. 

Experiments

Six different experiments were run, which differed in the amount of illumination information available to the observer. The amount of illumination information was controlled by varying the peripheral surround, the central viewing field, and the information that could be acquired at one time ( Figure 1b). Each experiment was run with the five different illuminations. 

In the first experiment with the most available information, the observer looked into the illuminated room and a small patch was presented on the monitor on a background with the same color as the illumination. The second experiment was identical to the first one, except that the viewing field was restricted to the monitor screen by a tunnel to study the possible influence of the remote full-field surround. In Experiments 3–5, the test patch covered the whole screen. Apart from the size of the test patch, Experiment 3 was identical to Experiment 1. In Experiment 3, the spatial influence of the illumination on the test patch was restricted to the periphery, whereas the temporal contrast between test patch and illumination was present on the whole monitor. Temporal contrast occurs when the test patch is presented and then replaced by the illumination color. In Experiment 4, we eliminated the temporal contrast between the test patch and the illumination color by setting the monitor to black after the 500-ms presentation of the test patch. In Experiment 5, the spatial component was eliminated by restricting the viewing field to the monitor by a tunnel. In Experiment 6, a small central test patch was presented on a black background, and influences of the illumination were possible only from the peripheral surround. The contextual differences in the experiments result in different amounts of spatial contrast and temporal contrast between test patch and surround illumination available to the observer. Local spatial contrast between test patch and illumination was present in all experiments except for Experiments 5 and 6, and temporal contrast occurs in all experiments except for Experiments 4 and 6. In Experiments 1 and 2, there were both spatial contrast and temporal contrast between the small test patch and the adjacent illumination color. In Experiment 6, no temporal contrast and no local spatial contrast between adjacent surfaces was present. In this experiment, information about the illumination could only be obtained from the remote spatial surround. In Experiment 3, there were, again, both spatial contrast and temporal contrast, but now for a large test patch. In Experiment 4, only spatial contrast was present, whereas in Experiment 5, only temporal contrast was present. 

Observers

Eleven observers participated in the study: three males and eight females aged between 20 and 44 years (average, 28 years); all of them were naïve except for author S.W. All subjects had normal color vision and normal or corrected-to-normal visual acuity. Due to limited availability of the observers, only eight observers participated in Experiments 4 and 6. 

Data analysis

Based on the categorization of all 417 colored patches in each trial, one can define a neutral point in color space. We used two different definitions of the neutral point. The neutral point was defined either as the achromatic point, which is the average of all color patches named gray, or as the convergence point of lines of equal hue. It has been shown that both points need not coincide (Ekroll, Faul, Niederee, & Richter, 2002). While the achromatic point is easy to determine, its definition is based only on a small subset of all 417 color categorizations and is vulnerable to occasional misclassifications. Further, the achromatic point is not available if the observer did not name any patch as gray. The convergence point of lines of equal hue, on the other hand, is based on almost all categorizations made and, thus, is better constrained by the measurements. To determine the convergence point, we varied the position of the point and the angles of the category boundaries. The position was varied on a discrete grid of 25 × 25 points with a spacing of 0.05 units, centered at the chromaticity of the illumination color. For each position on the grid, we first determined the optimal boundary angle between each pair of two adjacent categories and then computed the overall number of false classifications. Here, and in the following, a “false” classification means an empirical classification that was not predicted by the straight-line classification boundary. Likewise, a “correct” classification means an empirical classification that falls within the straight-line classification boundaries of that category. The point with the fewest overall false classifications was chosen as the convergence point. The optimal boundary angle between two categories was determined figuratively by rotating the angle in steps of 1° and finding the minimum number of false classifications. This operation was formalized by first determining an approximation of the category boundary from the angular mean of the average directions of the two categories. Then, the color circle was split at the angle opposite to this approximative category boundary, and a cumulative circular histogram of each color category was computed such that the value of each cumulative histogram for a particular angle gives the number of correct classifications for this category. The optimal angle is then given by the angle where the sum of the two cumulative histograms has a maximum. A color constancy index was used to quantify the degree of constancy in the different experimental conditions. This color constancy index relates the measured shift of the neutral point to the shift of the chromaticity of the illumination. The neutral point could be either the achromatic point or the convergence point of lines of equal hue. Let i_n be the chromaticity coordinate of the neutral illumination ((0, 0) in the DKL space) and i_c be the chromaticity coordinate of achromatic illumination (e.g., (0.5, 0) for reddish illumination), with the corresponding neutral points measured at a_n and a_c. The color constancy index c is then defined as the ratio of the distance between the measured neutral points and the distance of the illuminations: 

$c=\frac{\parallel {\mathbf{a}}_{\mathrm{c}}-{\mathbf{a}}_{\mathrm{n}}\parallel }{\parallel {\mathbf{i}}_{\mathrm{c}}-{\mathbf{i}}_{\mathrm{n}}\parallel }.$

(1)

The degree of color constancy index is 100% if the projection coincides with the adapting background (perfect constancy) and 0% if it coincides with the white point (no effect of the background color). The same measure has been used by others (e.g., Smithson & Zaidi, 2004; Yang & Shevell, 2002). 

Angles that define category boundaries were determined in the DKL space. To investigate the change of category angles under different illumination conditions, we analyzed the category angles relative to the chromatic direction of the illumination. Then, for each category, the angular distance between the angle in the reference condition (Experiment 1, neutral illumination) and the angle under the chromatic illumination was determined. For each subject, a spline interpolation was used to interpolate between these angular deviations of the categories. The mean deviation was obtained as the average across the spline interpolations for each subject or illumination condition. 

Results

Validation of the segmentation method

First, we analyzed how well the radial segmentation method fits the categorizations made by the observers. We computed the percentage of assignments that fall into a wrong category. If category boundaries were strongly curved or would not coincide at a single point in particular experimental conditions, one would find a large number of wrong assignments. This was not the case. We found only 10% ± 1% SD wrong assignments, consistently across experiments and illumination conditions (minimum, 9% in Experiment 2; maximum, 12% in Experiment 6). The 10% errors mostly result from errors near the category boundaries. We computed a histogram of the angular distance of each error from its nearest category boundary. The histogram peaks at the first bin (0–1°) and steeply decreases such that 50% of all errors have an angular distance less than 6.5° from a boundary. We conclude that the radial division into different categories is a good description of the categorical color space throughout all experimental conditions. 

We also computed straight-line fits without the constraint that the boundary lines between categories need to intersect at a point. This leads only to a small improvement of 2% less errors in the model fit, which was, in no condition, significant. Moreover, the center points of the individual category boundaries were displaced more or less randomly around the intersection point and did not reveal a systematic bias. 

Finally, we addressed the question of how well the convergence point is determined by the data. We computed how many more errors we get on average if we move the convergence point from its optimal location where the number of errors is minimal. In the immediate neighborhood, there are, on average, 1% more errors, 2% more at a distance of 4 units, 3% more at a distance of 6 units, and 5% more at 10 units. Given that the color patches were drawn from a grid with a spacing of 5 units and that the number of errors was, on average, 10% ± 1% SD, we conclude that the convergence point is reasonably constrained by the data. Moving the convergence point by 10 units to the next but one patch in color space would increase the average error from 10% to 15%. 

Color categories in the reference condition

Next, we analyzed the angles of the color categories under neutral illumination and how they vary within and across observers. Observers made two runs of the neutral condition in the experiment with full illumination context (Experiment 1). In this experiment, the subjects viewed the neutrally illuminated room that matched the color of the gray screen background and categorized a small circular patch in the center of the screen ( Figure 1b). The angles of the categories in the DKL space averaged across runs and across observers are shown in Figure 2a. A transformation of the data from the DKL stimulus space to the CIE 1931 chromaticity chart is shown in Figure 2b. 

Figure 2

 

Color categories in the reference condition (Experiment 1, neutral illumination). (a) Average color categories (colored in the color of the category) and category boundaries (black) in the DKL color space (b) and in the CIE 1931 diagram. (c) Overlay of the category centers with the Munsell color space shows good agreement. (d) Individual variations in the reference condition. The categories obtained from two runs are plotted as two adjacent color bars for each observer. Chromatic direction is given as degree in DKL space. The percentage agreement between the two categorizations is given to the right of the bars. Color categories are largely stable. (e) Variation of the seven color categories across observers (squares) and within observers (circles), given as degree in Lab space. Color categories are largely stable.

Next, we compared the main categories obtained in our study to the Munsell color space. The Munsell color space, developed by the artist Albert Munsell around 1905, is a collection of painted color chips arranged in a 3D color space at equal perceptual intervals. At the center of the color space are the achromatic axes running from black to white with Munsell values from 0 to 10. Around the achromatic axes, there are 10 main color sectors (red, red–yellow, yellow, yellow–green, green, blue–green, blue, purple–blue, purple, and red–purple). Distance from the achromatic axes represents saturation. For comparison to our data, we used the renotation of the Munsell color space in the CIE 1931 space for patches of a medium Munsell value of 5 (Newhall, Nickerson, & Judd, 1943). First, lines of equal hue in the Munsell system form nearly perfect straight lines in the CIE 1931 color space, in particular when restricted to the gamut used in the present study (dashed lines in Figure 2c). We conclude that lines of equal hue can be safely approximated by straight lines. Second, there is remarkable agreement between the categories measured in our study and the location of the Munsell hues (denoted by capital letters): orange equals YR, turquoise equals BG, blue equals B, purple equals P. Only two categories differ: red equals RP, and green equals no hue line but nicely falls between G and GY. The agreement is particularly striking given the difference in methods and may be viewed as further evidence for the universality of color categories (Berlin & Kay, 1969; Lindsey & Brown, 2006). 

The variations of the angles of the color categories and the category boundaries within and between observers are depicted in Figures 2d and 2e. In Figure 2d, we plot the categories in the reference condition obtained from two runs for each observer. The categories were largely stable within and across observers. To quantify the variation within observers, we determined the percentage agreement between the two runs, that is, the percentage of patches that were equally categorized in the two runs. All but one observer (E.M., who disagreed considerably for red–purple categorizations) showed above 75% agreement. The agreements ranged between 67% and 93%, with an average of 80% ± 8% SD. The values for the individual observers are given in Figure 2d. Inconsistent namings occurred close to the category boundaries. 

To further analyze the variations of the angles, the angles were converted to a perceptually more uniform color space, the CIE Lab space ( Figure 2e). The overall variation of the center of each category is small (3° ± 3° SD) and differs slightly between categories, with the smallest variation occurring for green (1° ± 1°) and the largest occurring for red (5° ± 5°). Computing the variation in the original DKL stimulus space results in the same small average variation of 3° but changes the variation in the individual categories: In the DKL space, the smallest variations occurred for orange (2° ± 2°) and the largest occurred for purple (5° ± 5°). 

The mean individual variability of a category center of 3° is only 1.6% of the maximally possible deviation of ±180°. The average variation across observers is about twice the individual variation (7° ± 6° SD). 

Color constancy

For color constancy to take place, the neutral point needs to follow the shift in chromaticity of the illumination. When the convergence point coincides with the chromaticity of the illumination, 100% color constancy occurs. This is a sufficient but not a necessary condition for color constancy. In general, for perfect color constancy, the neutral point in color space need not coincide with the illumination. The neutral point could be slightly offset, and when an equivalent offset is present under changing illumination, color constancy is also perfect. This idea has been formalized in the computation of an inferred achromatic surface (Brainard, 1998; Brainard, Longère, Delahunt, Freeman, Kraft, & Xiao, 2006; Kraft & Brainard, 1999). Because we observed no systematic offset of the neutral point from the chromaticity of the illumination under neutral illumination, we did not employ the concept of an inferred achromatic surface. 

First, we show the raw categorization results for three observers (those with best, median, and worst error of the straight-line model fit) in the two extreme experiments with maximum (Experiment 1, Figure 3) and minimum (Experiment 6, Figure 4) available context information. In the first experiment, the observers followed the shift in the illumination color almost completely: The lines of equal hues converge at the chromaticity of the illumination. For example, if the illumination was shifted halfway along the L–M axes to a reddish color, the convergence point almost perfectly coincided with the chromaticity of the illumination. Similar patterns occurred for illumination shifts in the other directions. The average category boundaries in the reference condition, translated to the new convergence point, give a good prediction of the angles of the category under changing illumination ( Figure 3). Note that the average category boundaries in the reference condition were determined for most observes from two runs and, thus, might differ from a single categorization in the reference condition (first row of Figure 3). 

Figure 3

 

Color-naming results in Experiment 1. Data of three observers (those with best, median, and worst error of the straight-line model fit) under different illuminations (neutral, reddish, bluish–greenish, yellow-greenish, and purplish; top to bottom). Categorization of each test patch is shown by the respective color. Bold, black lines mark the category boundaries; thin, black lines mark the average category boundaries in the reference condition, translated to the new convergence point (color of the illumination: plus sign; mean position of the gray area: open square). For every illumination condition, the convergence point of the category boundaries coincides with the illumination color, indicating perfect color constancy. The mean position of the gray assignments is also close to the position of the illumination color.

Figure 4

 

Color-naming results in Experiment 6. Format is identical to that of Figure 3. The convergence point differs considerably from the chromaticity of the illumination and lies about halfway between the white point and the illumination. The size of the gray regions is expanded, and the mean is considerably different from the intersection point of the category boundaries.

The categorization results for Experiment 6 showed a completely different pattern. In this experiment, information about the illumination was available to the observer only in the periphery. For all illumination conditions, the convergence point differed considerably from the chromaticity of the illumination. In the extreme case, the observer might be almost completely ignorant about the illumination and categorized the color patches similar to the neutral condition. For example, shifting the illumination along the (L + M)–S axes results in settings similar to the neutral condition ( Figure 4, observers D.A. and J.G.). A further difference to Experiment 1 is in the number of colors that are named gray. When averaged across all subjects, in the first experiment, only a single color is named gray, whereas in Experiment 6, about 20 colors are named gray. Also, in Experiment 6, the mean of the gray area (large square) did not coincide with the convergence point of the category boundaries. A dissociation between the perceptual criterion of gray and the structural criterion of the convergence of lines of equal hue has been reported previously (Ekroll et al., 2002). 

So far, we have presented sample data for three observers. What is the general pattern of the shifts of the convergence point in the different experiments? Figure 5a shows the shift of the convergence point for each of the five illumination condition in the six experiments. Data are averaged across observers. For Experiments 1 and 2, the average convergence point (circle and square) is close to or even coincides with the surround illumination (plus sign). When the test patch covered the whole monitor and the surround illumination was present on the wall and between trials on the monitor (Experiment 3), only on the wall (Experiment 4), or only on the monitor (Experiment 5), the convergence point for the different colored adaptation conditions lies between the white point and the chromaticity coordinate of the illumination. In Experiment 6, where the test patch was presented on a black background and the adapting color was present only in the periphery, the smallest shift toward the chromaticity of the surrounding illumination occurred. However, even in this experiment, where neither spatial nor temporal contrast between the test patch and the surround illumination was present, the surround illumination affected the categorization by shifting the convergence point about halfway toward the illumination. 

Figure 5

 

Shift of the convergence points and color constancy performance. (a) Mean positions of the convergence point of lines of equal hue for all observers, plotted in the isoluminant plane of DKL color space. The positions of the five different adaptation colors are indicated by bold, black plus signs. The symbols refer to the different experiments (Experiment 1: disc, Experiment 2: square, Experiment 3: diamond, Experiment 4: upward pointing triangle, Experiment 5: downward pointing triangle, Experiment 6: star). The colors of the symbols refer to the five different adaptation colors. (b) Color constancy performance for the different illumination conditions in the experiments. Close-to-optimal performance occurs for the small test patch on the illuminated background (Experiments 1 and 2). Reducing the available information about the illumination reduces color constancy. Colored symbols show indices based on the convergence point of lines of equal hue; black symbols show indices based on the mean of patches named gray. Average color constancy values in percentage are given for each experiment based on the convergence point/achromatic point. (c) Number of gray namings in the different experiments increases with decreasing color constancy. Error bars in all plots denote standard error.

To quantify the results, we computed a color constancy index ( Equation 1) for each condition and experiment ( Figure 5b). The constancy index is 100% if the neutral point coincides with the adapting background (perfect constancy) and 0% if it coincides with the white point (indicating no effect of the illumination color). Color constancy was computed based on the convergence point of the category boundaries and alternatively based on the mean position of the area named gray. Both methods gave similar values of constancy ( Figure 5b). In the following, we refer to the color constancy indices based on the convergence point. 

In Experiment 1, the average color constancy index was 99%. Limiting information about the illumination to the local surround by introducing a tunnel (Experiment 2) resulted in an equally perfect performance of 98%. When the test patch covered the whole monitor and a spatial comparison to the illumination was possible only in the periphery (12° × 8°), performance dropped slightly to 88% (Experiment 3). Color constancy performance dropped further down to 71% when no temporal contrast between test and surround illumination was present (Experiment 4). In Experiment 5, where only temporal contrast was available, the color constancy was only 66%. In Experiment 6, where neither temporal nor spatial contrast between the illumination and the test patch was present, color constancy dropped to about 50%. An interesting finding was that even in this condition, the convergence point was shifted about halfway from the white point in the direction of the illumination. 

We also evaluated the number of colors that were categorized as gray ( Figure 5c). As color constancy performance decreased, the average number of gray namings increased from 1 in Experiment 1 to 24 in Experiment 6. 

Control experiment

In the first experiment, we found higher degrees of color constancy than what was reported in most previous studies. This could be attributed either to the color-naming task or to the contextual stimuli used. In a control experiment, we determined color constancy using an achromatic adjustment paradigm. Observers ( N = 7) had to adjust the color of a central square stimulus to neutral gray. Context and illumination conditions were the same as in the first experiment. We found high degrees of color constancy for all illumination conditions, with no significant difference from the results obtained in the color-naming task ( p > .21). Consequently, the high degree of color constancy we have found was not due to the particular naming paradigm we have used but can be attributed to the contextual stimuli. This finding corroborates an earlier study where good agreement between achromatic adjustment and the mean of stimuli named neutral has been found (Speigle & Brainard, 1996). 

Variations of the angles of the color categories

How do the angles of the category boundaries change with changing illumination? In all experiments, the category boundaries shift depending on the illumination, but the angles of the boundaries are relatively stable within the same observer. This means that the color categories are shifted almost parallel to the changes in illumination. 

A closer inspection revealed that the rotation of the angles followed a particular pattern: The angles tended to rotate away from the direction of the illumination. Examples can be found in Figure 3: For observer J.G., shifting the illumination to yellow-greenish (fourth row) rotated the orange–red boundary to red and the green–turquoise boundary to turquoise when compared to the angles under neutral illumination. In both cases, the angles rotated away from the yellow-greenish illumination. Shifting the illumination to purple (last row) rotated the orange–red boundary to orange and the green–turquoise boundary to green, again away from the chromatic direction of the illumination. This rotation was most prominent for the angles of category boundaries that had an angular distance of 90° relative to the chromatic direction of the illumination. 

To quantify this effect, we plotted for each category the amount of rotation away from the chromatic direction of the illumination. More precisely, the angular distance (ranging from 0° to 180°) between the angles of the color categories and the chromatic direction of the illumination is plotted on the x-axis, and the amount of rotation away from the illumination when compared to the angle in the reference condition is plotted on the y-axis. A positive y value denotes a rotation away from the chromatic direction of the illumination. In the example above, all rotations take positive values. The analysis shows that the angular deviation from the neutral condition is, on average, positive and maximum for categories about 45–90° away from the illumination ( Figure 6a). 

Figure 6

 

Change of category angles. (a) Change of category angles under different illumination conditions in Experiment 1. Each point denotes how a category of a particular observer is rotated relative to its angle under neutral illumination. On average, categories rotate away from the illumination color, as indicated by the positive values of the average curve. (b) Average change of the relative category angle for the different experiments (1–6). On average, categories rotate away from the illumination color. The more the center of chromaticity is shifted toward the illuminating color, the larger is the mean deviation.

Comparing the average deviation of the category angles from the neutral condition across the different experiments shows an increase of the deviation with increasing shift of the neutral point ( Figure 6b). 

Discussion

Summary of findings

We investigated color constancy in a color-naming task. We found an almost perfect color constancy of 99% when small test patches were presented on an illuminated background. By systematically varying the size of the test patch as well as the spatial contrast and the temporal contrast between the patch and the surround illumination, we were able to draw conclusions about the importance of different cues for color constancy. 

1.  
   Nearly perfect color constancy (99%) occurred when both spatial contrast and temporal contrast between the test patch and the illumination were available in the center of the visual field. In this case, no contributions from the peripheral illumination beyond 10° × 8° were found: Despite the lack of any information from the illuminated room in Experiment 2, observers showed equal color constancy performance as in Experiment 1.
2.  
   The spatial comparison is best in the center of the visual field: Enlarging the test patch to 10° × 8° in Experiment 3 decreased color constancy by about 10%.
3.  
   Both spatial contrast from the periphery and temporal contrast could be used to obtain information about the illumination: Eliminating temporal contrast (Experiment 4) results in a reduction of color constancy performance of about 15%, and eliminating spatial contrast (Experiment 5) results in a reduction of about 20% relative to Experiment 3, where both cues were available.
4.  
   If neither spatial nor temporal contrast is available, the presence of a constant peripheral illumination leads to constancy of about 50%. Overall, our results clarify the extent to which different spatial and temporal mechanisms are involved in color constancy.

In all experiments, the color categories are shifted almost parallel to the illumination change. To a first approximation, the angles of the category boundaries remained relatively stable across the different illumination conditions and experiments. The average variation of the angles of the category boundaries is small and decreases with color constancy, that is, with smaller shifts of the neutral point in the direction of the illumination. A closer inspection revealed that the average variation of the angles followed a consistent pattern across all experiments. We observed a repulsion effect, where angles of the categories tended to rotate away from the chromatic direction of the illumination. Largest offsets occurred for categories about 90° apart from the illumination. To our knowledge, this effect has never been reported in previous work. For a better understanding, it would be interesting to see how actual chromatic surfaces, for example, the Munsell chips, would transform under changing illumination. Perfect color constancy implies not only that a single neutral point is shifted according to the illumination but also that all points in color space undergo a consistent transformation. In particular, we predict that the repulsion effect also occurs for real surfaces under varying illumination. We plan to investigate this in future work. It has been shown that the unique hues were associated with singularity-based categorizations of surfaces, predictable from the physics of light, surfaces, and photopigment absorptions (Philipona & O'Regan, 2006). Similarly, the repulsion effect we have found might also be explained from the underlying physics. 

Comparison to previous work

Measurements of color constancy most often used a matching procedure (e.g., Bäuml, 2001; Brainard, 1998; Kraft & Brainard, 1999; Yang & Maloney, 2001; for an overview, see Brainard, 2004). Only comparably few studies employed a color-naming paradigm (Boynton, Fargo, & Collins, 1990; Boynton & Purl, 1989; Smithson & Zaidi, 2004; Troost & de Weert, 1991; Uchikawa, Emori, Toyooka, & Yokoi, 2002; Uchikawa, Yokoi, & Yamauchi 2004). 

Troost and de Weert (1991) have previously employed a color-naming paradigm in a color constancy task because the results of their matching experiments were sensitive to instruction effects and difficult to interpret. Using the naming paradigm, they obtained a reliable measure of object color. Unlike in our experiments, subjects named the color of a single simulated patch. This is a rather coarse method due to the discrepancy between few available color names and many distinguishable colors. In our experiments, subjects named more than 400 patches in each condition, and color constancy was inferred from the shift of the convergence point. 

Smithson and Zaidi (2004) studied the effect of context on color constancy. Observers had to categorize patches into four categories (red, green, yellow, and blue). Test surfaces were displayed against a variegated background. They found high values of color constancy (between 58% and 94%) when the mean chromaticity of the background was unbiased. Constancy did not drop significantly for biased backgrounds, where the mean chromaticity did not provide a reliable estimate of the illuminant. 

In our experiment, we found perfect constancy under full context conditions. Most previous studies have reported smaller values. For example, Kraft and Brainard (1999) found maximal degrees of constancy of 83%, even with natural scenes. The large degrees of constancy we found were not due to the naming paradigm; they also occurred in a control experiment where a matching paradigm was employed (Control experiment section). One likely reason for the large values is the large homogeneous surround, covering 45° × 64° of the visual field, that provided an unambiguous cue about the illumination. Almost complete color constancy has been found in a previous study using the same experimental chamber (Rinner & Gegenfurtner, 2002). Recently, it has been confirmed that size matters: Almost complete color constancy was achieved with full-field adaptation to a 120° surround, whereas performance under equal conditions for a 20° surround was poor (Murray, Daugirdiene, Stanikunas, Vaitkevicius, & Kulikowski, 2006). 

Color constancy, as observed in our experiments, can be understood as adaptation to the surround illumination. Chromatic adaptation is an important mechanism involved in color constancy (Burnham, Evans, & Newhall, 1957; Jameson & Hurvich, 1989; Kaiser & Boynton, 1996; Webster & Mollon, 1995). When we walk into a room illuminated by tungsten light that has high energy at long wavelengths, the visual system adapts to the particular illumination by decreasing its sensitivity to the long wavelengths. The effect of adaptation is a shift of the neutral point, such that an achromatic sensation is evoked by stimuli that approximate the chromaticity of the illuminant (Jameson & Hurvich, 1989). Chromatic adaptation has previously been found to be largely complete after only 25 ms (Rinner & Gegenfurtner, 2002), showing that receptor adaptation cannot be the sole basis of color constancy. Besides early adaptation of cone photoreceptors (von Kries, 1878, 1905), second site adaptation of cone-opponent signals is involved in this process (Pugh & Mollon, 1979). One possible neural substrate in the cortex involved in color constancy might be found in V4 (Kusunoki, Moutoussis, & Zeki, 2006). Neurons in V4 shift their color-tuning profile with a change in background illumination, similar to the pattern observed in the present study (Kusunoki et al., 2006). 

It has been suggested that cone excitation ratios, which are statistically almost invariant under changes in illumination, provide the basis underlying perceptual color constancy (Foster & Nascimento, 1994; Nascimento, Ferreira, & Foster, 2002). In our study, we found that illumination of the remote surround, which was separated by the test patch by a 3° black annulus, can still shift the convergence point, on average, by 50% (Experiment 6). Further, we have shown that a purely temporal context, where the test surface is never viewed together with the illumination (Experiment 5), results in 66% color constancy. Cone excitation ratios need to be computed also in these conditions if the brain relied exclusively on cone excitation ratios. 

No advantages for natural illumination shifts

The color of natural daylight varies along the blue–yellow direction from dawn till dusk. Based on ecological relevance, one could speculate that color constancy may be better for illumination changes along the blue–yellow axis than along the red–green axis. We did not find such an advantage. Overall, color constancy indices were similar across illumination conditions in all experiments. This is in accordance with previous reports, which also did not find better color constancy performance for natural daylight illumination changes (Brainard, 1998; Delahunt & Brainard, 2004; Foster, Amano, & Nascimento, 2003). Also, a Bayesian model of human color constancy using a daylight prior for the illumination did not fit the experimental data well. Using a broad illumination prior substantially improved the fit (Brainard et al., 2006). 

Dissociation of the convergence point from the average gray point

The neutral point at the center of a color space is usually assumed to satisfy two criteria simultaneously: The neutral point is perceived as gray, and it is the convergence point of lines of equal hue. Although this is true in simple experimental paradigms using isolated light spots on a black background, both criteria are not simultaneously met in more complex experiments using infields in chromatic surrounds (Ekroll et al., 2002). Our findings corroborate these results. In Experiments 2–6 with reduced context, the mean location of the gray area differs from the convergence point of the equal hue lines. One may argue that this dissociation simply reflects the inadequacy of the linear model rather than revealing a dissociation of perceptual representations. Allowing curved lines of equal hue would cancel the dissociation. Although this is true in principle, several reasons speak against this interpretation. First, this would require lines of equal hue to bend strongly near the neutral point. However, only small deviations from linearity have been reported, which occur at high saturation, away from the neutral point (Abney, 1910; Wyszecki & Stiles, 1982). Also, the Munsell hue lines can be safely approximated by straight lines within the gamut used by the experiments (Figure 2c). Second, Ekroll et al. (2002) reported data that ruled out an explanation based on curved hue lines (their Experiment 3), showing that the lines of equal hue extend straight to the surround chromaticity and are separated from the achromatic locus by an extended region that is classified neither as achromatic nor as a hue match. Third, if the dissociation we found in our data would have been caused by a failure of the straight-line assumption, a larger dissociation should be paralleled by a larger number of false classifications by the model. However, we found a constant number of 10% false classifications in all experiments. 

Influence of the edge between different media

In principle, it might have been possible that the differences between Experiments 1 and 3 were confounded by the fact that the edge between the test patch and the background was not only between different colored surfaces but also between two different media at different depths, that is, the CRT screen and the chamber wall. When viewing the scene in the experiments, the two homogeneous surfaces appear as adjacent to each other, without any differences in texture or depth. 

Summary

Color constancy can be almost perfect in human observers. Observers followed an illumination shift almost completely when a small test patch was viewed in the center of the visual field. Color constancy processes evaluated both spatial and temporal context of the scene. Reducing the amount of either spatial or temporal contrast revealed that both contribute to color constancy, even in the periphery. Color constancy is largely the same for illumination shifts in different directions. Color categories can be well described by radial lines converging at a single point and are quite stable within and across observers and under illumination changes. 

Acknowledgments

We thank Maria Olkkonen for running the control experiment, David H. Brainard for insightful suggestions, and Miriam Spering for valuable comments on the manuscript. 

This research was supported by the German Science Foundation Grant Ge 879/5 and by a stipend from the Justus-Liebig University Giessen to S.W. 

Commercial relationships: none. 

Corresponding author: Thorsten Hansen. 

Email: [email protected]. 

Address: Justus-Liebig-Universität Giessen, Abteilung Allgemeine Psychologie, Otto-Behaghel-Str. 10F, 35394 Giessen, Germany.