Adaptable mechanisms sensitive to surface color in human vision

Abstract

“Color constancy” refers to our ability to recognize the color of a surface despite changes in illumination. A range of cues and mechanisms, from receptoral adaptation to higher order cognitive cues, is thought to contribute to our color constancy ability. Here we used psychophysical adaptation to probe for an adaptable representation of surface color. We used stimuli that were matched for cone contrast when averaged over time but were consistent with either a constant scene under changing illumination or a changing scene. The color opponent aftereffect during adaptation to the constant scene was greater than that induced by the changing scene stimulus. Since the stimuli were matched for the responses they would elicit in receptoral mechanisms, the increased aftereffect in the constant scene condition cannot be wholly attributed to adaptation of receptors and neural mechanisms responsive to raw quantal catch. We interpret our result as most parsimoniously explained by the existence of adaptable mechanisms responsive to surface color, most likely located in early visual cortex.

Introduction

Color constancy

To identify the color of surfaces in a scene correctly, we must separate out the reflectance properties of surfaces from the spectral properties of the light illuminating the scene, an ability called “color constancy.” Humans generally perform well, though imperfectly, when asked to identify surface properties under different illumination conditions (Brainard, Brunt, & Speigle, 1997; Craven & Foster, 1992; Helson, 1938; Judd, 1940; Worthey, 1985). 

Many higher order scene features may be used as cues when accomplishing color constancy, including specular highlights (Lee, 1986), inter-reflections (Bloj, Kersten, & Hurlbert, 1999), luminance color correlations (Golz & MacLeod, 2002), color memory (Hansen, Olkkonen, Walter, & Gegenfurtner, 2006), and Gestalt laws (Gilchrist et al., 1999). Such cues are likely to require interactions with higher order object representation, abstract color knowledge, and lexical representation of color names. For example, neuropsychological literature points to a dissociation between the coding of color object knowledge and lexical coding of color names and shows that both these abilities can be impaired in patients with normal color vision (Beauvois & Saillant, 1985; Luzzatti & Davidoff, 1994; for review, see Tanaka, Weiskopf, & Williams, 2001). 

Neural mechanisms for color constancy

That color constancy is accomplished requires that surface color is neurally encoded, but the stage at which that is done remains unclear. Receptoral mechanisms, particularly multiplicative scaling (Brainard & Wandell, 1992; Foster & Nascimento, 1994; Ives, 1912; Land & McCann, 1971; Shapley & Enroth-Cugell, 1984), can account for a large proportion of color constancy (Smithson, 2005). However, retinal mechanisms alone are insufficient to account for all of our color constancy (Brill & West, 1986; Worthey & Brill, 1986). 

Beyond the retina, it is less clear what neural mechanisms might underlie color constancy. In occipital cortex, a potential candidate is area V4. In non-human primates, area V4 has been argued to contain a neural representation of surface reflectances (Kusunoki, Moutoussis, & Zeki, 2006; Wild, Butler, Carden, & Kulikowski, 1985; Zeki, 1983), but the question of whether V4 is unique in these properties remains controversial (Conway & Tsao, 2006; de Monasterio & Schein, 1982; Gegenfurtner, 2003). In humans, there is less evidence that hV4 (the suggested homologue of V4) is a specialized “color center” (Gegenfurtner, 2003), and no work specifically shows color constant response properties in hV4 or other areas. 

Finally, the higher order nature of many cues thought to contribute to color constancy and the neuropsychological literature suggest the involvement of cognitive areas beyond occipital cortex, but how these influences are implemented neurally remains ill-defined. 

Models of color constancy typically imply a processing stage in visual cortex that transforms photoreceptor activations into a representation of surface color (Brainard & Freeman, 1997; D'Zmura & Lennie, 1986; Lennie, 1999; Logvinenko & Maloney, 2006; Robilotto & Zaidi, 2004). Yet the large proportion of color constancy accomplished at the level of the retina, combined with the cognitive nature of many cues thought to contribute to color constancy, leaves open the question whether a neural representation of surface color intermediate to the retinal and cognitive stages exists. Here we sought to address this question using a color opponent aftereffect. 

Color opponent aftereffects

Prolonged viewing of a field of one color induces a color opponent aftereffect. For example, after viewing a red field, a neutral gray appears tinged with green. This effect is mediated predominantly by mechanisms at a low level: photoreceptors reduce their responsiveness over a period of prolonged stimulation, changing the response of post-receptoral mechanisms to subsequent stimuli, and shifting the appearance of those stimuli away from the color of the adapting stimulus. When adapted to red, the photoreceptor response to a neutral gray is the same as the response to green under neutral adaptation. 

Here we used prolonged viewing of two types of stimuli to investigate the aftereffects of adaptation to surface color. We used stimuli that induce the same amount of receptoral adaptation but which evoke different percepts of surface color. We will show an aftereffect that cannot be wholly attributed to adaptation of receptors and neural mechanisms responsive to the time-averaged input signal. We attribute this effect to the adaptation of neural mechanisms that encode surface color in a manner more robust to illuminant changes than the encoding by retinal mechanisms, located beyond the retina and most likely beyond other subcortical areas, but before cognitive areas. 

Methods

Color calibration procedures and display system

Stimuli were generated and displayed using Matlab (version 7) software, with routines from PsychToolbox (Brainard, 1997; Pelli, 1997), on a Dell OpliPlex 755 computer driving an ATI Radeon HD 2400 Pro graphics card (8 bits per channel) to draw stimuli to a 33 × 24 cm Sony Trinitron E220 cathode ray tube monitor, refreshed at 85 Hz. Experiments took place in a darkened room with black walls, and the monitor was viewed from a distance of 0.57 m. The monitor was calibrated using a ColorCAL colorimeter (Cambridge Research Systems). Changes in both chromaticity and luminance of the screen with increasing R, G, and B values were taken into account when generating the experimental stimuli. The CIE (xyY) coordinates measured for 16 values during calibration were interpolated to 255 values using the best fitting spline, and these were used to calculate the xyY coordinates of each combination of R, G, and B intensity values, resulting in a 256 × 256 × 256 matrix of xyY coordinates. 

We did not obtain color matching or spectral sensitivity functions for individual observers but used measurements from “standard observers” (see Brainard & Stockman, 2010 for a discussion of errors that could be introduced by this assumption). Each xyY coordinate was transformed into an LMS cone excitation coordinate using a conversion matrix derived from the left matrix division of the CIE 1931 color matching functions (Wyszecki & Stiles, 1982) by the Stockman and Sharpe (2000) 2-degree cone spectral sensitivity functions. Both the CIE (1931) color matching functions and the Stockman and Sharpe (2000) 2-degree cone spectral sensitivity functions were obtained from the “PsychColorimetricData” folder of PsychToolbox, and the CIE (1931) color matching functions were scaled by 683 in order that the Y (luminance) coordinate had units of candelas/m². Stimuli were specified in terms of their LMS cone excitation coordinates and the closest RGB value was found by finding the lowest root mean square error between the target cone excitation and the values achievable on the monitor. Where the lowest root mean square error was greater than 0.01, or the best fitting RGB value included a channel at maximum intensity, the stimulus was considered to be out of range and excluded. 

Observers

Ten observers (five males), aged 20 to 35 years old, took part; eight were naive to the purposes of the investigation (all observers except the authors EG and SS). All had normal or corrected-to-normal visual acuity and normal color vision as assessed using Isihara (1990) and the Hardy–Rand–Rittler (HRR, 4th edition, published by Richmond Products) psuedoisochromatic plates. Observers provided informed consent, and the entire study was carried out in accordance with guidelines of the Human Research Ethics Committee of the University of Sydney. 

Adapting stimuli

Adapting stimuli simulated either a constant scene under changing illumination or a changing scene under constant illumination, as illustrated schematically in Figure 1. In the constant scene condition, the stimulus simulated an array of flat, matte surfaces under diffuse illumination that varied sinusoidally between two different illuminants at 1 Hz. In the changing scene condition, the same surfaces, illuminants, and rate of illuminant change were used as for the constant scene condition, but the phase of the illuminant modulation was randomized across the surfaces. The two conditions were identical when averaged over time; across each cycle of illumination change (1 s), they had the same surfaces under the same local illumination. The adapting conditions differed only in whether the illuminant modulation was the same for all surfaces (the “correlated” condition, Figure 1A) or had a different randomly chosen phase for each surface (the “decorrelated” condition, Figure 1B). 

Figure 1

 

Sample stimuli for the “correlated” and “decorrelated” conditions, with fixation cross. Adapting stimuli consisted of up to 3000 ellipses with colors that simulated flat matte surfaces under diffuse illumination. Over time, the illumination sinusoidally modulated from an incandescent bulb (CIE Illuminant A) to daylight (CIE Illuminant D65). Across one cycle of this illumination change, the two adapting stimuli had the same surfaces under the same illumination. The difference between the two conditions was that the changes of different surfaces were either (A) correlated with each other or (B) decorrelated, with random onset phases. (C) Sample frames of the test stimuli. On the left is a sample frame with the red reference to the right of fixation ((L − M)/(L + M) chromaticity coordinate of 0.67). On the right is a sample frame with the green reference to the left of fixation ((L − M)/(L + M) chromaticity coordinate of 0.59).

Rendered surfaces were randomly drawn from 184 surface reflectance functions supplied by Hannah Smithson (Smithson & Zaidi, 2004). These were derived from a series of measurements of natural and man-made objects (Chittka, Shmida, Troje, & Menzel, 1994; Hiltunen, 1996; Marshall, 2000; Vrhel, Gershon, & Iwan, 1994), including a broad range of colors. The two illuminants were an incandescent bulb and daylight (CIE Standard Illuminants A and D65, respectively) and were scaled such that they had approximately equal photopic luminance, using the Stockman and Sharpe (2000) 2-degree luminosity function. 

Three thousand ellipses were drawn at random locations within the stimulus field and each was rendered with one of the surfaces. The heights of the ellipses ranged from 0.03 to 1.3 degrees visual angle, with the probability of each height h inversely proportional to h ³. The width of each ellipse varied randomly between 1/3h and 3h. Overlaid on this array of surfaces were the circular adapting stimuli, located to the left and right of fixation, with diameters subtending 8 degrees. The total arrangement was a rectangle subtending 30 × 16 degrees, and the remainder of the screen was black. 

For each observer, the adapting stimuli were either green to the left and red to the right of fixation, or yellow–green to the left and blue to the right of fixation; chromaticities of all adapting stimuli under both illuminants are shown in Table 1. Adapting surfaces were rendered under the same illumination as the background surfaces; in the “correlated” condition, the modulation was in the same phase for each of the two adapting surfaces and for the background; for the “decorrelated” condition, the modulations of the two adapting surfaces were perfectly out of phase with each other. In the correlated condition, the illuminant modulation always commenced and finished with the daylight (CIE Standard Illuminant D65) illumination, while the onset phases of the illuminant modulation for surfaces in the decorrelated case were randomly drawn from all points in the cycle. In the decorrelated condition, the adapting surface on the left commenced and finished with CIE Standard Illuminant A illumination and the adapting surface on the right commenced and finished with CIE Standard Illuminant D65. Each adaptation run began with a minute of adaptation before the first trial, and there was 6-s (6 stimulus cycles) top-up adaptation before each subsequent trial. 

Table 1

 

Adapting stimuli chromaticities. The cone fundamental primaries (LMS) and Macleod–Boynton chromaticity coordinates ((L − M)/(L + M) and S/(L + M)) are shown for each of the adapting surfaces, under both illuminants. Over time, the illumination of the adapting stimuli modulated sinusoidally between CIE Standard Illuminant D65 (daylight) and CIE Standard Illuminant A (an incandescent bulb).

Illuminant	Adapting surface	Cone fundamental primaries			Macleod–Boynton chromaticity coordinates
		L	M	S	(L − M)/(L + M)	S/(L + M)
CIED65	Red	0.027	0.015	0.008	0.742	0.361
	Green	0.022	0.023	0.008	0.601	0.362
	Blue	0.023	0.022	0.019	0.624	0.801
	Yellow	0.023	0.022	0.015	0.624	0.647
CIE A	Red	0.026	0.011	0.002	0.782	0.111
	Green	0.021	0.018	0.002	0.652	0.115
	Blue	0.022	0.017	0.005	0.674	0.252
	Yellow	0.022	0.017	0.004	0.674	0.204

Test stimuli and observer's task

We measured the perceived chromaticity of test surfaces in each of three conditions: prior to adaptation and during adaptation to correlated and decorrelated adapting stimuli. On each trial, the period of top-up adaptation was followed by a test stimulus, where the two adapting stimuli were replaced by a test and a reference surface. If the observer failed to respond within 3 s of the test stimuli being presented, the display returned to another 6 s of top-up adaptation before presenting the same trial again. The spatial arrangement of background surfaces remained the same as they were during the adapting stimulus, and the illumination of the entire scene was CIE Standard Illuminant D65. For the condition without adaptation, all stimulus and timing parameters were the same as for the adapting stimuli, with the exception that the initial adaptation period and the top-up adaptation periods were omitted; one test stimulus was replaced immediately by the next, meaning that the total duration of this condition was approximately a quarter the duration of the adaptation conditions. 

Response times did not vary greatly between conditions; the median reaction times in each condition, averaged across subjects (±1 standard deviation), were 1.18 s (±0.32 s) in the correlated adaptor condition, 1.15 s (±0.32 s) in the decorrelated adaptor condition, and 1.25 s (±0.47 s) in the no adapt condition. 

In conditions in which observers were adapted to red and green, they were asked to respond with a button press to indicate whether the surface to the left or right of fixation appeared “more red” (observers were instructed to treat “more red” and “less green” as equivalent). We estimated the chromaticity of the test that had perceptual equality with the reference (the test that was equally likely to be reported more red or more green than a reference) during two sessions, each of which was made up of eight interleaved adaptive psychophysical staircases (Kontsevich & Tyler, 1999) consisting of 30 trials each. For each staircase, the test surface had a fixed S/(L + M) chromaticity coordinate and position (either S/(L + M) = 0.24 or 0.71, located to the left of fixation, or S/(L + M) = 0 or 0.47, located to right), while the (L − M)/(L + M) chromaticity coordinate was set by the adaptive staircase. On each trial, the reference surface located on the opposite side to the test had the same S/(L + M) chromaticity coordinate as the test surface. Its (L − M)/(L + M) chromaticity coordinate was 0.59 (green, when the reference was on the left and the test was on the right) or 0.67 (red, when the reference was on the right). We shifted the chromaticity of the reference stimuli toward the adapting chromaticity at that location in order to ensure that observers point of equal redness/greenness was within the gamut of our monitor. We introduced this shift after a pilot experiment with the same reference stimuli on the left and right where the adaptive staircase reached the end of the range of our monitor before finding a chromaticity that observers reported to be “more red” 50% of the time. Ten randomly generated spatial arrangements of the background surfaces were interleaved throughout the run; the adaptation and test phases of each trial were formed from the same configuration. 

The data from each staircase (30 trials) were used to make one estimate of the chromaticity of the test surface that the observer was equally likely to report as more red or more green than the reference. The proportion of “more green” responses (

$\widehat{Y}$

) as a function of test chromaticity (x) were fit using the following logistic equation: 

$\widehat{Y}=c/2+\frac{1-c}{1+e^{-(x-a)/b}},$

(1)

where a is the test chromaticity of perceptual equality with the reference (the value of x where

$\widehat{Y}$

= 0.5), b is a curvature parameter, and c is the miss rate. Sample data from 2 staircases for one observer, along with fitted curves, are shown in Figure 2. 

Figure 2

 

A sample pair of psychometric curves shows the adaptation-induced shifts in chromaticity of the test surface during adaptation to green, for one naive observer, JC. Positive values along the abscissa are test surfaces that are more green than the reference. For both conditions, the curves are shifted to the right, indicating a color-opponent aftereffect. Any extra shift in the “correlated” condition cannot be attributed to receptoral adaptation, since when average over time the two conditions contain the same LMS cone contrast.

Four observers adapted to red and green surfaces, and two of these observers along with additional four observers adapted to blue and yellow surfaces. The design for blue and yellow surfaces was equivalent: observers reported whether the left or right surface was “more blue” (equivalent to “less yellow”). Test surfaces had a fixed (L − M)/(L + M) chromaticity coordinate and position (either (L − M)/(L + M) = 0.62 or 0.72, located to the left of fixation, or (L − M)/(L + M) = 0.57 or 0.67 located to right), while the S/(L + M) chromaticity coordinate was set by the adaptive staircase. The reference surface had the same (L − M)/(L + M) chromaticity coordinate as the test surface and had an S/(L + M) chromaticity coordinate of 0.10 (yellow, when reference was on the left and test on the right) or 0.45 (blue, when reference was on the right). 

Results

Opponent aftereffect for surface color

We measured the shift in perceived chromaticity of a test surface while subjects were adapted to red and green, or blue and yellow surfaces, using a two-alternative forced-choice paradigm (see Methods section for details). 

Figure 2 shows for one observer how the appearance of a single test surface changed with adaptation to a green surface. The magnitude of this change in appearance depended on the context of the green adapting surface, which varied with adaptation condition. Without adaptation, the chromaticity of subjective equality with the reference was nearly veridical (the point of subjective equality was 0.0012, for clarity that curve is not shown). During adaptation to the decorrelated stimulus, the curve shifted substantially to the right—the observer required the test surface to be greener than the reference to see it as the same. This shift is expected because the adaptor changes the state of the receptoral and neural mechanisms that lie over the test surface. If there were only adaptation of mechanisms responsive to the time-averaged input signal, then the aftereffect induced during the correlated condition should be of the same magnitude. It was not: instead the psychometric function shifted further to the right. 

The magnitude of the aftereffect in the two adaptation conditions for individual observers are shown in Figure 3. In Figure 4, the results are summarized; the average difference between the conditions for each adapting color is expressed as a percentage of the shift in the decorrelated case. For each adapting color, the perceived chromaticity of the test surface shifted away from the color of the adapting surface, consistent with a color opponent aftereffect. As in the example in Figure 2, the color opponent aftereffect was usually of greater magnitude in the correlated condition than in the decorrelated condition. 

Figure 3

 

Adaptation-induced shifts in perceived chromaticity of a reference surface, after adaptation to (A) red and green or (B) blue and yellow. Mean shifts in perceived chromaticity along the (A) (L − M)/(L + M) and (B) S/(L + M) dimensions are shown for each observer (points, ±1 standard error of the mean) and averaged across observers (gray bars). The magnitude of shifts in perceived chromaticity of test surfaces is plotted relative to the physical chromaticity of the reference surface. The direction of these shifts depended on adaptor color; when observers adapted to red or blue, the shifts were positive along the (L − M)/(L + M) and S/(L + M) dimensions, respectively; for green and yellow, the shifts were negative. In each case, the direction of the shift was consistent with the subject requiring a test surface whose chromaticity was shifted toward the adapting color for a perceptual match with the reference. When the adapting surfaces were red, green, or blue, the correlated condition (C) induced an aftereffect of significantly greater magnitude than the decorrelated condition (D). When the adapting surface was yellow, the effect went in the same direction, but the difference between conditions was not significant (see text for details).

Figure 4

 

Adaptation-induced shift in chromaticity: average difference between the correlated and decorrelated adaptation conditions, as a percentage of the shift in the decorrelated condition. Here data from Figure 3 are replotted, averaged across all observers. Error bars indicate ±1 standard error of the between-subjects mean.

Statistical analysis

For our statistical analysis, we excluded the data collected without adaptation and for each adapting color (red, green, blue, and yellow) performed a 2-way analysis of variance. In each 2-way analysis of variance, we included observer as a random effect and tested for a main effect of adaptation condition (correlated versus decorrelated). For adaptation to red, green, and blue, the correlated adaptor induced an aftereffect of significantly greater magnitude (red: p < 0.01, F _1,3 = 30.19, green: p < 0.05, F _1,3 = 27.30, blue: p < 0.05, F _1,5 = 10.20). For adaptation to yellow, the correlated adaptor induced an aftereffect of greater magnitude, but this difference was not significant (p = 0.20, F _1,5 = 2.03). 

Effect cannot be explained by frame-by-frame differences in spatial cone contrast

The correlated and decorrelated adaptors above have identical distributions of colors over space and time, but within any frame of the stimulus, the spatial variance in chromaticities is lower in the correlated condition. Brown and MacLeod (1997) demonstrated that colors embedded in a background of greater variance appear less saturated; for our stimuli, this would predict that the adapting stimuli would appear less saturated in the decorrelated condition, which might diminish the opponent aftereffect. To test this possibility, we matched the correlated and decorrelated conditions for variance in LMS cone excitation coordinates across individual frames of the adapting stimulus. 

To generate adapting stimuli with the same variance, the L, M, and S cone excitation coordinates of decorrelated stimulus surrounds were scaled by 1.00, 0.97, and 0.66, respectively, as detailed in 1. All other stimulus properties were unchanged. Using these stimuli, we repeated the decorrelated condition for two observers (one naive) for both the red–green and blue–yellow adapting stimuli. Individual results are plotted in Figure 5 and summarized in Figure 6. 

Figure 5

 

Adaptation-induced shift of perceived chromaticity during adaptation to red, green, blue, and yellow. Data for observers AG, EG, and TF from Figure 3 are replotted beside the adaptation-induced shift found using a low-contrast version of decorrelated adapting stimulus (L). The low-contrast decorrelated stimulus was matched to have the same frame-by-frame L, M, and S variances as the correlated adapting stimulus. Conventions as in Figure 3.

Figure 6

 

Adaptation-induced shift in chromaticity: average difference between the correlated and low-contrast decorrelated adaptation conditions, as a percentage of the shift in the low-contrast decorrelated condition. Here data from the red–green and blue–yellow conditions (shown in Figure 5) are averaged across 2 observers and across different reference chromaticities. Error bars indicate ±1 SEM.

As in the main experiment, the correlated adaptor induced an aftereffect of greater magnitude than the low-contrast decorrelated adaptor. This difference was not significant when analyzed in the same manner as above, with a 2-way analysis of variance for each adapting color (red: p = 0.22, F _1,1 = 7.77, green: p = 0.26, F _1,1 = 5.31, blue: p = 0.13, F _1,1 = 22.41, yellow: p = 0.29, F _1,1 = 4.33), but unlike the main experiment, where each condition was completed by at least four subjects, only two subjects completed each condition here. Subjects who completed both experiments generally showed comparable magnitude of illusion in the decorrelated condition and the low-contrast decorrelated condition, and in all cases, adaptation to the correlated scene induced an effect of greater magnitude than the low-contrast decorrelated scene. The difference in magnitude of the average effect here (Figure 6) and in the main experiment (Figure 4) therefore can mostly be attributed to inter-subject variability: some of the subjects with high illusion magnitudes in the main experiment were not available for this control experiment. 

Overall, these results show that the difference in aftereffects seen during adaptation to correlated and decorrelated surrounds cannot be accounted for by differences in spatial cone contrast. 

Effect remains when adapting stimuli are matched for onset and offset phases

In the main experiment, the onset and offset phases of the adapting stimuli varied between the correlated and decorrelated adaptation conditions. In the correlated condition, all adapting surfaces were rendered under the same illuminant as the test surfaces (CIE Standard D65) at the beginning and end of each adaptation period. In the decorrelated condition, only the red and blue adapting surfaces were rendered under CIE Standard D65 at the beginning and end of each adaptation period; the green and yellow adaptors were rendered under CIE Standard A. To test the possibility that this difference between the conditions at the transition between adaptation and test may have contributed to the differing aftereffects, we performed an additional control experiment where the adapting stimuli were all rendered under an equal mixture of the two illuminants at their onset and offset. 

For this experiment, we repeated a subset of the original experiment for two observers who completed the original version (one naive) and for two new naive observers. We tested the red–green adaptation condition, using one of the four S/(L + M) coordinates for the test stimuli (which was the same as for the adapting stimuli). If the modulation of the illuminant over time is CIE Standard D65 at 0° phase, CIE Standard A at 180° phase, and an equal mixture of the two occurs at 90° and 270°, then in the original experiment the adaptation in the correlated condition always commenced at 0°, and in the decorrelated condition, the red and blue adapting surfaces commenced at 0° while the green and yellow adapting surfaces commenced at 90°. In this control experiment, adapting stimuli in the correlated condition commenced at either 90° or 270°, and each staircase included an equal number of both types, randomly interleaved in order. In the decorrelated condition, the two adapting surfaces remained 180° out of phase with one another, but on each trial one surface commenced at 90° and the other at 270°. Each staircase in the decorrelated condition included an equal number of trials where the surfaces commencing at 90° and 270° were on the left and right, or on the right and left, respectively. Since each adaptation period included an integer number of cycles, the onsets and offsets of the two adapting surfaces were always an equal combination of the two illuminants. All other stimulus properties were unchanged. Individual results are plotted in Figure 7, and summarized in Figure 8. 

Figure 7

 

Adaptation-induced shift of perceived chromaticity during adaptation to red and green, for correlated and decorrelated adapting surfaces that were matched in their onset and offset phases (see text for details). Conventions as in Figure 3.

Figure 8

 

Adaptation-induced shift in chromaticity for adapting stimuli that were matched in their onset and offset phases (see text for details). Bars show the average difference between the correlated and decorrelated adaptation conditions, as a percentage of the shift in the low-contrast decorrelated condition. Data from Figure 7 are averaged across 4 observers. Error bars indicate ±1 SEM.

In this control experiment, the correlated adaptor induced an aftereffect of greater magnitude than the decorrelated adaptor for both the red and green adapting surfaces. As for the previous experiments, we performed a 2-way analysis of variance for each adapting color and found that this difference was significant for the red adapting surface (p < 0.01, F _1,3 = 51.33) but not the green (p = 0.12, F _1,3 = 4.08). Subjects who completed the original experiment showed a similar pattern of results in this control condition. 

In the original experiment, where the adapting stimuli differed in their onset and offset phases, it is possible that they induced different degrees of adaptation in a transient, low-level mechanism. In this control experiment, the onset and offset phases of the adapting stimuli were always at a point in the cycle where the illuminant was an equal mixture of the two illuminants. Additionally, for both adapting surfaces and for both adaptation conditions, there was an equal number of 90° and 270° onset/offset phases. Any transient, low-level mechanism should be adapted to the same extent in the correlated and decorrelated adaptation conditions, but the magnitude of the aftereffect induced in the correlated condition remains greater than that induced in the decorrelated condition. Overall, these results suggest that the finding in the original experiment cannot be accounted for by differences in the onset and offset phases. 

Discussion

Implications for representations of surface color

We show that a color opponent aftereffect of greater magnitude is induced by adaptation to scenes consistent with changing illumination than by those that are consistent with a changing scene. These scenes recruit the same adaptation of receptors and of neural mechanisms responsive to the time-averaged input signal. We interpret this increased aftereffect as evidence of adaptable neural mechanisms sensitive to the color of surfaces, independent of illumination, which are adapted to some extent by the display that simulates a changing scene (the “decorrelated” condition) and to a greater extent when the display simulates a constant scene (the “correlated” condition). While this representation of surface color may not include all cognitive factors thought to contribute to color constancy, it must incorporate the result of mechanisms beyond retinal adaptation, since the stimuli in our two conditions were matched for the extent to which they would adapt retinal mechanisms. 

At a minimum, the neural mechanisms giving rise to color constancy must include three broadly defined stages: First, the receptoral processes that compensate for gross shifts of light intensity through multiplicative scaling; second, an adaptable stage by which the processing of some global stimulus statistics have led to a further discounting of the illuminant, providing a more robust representation of surface reflectance; and third, the cognitive processes, including the influences of object knowledge, language, and scene interpretation. 

Location of adaptable surface reflectance detectors

What mechanisms might provide the adaptable representations of surface color? It is unlikely that these representations could arise before visual cortex, both because they are adaptable, and because there must be a separation of surface and illuminant properties beyond that accomplished by retinal mechanisms. Neural mechanisms of the LGN important for color perception do not appear to habituate (de Valois, Abramov, & Jacobs, 1966; Solomon, Peirce, Dhruv, & Lennie, 2004; Tailby, Solomon, & Lennie, 2008), implying that adaptation of these mechanisms cannot be responsible for the increased aftereffect. Furthermore, there is no evidence of neurons in the LGN with chromatic properties thought to be essential for color constancy beyond that achieved by the retina. Specifically, double opponent cells are argued to be necessary for separating surface and illuminant (Hurlbert, 1996; Lennie & D'Zmura, 1988). Double opponent cells have receptive fields that include two L–M opponent components of opposite sign, which leads to a preferential response to chromatic edges (Solomon & Lennie, 2007). While they may exist, no cells with this property have been reported in retina or LGN. Within visual cortex, it is possible that adaptable mechanisms occur as early as V1, where double opponent cells have been reported in macaque (Conway, 2001; Conway & Tsao, 2006; Johnson, Hawken, & Shapley, 2001). 

Relation to other scission perception

Color constancy, and the task of separating illuminant and surface color properties, may helpfully be considered to lie within the broader category of any perceptual scission of layers within a scene. For example, we think it likely that mechanisms that underlie color constancy are also involved in the perception of transparent layers in a scene (Anderson & Winawer, 2005; Gerbino, Stultiens, Troost, & de Weert, 1990; Westland & Ripamonti, 2000). In relation to transparency, our result would imply that there is an adaptable representation of at least the lowermost layer in the scene. A possibility not explored by this study is that there are separate representations of different layers in each scene, each of which is independently adaptable. This would predict separate adaptation to illuminant color, or transparent surface color, along with surface reflectance properties. Such a mechanism would be a neural correlate of the separate layer representations in intrinsic image models, among others, which propose that scene properties such as reflectance, illumination, and transparency are decomposed and represented as separate layers or “images” (Adelson, 2000; Anderson & Winawer, 2008). Future work could examine whether representations of other layers are independently adaptable, and if so, whether there is a limit to the number of layers that the visual system encodes in this way. 

Potential limitations of this study

In interpreting the results of this study, we have assumed that prolonged viewing of the “correlated” and “decorrelated” stimuli should induce the same degree of adaptation in receptors and neural mechanisms whose responses scale with the output of the receptors. This assumption may be invalid if the stimuli are not matched in the responses they evoke in such mechanisms. In our results, we describe a control experiment where we tested whether the lower spatial variance in chromaticities in the correlated condition could account for the increased magnitude of the aftereffect in this case; we found that it could not. 

Our design also assumes that we are engaging mechanisms that adapt on a time scale of 1 s or longer, since the stimuli in the two conditions are only matched for cone contrast when averaged over the entire stimulus cycle. If more transient low-level adaptation were contributing to the effect, the time-dependent differences between the two types of stimuli will no longer be matched for these mechanisms. In the second control experiment described in our results, we balanced the onset and offset phases of our adapting stimuli in order to match the stimuli for any transient low-level mechanisms and found an effect of comparable magnitude to that in the original experiment. However, it remains possible that our results were influenced by other adaptable mechanisms integrating information over a longer period than a transient mechanism, but less than a second, or that give unequal weighting to stimuli presented in the preceding second. Chromatic adaptation likely includes many processes with varying time constants (Fairchild, 2005; Webster, 1996). For example, Fairchild and Reniff (1995) report psychophysical evidence suggesting that chromatic adaptation includes mechanisms that adapt over the course of a few seconds, or over many tens of seconds. 

Conclusions

One problem with approaching the neural mechanisms of color constancy experimentally is that it is difficult to separate out responses of neurons to the raw wavelengths, or color contrast, from responses that show an additional degree of color constancy. We have demonstrated psychophysically that such neural representations exist and that they are adaptable. The procedure here reveals intermediate mechanisms in the representation of surface reflectance and may therefore be useful for future investigations of color constancy and animal models. We hope to use this aftereffect in conjunction with functional imaging to further elucidate the neural substrates of human color constancy ability. 

Appendix A

Stimulus LMS statistics

Figure A1. 

Figure A1

 

Stimulus statistics for the background surfaces in the (A) correlated illuminant condition, (B) decorrelated illuminant condition, and (C) the reduced LMS variance decorrelated illuminant condition. The mean L, M, and S values are shown in the top, middle, and bottom plots, respectively, for one cycle of the adapting stimulus. The gray shaded region is the mean ±1 standard deviation, and the black lines plot the maximum and minimum L, M, and S values on each frame. The mean (±1 standard deviation) is also written in the top right of each plot. The reduced LMS variance decorrelated illuminant condition (C) was generated by scaling the decorrelated illuminant condition B to have the same variance of L, M, and S values as the correlated illuminant condition A.

Acknowledgments

We thank Hannah Smithson for providing reflectance spectra of the simulated surfaces. This work was supported by an Australian Postgraduate Award to EG and an Australian Research Fellowship to CC and by the Australian Centre of Excellence in Vision Sciences. 

Commercial relationships: none. 

Corresponding author: Erin Goddard. 

Email: [e-mail]. 

Address: School of Psychology, The University of Sydney, Sydney, NSW 2006, Australia.