Separating color from color contrast

Abstract

Visual objects can be described by their color and by their color contrast. For example, a red disk in front of a white background appears “red with high color contrast,” whereas a red disk in front of a slightly less-saturated red background will appear “red with low color contrast.” This paper examines the visual response to color contrast in a cone-based color space. The stimulus consists of two disks whose chromaticity and/or luminance modulate in time along a line in a DKL color space; the chromaticity and luminance levels of the two disks are always identical. One disk is surrounded by a static ring whose color is at one end of the color line, and the other disk is surrounded by a static ring whose color is at the opposite end of the color line. The disks appear to modulate in antiphase (following the contrast information), yet they can also appear to be approximately the same color (following the chromatic/luminance information). The observers' task was to adjust the color angle of modulating disks until the antiphase appearance was eliminated—creating a contrast null. Observers set contrast nulls at a color angle approximately 90 deg away from the line connecting the colors of the surround rings; this result occurred in both chromoluminant and equiluminant color planes, although two observers showed a flattening near equiluminance in the chromoluminance planes. To account for the data, I present a model that contains one pathway for color and another pathway for color contrast. I show that (1) the model correctly predicts orthogonal directions in color space for the contrast nulling task; (2) the response of the contrast pathway appears to be faster than the response of the color pathway; (3) the response of the contrast pathway may mediate detection thresholds under some conditions (a finding that can account for some of the effects of surround luminance on temporal sensitivity); (4) the asynchronous modulation can be seen even when the stimulus is blurred; and (5) the asynchrony does not require a disk-ring configuration.

Introduction

Visual objects can be described in terms of color and color contrast. For example, a red disk in front of a white background appears “red with high color contrast,” whereas a red disk in front of a slightly less-saturated red background will appear “red with low color contrast.” In this paper, I examine how responses to contrast information are organized within a cone-based color space, and I present a model that explicitly incorporates separate color and color contrast pathways. 

This project stems from a number of recent studies that have shown that contrast adaptation occurs at rates much faster than the temporal response of chromatic systems (Shapiro, Beere, & Zaidi, 2003; Shapiro, Hood, & Mollon, 2003; Webster & Wilson, 2000; Zaidi, Spehar, & Debonet, 1998). Based on these results, Shapiro, Hood, et al. (2003) concluded that the system that controls contrast adaptation is not the same as the color pathways that carry the signal that was ultimately detected. This conclusion is consistent with later physiological investigations that show separate gain controls for luminance and contrast information (Mante, Frazor, Bonin, Geisler, & Carandini, 2005). 

The Shapiro, Hood, et al. (2003) conclusion led to a search for methods capable of separating the visual response to chromatic and luminance information from the visual response to contrast information. As a result, Shapiro, D'Antona, Charles, et al. (2004) developed a stimulus configuration, referred to as “contrast asynchronies,” that allows for the identification of the contrast response. The principle underlying this type of stimulus is illustrated in Figure 1 (see also Movie 1). The display consists of two disks, one surrounded by a light ring, and the other by a dark ring. The luminance levels of the central disks modulate in time and always are identical to each other. The contrast levels (i.e., the difference between the luminance of the disk relative to the surrounding ring) modulate in antiphase. At 1 Hz modulation observers have the paradoxical perception that the disks modulate out of phase with each other (i.e., the percept follows the contrast information) yet also become light and dark at the same time (i.e., the percept follows luminance information). At higher temporal frequencies, the contrast response predominates and the disks only appear to modulate out of phase with each other. 

Figure 1

 

(See demonstration program.) The configuration for the two-disk contrast asynchrony for changes in luminance (from Shapiro, D'Antona, Charles, et al., 2004). The luminance levels of the center lights are modulated in time. The luminance and/or chromatic information modulates in phase, and the contrast information modulates in antiphase. At 1 Hz, observers can perceive both chromatic/luminance information and the color contrast information. The demonstration program contains allows the phase of the disks to be shifted. When the luminance is out of phase and contrast is in phase, the disks appear to modulate synchronously, even though they are not the same color. Most models of simultaneous contrast would not make a predication concerning the apparent synchrony of the modulation.

Other studies have, of course, examined the effects of surrounding fields on the appearance of a colored patch—a phenomenon referred to as simultaneous contrast (e.g., Chevreul, 1839/1854/1967; Goethe, 1810/1840/1970; Helmholtz, 1867/1924; Hering, 1905/1964). Simultaneous contrast is typically studied with either an asymmetric matching technique (see Brainard, 2003) or with a nulling technique (e.g., Krauskopf, Zaidi, & Mandler, 1986) in which a surrounding field modulates in time, and the observer adjusts the phase of the center modulation so that the appearance of the center disk remains constant. Both asymmetric matching and nulling techniques ask observers to make judgments concerning the color of the center patch. 

In this paper I will not examine the changes in the appearance of the center disks, but instead concentrate on the visual response to the contrast information. For example, Figure 2 shows two frames of the contrast asynchrony display, with two disks in their white-most phase of modulation and two disks in their black-most phase of modulation. There are two main observations: (1) the disks of the same luminance level do not appear the same shade (e.g., the white disk with the white ring appears a different shade than the white disk with the black ring); and (2) the disks at the diagonals appear to have similar contrast levels. Most studies of simultaneous contrast have examined changes in apparent color or lightness (i.e., Observation 1); the contrast asynchrony examines the relative contrast (i.e., Observation 2). 

Figure 2

 

Two frames of the contrast asynchrony. The disks in the top row are in the light phase; the disks in the bottom row are in the dark phase. There are two main observations: (1) the disks of the same luminance level do not appear the same shade (i.e., the light disk with the white ring appears a different shade than the white disk with the black ring, and same with the dark disks); and (2) the disks at the diagonals appear to have similar contrast levels. Most studies of simultaneous contrast have examined changes in appearance (i.e., Observation 1). In this study, the observer nulls the contrast responses (Observation 2) by eliminating the asynchronous appearance even though the disks may have a different shade. Shapiro and Hamburger (2007) have shown that scene organization can be determined by the contrast response.

There are several reasons to believe that the asynchronous appearance is fundamentally different from the changes in appearance measured in simultaneous contrast experiments. First, when the center disks modulate 180 deg out of phase with each other, the luminance information modulates in antiphase, and the contrast levels modulate in phase. In this configuration, which can be seen in the interactive display in Figure 1 by clicking on the antiphase button, the disks appear to modulate synchronously—that is, they modulate with the contrast information—even though the disks appear to alternate in color. Standard models of the effects of simultaneous contrast would certainly predict that the lightness of the center disks would modulate in antiphase (one disk appears white, while the other appears black, and vice versa), but they would not make a prediction concerning the apparent synchrony of the modulation. 

Second, the asynchronous appearance can be disrupted in ways that do not affect the color or brightness of the modulating field. In Shapiro, Charles, and Shear-Heyman (2005), Figure 11a shows two rectangles that appear to modulate in antiphase even though they have the same luminance; however, when thin white and black lines are added to the short sides of the rectangles, the asynchrony disappears, and the fields appear to modulate in phase. The disruption of the asynchronous appearance can be created by edges as thin as 10 min of arc, about the size of a ganglion cell receptive field center (Derrington & Lennie, 1982). Standard models of simultaneous contrast do not address the elimination of the asynchrony because the color and brightness of the center fields are not affected by the addition of the edge. Furthermore, many models of simultaneous contrast have fairly broad spatial weighting functions for the effects of the area of the surround on the appearance of the center (e.g., Zaidi, Yoshimi, Flanigan, & Canova, 1992; Zaidi & Zipser, 1993), so two 10-min wide bars should not create dramatic changes in the lightness of the disks. 

Third, simultaneous contrast is a slow process, but the contrast asynchrony is fast. The amount of color induction produced by a modulated surround ring on a central disk decreases with the temporal frequency; the induced effect is nearly absent at 3 Hz modulation (Aurtrusseau & Shevell, 2006; DeValois, Webster, & DeValois, 1986; Rossi & Paradiso, 1996). The contrast asynchrony, on the other hand, is most evident when the center disk modulates between 3 and 6 Hz (Shapiro, D'Antona, Charles, et al., 2004). Shapiro, D'Antona, Charles, et al. (2004) examined the temporal response for achromatic lights; the temporal response for equiluminant lights will be examined in this paper. 

Lastly, the contrast response can be separated from the luminance response for a number of visually important tasks. Shapiro and Hamburger (2007) showed that gestalt pairings occur for contrast information even though contrast cuts across standard figure/ground divisions. For instance, a row of modulating disks placed on a gradient background produces the appearance of 2nd-order motion that sweeps across the disks; the motion arises because of phase shifts in the contrast information. The visual system groups the 2nd-order motion across rows of disks even though each row of disks changes from light to dark at the same time (and should therefore be grouped by the gestalt principle of similarity). The results from Shapiro and Hamburger imply that the response to contrast information per se may serve as a driver for many processes that organize the visual scene. 

The contrast response in color space

In this paper, I will examine the contrast response in a cone-based color space. Many studies of color vision define lights within a physiologically based color space (Derrington, Krauskopf, & Lennie, 1984). Krauskopf (1999) published a review of how these spaces can be created. Figure 3 shows three color planes from a DKL color space: one equiluminant (L–M vs. S) and two chromoluminant (L–M vs. Lum and S vs. Lum). Lights that change along the axes of these planes isolate particular post-receptoral opponent channels. These axes are termed the cardinal axes, and the physiological systems tuned to the lights that change along these axes are often referred to as the cardinal color mechanisms. Lights that change along the intermediate lines stimulate combinations of the cardinal mechanisms. 

Figure 3

 

Depiction of three planes in Derrington, Krauskopf, and Lennie (DKL) color space: (A) equiluminant plane with axes L–M and S; (B) chromoluminant plane with axes L–M and Lum; (C) chromoluminant plane with axes S and Lum.

Shapiro, D'Antona, Smith, Belano, and Charles (2004) showed that the asynchronous appearance can be eliminated by changing the color angle of the center disks. In that study, one surround ring was always white, and the other ring was always black (i.e., Φ = 90 deg in one of the chromoluminance planes); the disks were modulated at 1 Hz along a line in the color plane with color angle θ. After each presentation, the observers responded as to whether the disks appeared in phase or appeared out of phase. The disks appeared to be primarily in phase when the modulation was at or near equiluminance (i.e., θ was close to zero), and appeared out of phase when the disks had a luminance component (i.e., θ more than ±10 deg away from zero). Thus, the contrast response was eliminated when θ was about 90 deg away from Φ. 

This paper extends the Shapiro, D'Antona, Smith, et al. (2004) experiment to non-achromatic surrounds (i.e., Φ, instead of being set to 90 deg, takes on values from 0 to 180 deg). To account for the data, I present a model that contains one pathway for color and another for color contrast. I show that (1) the model correctly predicts orthogonal directions in color space for the contrast nulling task; (2) the response of the contrast pathway appears to be faster than the response of the color pathway; (3) the response of the contrast pathway may mediate detection thresholds under some conditions (a finding that can account for some of the effects of surround luminance on temporal sensitivity); (4) the asynchronous modulation can be seen even when the stimulus is blurred; and (5) the contrast asynchrony does not depend on a disk-ring configuration. 

Experiment 1: Setting a null for the contrast response

In this experiment I examine whether the asynchronous appearance can be eliminated when the surrounds are not achromatic. I set the value of Φ to different points within one of the color planes and ask observers to adjust the modulation angle, θ, to a point where antiphase modulation disappears (see interactive Movie 4. 

Shapiro, D'Antona, Smith, et al. (2004) measured the contrast nulls when for achromatic surrounds (i.e., Φ was fixed at 90 deg within the chromoluminance planes). Those results could be explained by independent color and luminance mechanisms. Contrast nulls for surrounds that are placed at intermediate angles (i.e., when Φ is parametrically varied) would necessarily indicate a more complex type of interaction. 

Observers

There were four observers, all between the ages of 18 and 21. The observers had normal or corrected visual acuity and were color normal, as assessed by a Farnsworth–Munsell 100 hue test and an Ishihara plate test. 

Apparatus

The stimuli were presented on a 21-in. Sony Multiscan G520 monitor using a Cambridge Research VSG 2/4 graphics board. Gamma correction was conducted using a Cambridge Research OptiCal photometer and linearization software. Calibration and gamma correction were checked with a Spectroscan 650 spectroradiometer. The viewing distance was 93 cm. 

Color space

Lights were confined to three planes of the DKL color space (the S/Lum plane, the L–M/Lum plane, and the S/L–M plane; see Figure 3). The center of the color planes is labeled W, an achromatic light of 40 cd/m ². The distance along each axis was expressed in threshold units measured for each observer prior to the beginning of the experiment. Threshold measurements used to define the space were made with a spatial four-alternative forced-choice task, in which three of the quadrants contained a chromaticity of W, and the other quadrant contained the test light, which was a change from W in the S, L–M, or Lum directions. 

Procedure

The experiment parametrically manipulated the color angle of the surrounds. The chromaticity of one surround ring was set at color angle Φ, at a distance equal to the perimeter of the color plane (20× threshold). The chromaticity of the other surround ring was at the opposite end of the color plane (i.e., color angle = Φ + 180). There were 12 values of Φ for each of the three color planes (0, 15, 30, 45. 60, 75, 90, 105, 120, 135, 150, and 165 deg). 

The center disks modulated sinusoidally at 1 Hz along a line defined by the color angle θ and an amplitude of 15× threshold. The observer's task was to adjust the angle of modulation ( θ) until the disks no longer appeared to be modulating out of phase (i.e., the observer adjusted the angle to null the contrast response). The adjustment was controlled with the buttons on the keyboard (horizontal arrows ±10 deg; vertical arrows ±1 deg; the observer pressed the enter key when he or she successfully eliminated the antiphase appearance). After each trial, there was a 3-second gap; then, a new trial was presented. 

Results

For most conditions, the contrast signal disappeared at approximately 90 deg away from the line containing the surrounding annuli, thus indicating multiple orthogonal directions in color space. Figure 4 plots the null angle versus the angle of the surround. Each column shows the results for a different color plane, and each row, the results for a different observer. The error bars represent the standard deviations for six settings. Each observer ran the experiment twice. The solid lines indicate θ = Φ + 90 deg; the data closely follow this line. 

Figure 4

 

(See demonstration program.) The angle of contrast nulls (i.e., the angle at which the disks appeared to modulate in phase) vs. the color angle of the surround. Each column indicates the results for a different color plane, and each row, the results for a different observer.

In some conditions, the curve flattens, notably for observer CMC in the S_Lum and L–M_Lum planes, and for observer JBS in the S_Lum plane. The flattening indicates that some observers prefer nulls near the equiluminant axes when the surrounds are near the achromatic axis. The flattening is consistent with the observation that observers differ substantially in the range of lights that they accept as equiluminant (Shapiro, D'Antona, Smith, et al., 2004). 

A detailed examination of the flattening in some of the curves is a project beyond the scope of this paper. The flattening may represent a miscalling of the color angles for these observers. Then again, the flattening can also be modeled by a gain control placed either early or late in the contrast pathway. The gain control model explanation can be compared to recent physiological results (Solomon & Lennie, 2005) that show that chromatic tuning of chromatic–luminance cortical cells depends upon the contrast amplitude in the stimulus. 

Model: Contrast nulls can occur by summing the rectified response from the S, L–M, and Lum channels

Here, I present a model that explains the orthogonal directions found in Experiment 1 without recourse to higher order mechanisms. The model, depicted in Figure 5, relies upon the pooling of contrast responses across the cardinal mechanisms. The first two stages of the model are standard: S-, M-, and L-cone photoreceptors organized into post-receptoral opponent/luminance sites. The new feature of this model is that the cardinal signals separate into two pathways: one represents the response to color information (thin black lines), and the other to contrast information (blue lines). The two pathways are required because both aspects of the stimulus are clearly apparent to all observers when modulation is at 1 Hz, and because a contrast null does not eliminate the chromatic/luminance response. 

Figure 5

 

A rectification–summation model that predicts multiple orthogonal directions in color space. The model contains two separate pathways, one that responds to luminance and chromatic information (thin black line) and one that responds to contrast information (blue line). The figure on the left depicts contrast combinations across paired contrast channels. The model on the right depicts contrast combinations across all color channels. The two versions of the model cannot be differentiated by the data presented in this paper.

In this paper, I concentrate on the contrast response. The output of the contrast channel is calculated as the rectified difference between the post-receptoral response to the center and the post-receptoral response to the surround. I have not specified the spatial aspects of this comparison; in principle, the difference in the responses could arise in a variety of ways, such as a center-surround filter at the edge, or by the comparison of the outputs from a variety of spatial filters at larger scales. The rectification stage is required because experiments concerning temporal frequency, which I discuss below, imply that the contrast response is unsigned. In the final stage of the model, the contrast signals are summed across cardinal pathways. 

The rectification and pooling of contrasts across color channels is similar to the contrast gain control of Chen, Foley, and Brainard (2000a, 2000b), D'Zmura and Singer (1996), and Singer and D'Zmura (1995). The contrast pathway here is different from the contrast gain control because the contrast pathway constitutes a separate pathway, while the contrast gain control modifies the primary signal transmitted to the brain. 

The model is presented in two ways: In the schematic on the left in Figure 5, the model has a single summation site that pools contrast from across all three cardinal mechanisms; in the schematic on the right in Figure 5, the model pools contrast for each combination of cardinal signals. The two versions of the model cannot be differentiated from each other with the current techniques. Both versions of the model have a square rectification as opposed to an absolute value rectification, as used in the D'Zmura and Singer (1996) gain control. The squaring operation makes for cleaner algebra. 

The model predicts that multiple orthogonal directions should occur for the contrast asynchrony paradigm. The derivation of this fact (that uses only algebra and trigonometry) is shown in Table 1. The values of the variables are shown in the table caption. The output of the cardinal channels at white needs to be set to a non-zero value; I have chosen to set this value at 40 for each channel (i.e., at the luminance value of the monitor). 

Table 1

 

The model predicts multiple orthogonal directions in color space. A _s and ϕ equal the amplitude and angle of the surround light; A _c and θ, the amplitude and angle of the center light; ω and t represent the rate of modulation and time; and W represents the value of the center light.

Left disk:
Surround:	L_M response = W + A scos(Φ + π)	(1a);	S response = W + A ssin(Φ + π)	(1b);
Center:	L_M response = W + A ccos( θ)sin( ωt)	(2a);	S response = W + A csin( θ)sin( ωt)	(2b).
Right disk:
Surround:	L_M response = W + A scos(Φ)	(3a);	S response = W + A ssin(Φ)	(3b)
Center:	L_M response = W + A ccos( θ)sin( ωt)	(4a);	S response = W + A csin( θ)sin( ωt)	(4b)
The contrast response equals the pooled squared_difference for each cardinal mechanism; i.e.,
R left = (Equation 1a − Equation 2a) 2 + (Equation 1b − Equation 2b) 2				(5L);
R right = (Equation 3a − Equation 4a) 2 + (Equation 3b − Equation 4b) 2				(5R).
The difference in the contrast response produced by the left and right disks equals
Contrast difference = −4A s + A csin( ω* t)cos( θ − ϕ)				(6)
The time-varying portion of Equation 6 equals zero when θ and ϕ differ by 90 deg

For simplicity, I have limited this description to the equiluminant plane. A similar set of equations could be expressed for any plane in the color space. Equations 1a through 4b ( Table 1) show how to calculate the response of the model to the left and right centers and left and right surrounds. Equations 5L and 5R ( Table 1) show how to calculate the output of the contrast pathway for the left center/surround and the right center/surround. The difference between the contrast signals is calculated by subtracting Equation 5L from Equation 5R. 

The crucial point is that Equation 6 ( Table 1) has one time-varying term, A _csin( ωτ)cos( θ − Φ), and this term equals 0 when abs( θ − Φ) = 90. Therefore, the model predicts that the contrast of one disk and surround should equal the contrast of the other disk and surround whenever the color angle of disk modulation is 90 deg from the color angle of the surround. To demonstrate this, Figure 6 shows the output of the contrast pathway as a function of the particular combination of Φ and Φ. The red line shows the output for the left disk and surround, and the blue line, the output for the right disk and surround. The outputs are identical when Φ = 0 and θ = 90, and when Φ = 45 and θ = 135, and are out of phase otherwise. 

Figure 6

 

The output of the contrast channel for surrounds at 0 and 45 deg and center modulation along the 0, 45, 90, and 135 deg lines. The red line represents the response to the left disk/surround pair; the blue line represents the response to the right disk/surround pair. The contrast responses for each disk/surround pair are the same when the color angles are separated by 90 deg.

Orthogonal directions in color space for 2nd-order information can therefore be derived with only three color channels and without recourse to higher order mechanisms. This does not mean that high-order color mechanisms do not exist. It has yet to be determined whether this model can account for orthogonal directions under other experimental conditions. Indeed, it is possible that higher order directions are confined to the color (i.e., non-contrast) pathways. 

At 3 Hz, the contrast response predominates for both equiluminant and luminance modulation

There are several phenomenological aspects of a contrast asynchrony display that are consistent with a separate rectified contrast pathway. One of these is the temporal frequency at which the disks appear out of phase. 

The sensitivity for S and L–M modulation is relatively low-pass compared to sensitivity for achromatic modulation (for instance, see Shapiro, Hood, et al., 2003). Between 1 and 10 Hz, sensitivity to equiluminant lights decreases by about a factor of 10, whereas sensitivity to achromatic modulation between 1 and 10 Hz increases by about a factor of 3. In addition, Aurtrusseau and Shevell (2006), DeValois et al. (1986), and Rossi and Paradiso (1996) have shown that simultaneous contrast measured with nulling techniques disappear above 3 Hz. 

Given these empirical facts, one might expect that equiluminant contrast asynchrony would become less visible as the frequency of modulation increases. The current model, however, posits that equiluminant contrast merges with the luminance contrast in a single pathway. The contrast response for equiluminant modulation could potentially be faster than that predicted by the temporal response of the chromatic systems (see Shapiro, Hood, et al., 2003). 

In this experiment, I measure the probability of seeing (and perceptual strength of) the antiphase modulation when the disks are modulated along the S, L–M, and Lum axes. Shapiro, D'Antona, Charles, et al. (2004) showed that for achromatic lights, observers are most likely to see the disks in antiphase when the modulation is between 3 and 5 Hz. In this section, I extend Experiment 3 from Shapiro, D'Antona, Charles, et al. to include lights along the equiluminant axes. 

Procedure

The procedure was the same as that in Experiment 3 of Shapiro, D'Antona, Charles, et al. (2004). The independent variable was the frequency of modulation of the center circles. Each presentation lasted for 3 seconds. In a single session, all temporal frequencies were presented in random order, eight times each. The observer responded as to whether the circles appeared to be modulating out of phase (yes or no); on separate trials, the observer rated the perceptual magnitude of the antiphase signal (1 to 5: 5 when the circles appeared entirely out of phase; 1 when the circles appeared either entirely in phase or were modulating too fast to see them as out of phase). Each observer ran each set twice. 

The lights were modulated along the S, L–M, and Lum cardinal axes. The surrounds had chromaticities twenty threshold units away from the mid-white. The maximum amplitude of modulation was fifteen threshold units. Observers 1 and 2 are the same observers from Shapiro, D'Antona, Charles, et al. (2004) and ran the S and L–M lines 2 weeks following the achromatic experiment. The achromatic data for these observers are replotted from Shapiro, D'Antona, Charles, et al. Observer 3 ran all three conditions in ABCCBA order; this observer used a different range of temporal frequencies. 

Results and conclusion

The top row of Figure 7 shows the results from three observers on the yes/no procedure. The results are similar for three observers. The proportion of time the observers said that the center lights are out of phase is plotted versus the temporal frequency of modulation; each colored symbol represents modulation along a different axis. For all three lights, observers responded yes most frequently when the stimulus was between 3 and 5 Hz. Above 6 Hz the observers seldom saw the stimulus in antiphase. The lights at these frequencies were clearly flickering, but the observers could not tell the phase relationship. There was no distinction between the separate color mechanisms for the frequency of maximum asynchronous response. The response dropped to near zero at 1 Hz; previous demonstrations had shown that observers reported seeing the stimulus in antiphase at 1 Hz. A similar pattern is shown for the rating estimates (the bottom row of Figure 7). The results indicate that for both chromatic and equiluminant lights, the antiphase appearance predominates at higher temporal frequencies. This result can be verified using interactive demonstration 1. Set the modulation of both the disk and the surround to equiluminance. Notice that at 1 Hz it is possible to track the in-phase modulation, but at 3 Hz this is not possible. 

Figure 7

 

(See demonstration program.) Proportion of trials viewed as modulating in antiphase as a function of temporal frequency (top panels). Relative strength of antiphase to in-phase appearance as a function of temporal frequency measured with a rating procedure (bottom panels). Modulation was along S (blue), L–M (green), and Lum (black) cardinal axes. Each column represents the results for a different observer.

A rectified contrast signal may mediate some discrimination thresholds

One of the implications of a contrast pathway is that, for some stimulus conditions, the contrast response will modulate at twice the frequency of the chromatic modulation. For instance, a disk that modulates at 1 Hz against a mid-level background will have a contrast response that modulates at 2 Hz, whereas a disk that modulates against a black (or white) background will have color and color contrast responses that modulate with a fundamental frequency of 1 Hz. 

Here, I replot data from Shapiro, D'Antona, Charles, et al. (2004) to show that a rectified contrast pathway may mediate discrimination thresholds for temporally modulated lights. Shapiro et al. measured temporal sensitivity curves for lights against a mid-level background and against saturated or light and dark backgrounds. The task was a four-alternative forced-choice. The observer had to identify which of four spatial locations contained a light that modulated at a particular temporal frequency. The fields were surrounded by a white, a black, or an equiluminant edge. 

In Figure 8, panels A and C show the data with the axes of a conventional temporal sensitivity function, i.e., sensitivity plotted versus the temporal frequency of the modulated disk. The data replicate the well-documented observation that sensitivity decreases when the modulating field is surrounded by a light or dark field (see Watson, 1986). The standard explanation is that the decrease in sensitivity arises from some form of lateral inhibition from the surrounds (Kelly, 1959), although others have suggested that detection is mediated by contrast under some conditions (Brown, 1965; Shapiro, D'Antona, Charles, et al., 2004; Spehar & Zaidi, 1997). 

Figure 8

 

Temporal sensitivity measurements for fields surrounded by equiluminant edges (black circles), white edges (open squares), and dark edges (open diamonds). The panels on the left (A and C) show the thresholds plotted as a function of the frequency of luminance modulation of the test lights, and the panels on the right (B and D) show the thresholds plotted as a function of contrast modulation of the test lights.

In Figure 8, panels B and D show the same data replotted with the frequency of contrast modulation on the X axis. In this representation, a light surrounded by mid-level field will modulate at twice the frequency compared to a light surrounded by a white or black background. In this representation the curves for gray and white backgrounds fall on top of each other. The data suggest that, under some conditions, threshold is mediated by a rectified contrast pathway, thus offering support for the two-pathway model. It would be worth investigating the range over which such a relationship holds; it is probable that at low temporal frequencies and for equiluminant discrimination, threshold may be mediated by the color/luminance pathway and not the contrast pathway. 

The chromatic contrast response does not depend on a high spatial frequency edge

The model presented here does not specify the spatial scale over which contrast is computed. Shapiro, D'Antona, Charles, et al. (2004) showed that an antiphase appearance can occur when the rings that surround the modulating disks are thin, suggesting that the contrast signal can originate at the edge. Here, I demonstrate that the antiphase modulation can also be created when the disks and/or the surrounding rings are blurred. The asynchronous response does not necessarily depend on high spatial frequency information. 

Demonstration

In the interactive demonstration in Figure 9, the blue button applies a Gaussian blur to the center disks and to the surrounding rings. 

Figure 9

 

(See demonstration program.) An interactive demonstration that shows that the antiphase appearance can be created with blurry disks and surrounds. The contrast response therefore does not depend upon the high spatial frequency components in the edge.

What to notice

1.  
   When the blur is applied to the center and surround, the disks still appear to modulate in antiphase (i.e., the contrast asynchrony occurs when the disks are blurred and when they are not blurred).
2.  
   The antiphase appearance can be eliminated by changing the color angle of the surround relative to the center.
3.  
   A substantial negative afterimage can be created by fixating on the blurred surrounds for a short period of time and then clicking on the “add/remove surround button.” The afterimage will disappear after the next saccade. While the afterimage is present, the blurry disks will appear to modulate in antiphase. As soon the afterimage disappears, the disks appear to modulate in phase.

Conclusion

The contrast asynchrony can be created without high spatial frequency information. Shapiro, D'Antona, Charles, et al. (2004) and Shapiro et al. (2005) showed that the contrast asynchrony can also be created with very thin edges (10 min of visual angle). It is possible therefore that there are multiple contrast signals, any of which can lead to an asynchronous perception. The contrast response can also be created from signals that originate in the afterimage. 

The asynchrony does not depend on a disk-ring configuration

Disk-ring configurations are spatially complex in that they produce energy at multiple spatial frequencies. In this section, I demonstrate that contrast asynchronies can also be generated in other configurations. These configurations may be more suitable for future investigations of the spatial properties of the contrast response. 

Demonstration

In the interactive demonstration in Figure 10, two thin bars modulate in front of a grating. The observer has control over the bars' horizontal location, tilt angle, and temporal frequency and the grating's spatial frequency. 

Figure 10

 

(See demonstration program.) An interactive demonstration that shows two thin modulating bars placed in front of a sine-wave grating.

What to notice

1.  
   The luminance levels of the bars modulate in phase with each other and are always identical. In the interactive demonstration, adjust the position of the bars so that their position overlaps. The bars modulate in unison because they always have the same luminance levels.
2.  
   The bars can be made to appear to modulate out of phase (just like the standard two-disk contrast asynchrony). In the interactive demonstration, adjust one bar so that it sits in front of a black phase of the grating and the other bar so that it sits in front of a white phase of a grating. The bars will now appear to modulate out of phase.
3.  
   The antiphase appearance occurs for background gratings of different spatial frequencies. In the interactive demonstration, use the scroll menu to select the spatial frequency of the grating. Adjust the bars so that one bar is in front of a black portion of the grating, and the other bar is in front of a white portion of the grating.
4.  
   At low temporal frequencies the bars appear to modulate out of phase but get light and dark at the same time. As with the disks/ring configuration, this effect can be best demonstrated by repeating the procedure described in the Introduction (i.e., have half of a group of observers attend to one bar, and half attend to the other; ask both groups to say “white” when the bar they are attending to appears white).
5.  
   A bar will appear to modulate more quickly when it is placed against a gray background than when it is placed against a white or black background. Adjust the grating to a low spatial frequency. Place one bar on black (or white); place the other bar on the gradient between white and black. The bar on the gradient will appear to modulate faster than the bar on white or black. This is consistent with a rectified contrast response.
6.  
   If the bars are tilted and placed across a luminance gradient, the contrast pattern appears to spread across the bars. The shading that sweeps across the bars is similar to grating induction (Foley & McCourt, 1985; McCourt, 1982) and is separate from the contrast asynchronies.
7.  
   If the bars modulate in antiphase (click on the “modulate in antiphase” button) and if one bar is placed against the light part of the grating, and the other is placed against the dark part of the grating, then the bars will appear to modulate synchronously even though they are not the same shade. This effect is the same as that shown for antiphase modulation in the disk-ring configuration.

Conclusion

The color/luminance and contrast responses can be generated by bars placed against a grating; therefore, the contrast asynchrony does not depend on the disk-ring configuration. It seems as though it would be hard to account for the observation that the bars get light and dark at the same time in terms of a single low spatial frequency channel. The increase in flicker rate against the grey backgrounds gives further support for a rectified contrast pathway. 

General discussion

I have shown that the antiphase appearance of modulated disks can be eliminated if the color direction of modulation is orthogonal to the colors of the surrounding rings. I explain this finding with a model that has a contrast channel that is separate from the standard color/luminance channels. The contrast channel sums across the rectified response of all color channels, appears to be faster than the response generated from the center of the field, and may mediate discrimination under some stimulus conditions. 

The results presented here (and the aspects of contrast asynchrony outlined in the Introduction) imply that the asynchronous contrast response is distinct from effects measured by asymmetric matching and nulling techniques. The model therefore does not have direct implications for changes in appearance produced by simultaneous contrast. However, the form of the contrast pathway is similar to the contrast gain control of D'Zmura and Singer (1996), so I would not be surprised if the contrast pathway was related to the mechanisms of color induction. That being said, the interaction between the contrast pathway and the chromatic pathway is bound to be complex (see for instance, Shapiro et al. (2005, Figure 11a), and the asynchronous modulation often appears as a shaded field that overlays the chromatic centers. I therefore suspect that interaction between the two pathways will be related to color transparency as much as, if not more than, to color induction. 

Orthogonal directions in color space and contrast adaptation

For much of the twentieth century, psychophysical models of color vision consisted of two stages: three linear transducers (the S, M, and L cones) followed by two opponent combinations and one additive combination of cone responses (for a review, see Kaiser & Boynton, 1996). Two-stage models of color vision, however, cannot account for multiple orthogonal directions in color space (Clifford, Spehar, Solomon, Martin, & Zaidi, 2003; D'Zmura, 1991; D'Zmura & Knoblauch, 1998; Gegenfurtner & Kiper, 1992; Hansen & Gegenfurtner, 2006; Krauskopf & Gegenfurtner, 1992; Krauskopf, Williams, Mandler, & Brown, 1986; Krauskopf, Wu, & Farrell, 1996; Krauskopf et al., 1986; Li & Lennie, 1997; Lindsey & Brown, 2004; McGraw, McKeefry, Whitaker, & Vakrou, 2004; Webster & Mollon, 1991, 1994; Zaidi & Shapiro, 1993). Multiple orthogonal directions in color space are found when noise, distracters, modulating surround lights, or adaptation lights are presented along a color line intermediate to the cardinal directions. So, for instance, noise presented along a 45° color line will create maximal disruption along a 45° line, but minimal or no disruption for lights along a 135° line. Classic two-stage theories can not account for such findings because such models always predict maximal disruption along the cardinal axes. 

Multiple orthogonal directions in color space are typically accounted for by models that posit higher order color mechanisms. In these models, the outputs of the cardinal mechanisms additively combine so as to create a new layer of detection mechanisms. These mechanisms respond maximally to color changes at angles between those of the S and L–M axes and are therefore said to be directionally tuned to particular angles in color space. Models that contain higher order mechanisms are roughly consistent with the chromatic tuning of cells in the visual pathway. The chromatic tuning of the LGN cells (and early cortical cells) clusters around the cardinal axes. The chromatic tuning of V1, V2, and V3 neurons is distributed uniformly throughout the color plane (V1: Johnson, Hawken, & Shapley, 2001, 2004; Kiper, Fenstemaker, & Gegenfurtner, 1997; Lennie, Krauskopf, & Sclar, 1990; Wachtler, Sejnowski, & Albright, 2003). A number of fMRI studies have found a substantial amount of activity in the visual cortex for stimuli that respond purely to chromatic modulation (Engel & Furmanski, 2001). 

Here, I have shown that for asynchronous contrast response, orthogonal directions in color space can be created by a rectified contrast pathway. It is an open question as to whether a model with a contrast pathway (or a model that combines contrast pathways with chromatic pathways) can be expanded to account for orthogonal directions following contrast adaptation. The most direct approach is to test whether simple mathematical modifications to the model can produce orthogonal directions. In this approach contrast adaptation could be considered to affect the model at various sites, such as, (1) multiplicative gain controls placed early (prior to contrast addition); (2) multiplicative gain controls placed late in the contrast pathway; (3) changes in the shape of a response non-linearity; and (4) changes in the weighting of the pathways for an detection decision rule. The amount of change in each of the sites would be determined by the color direction of adaptation. 

To be clear, I am not suggesting that higher order color mechanisms do not exist, but that a contrast pathway might be integrated into the model to account for some effects of contrast adaptation. It is possible, for instance, that the chromatic pathways lead to multiple higher order mechanisms, whereas the contrast pathway simply pools the contrast from each system, as shown in the model. There is circumstantial evidence in support of this idea. The original cardinal direction study (Krauskopf, Williams, & Heeley, 1982) study found evidence for the primacy of the cardinal directions (i.e., the only orthogonal directions were the cardinal axes) in addition to higher order mechanisms, but many subsequent psychophysical studies have not. 

The discrepancy between Krauskopf et al. (1982) and subsequent studies may have to do with the specific stimulus conditions that have reduced the contrast response. In the Krauskopf et al. experiments, the test and adapting lights were small disks against a dark background. In all subsequent studies, the test and adapting lights have been surrounded with equiluminant backgrounds. These conditions produce markedly different effects on the contrast pathway (see Figure 8). Contrast adaptation studies with a black background may therefore be quite different from those measured against an equiluminant background. 

Indeed, the contrast pathway proposed here has many similarities to the cone-ratio detection model posited by Sankeralli and Mullen (1999, 2001) and Sankeralli, Mullen, and Hine (2002) to account for detection thresholds. The contrast pathway appears to mediate detection thresholds under some conditions (see Figure 8). It is therefore possible that a model that contains separate chromatic and contrast pathways may account for discrepancies concerning when (or if) higher order mechanisms are required to account for discrimination data (Knobloch & D'Zmura, 1997; Sankeralli & Mullen, 1997). 

As discussed in the Introduction, a number of studies have shown that the response to contrast modulation is much faster than would be predicted by the temporal response of chromatic systems (Shapiro, Beere, et al., 2003; Shapiro, Hood, et al., 2003; Webster & Wilson, 2000; Zaidi et al., 1998). Shapiro, Hood, et al. (2003) concluded that the system that controls contrast adaptation was not the same as the color pathways which carried the signal that was ultimately detected. In the present model, it is possible that the fast contrast adaptation represents, in part, an adaptive response of the contrast channel, whereas a slow temporal response represents a response of the chromatic pathway (which includes and achromatic response). Such a two-process system would be consistent with measurements of recovery following contrast adaptation (Hughes & DeMarco, 2003; Shapiro, Beere, et al., 2003). 

Physiological interpretations: Do the chromatic and contrast color signals serve separate visual functions?

Color representation in the cortex is at the heart of an active controversy. Some visual scientists argue that a special subset of cells in the primary visual cortex (dual-opponent cells of area V1) are specifically designed to report color (Conway, Hubel, & Livingstone, 2002; Livingstone & Hubel, 1987, 1988). In this view, the brain processes color separately and then, at some later stage of processing, integrates this information with other processes that determine form or motion. Other investigators point out that most V1 cells that respond to color also respond to luminance (Johnson et al., 2001, 2004); color cells therefore do not represent color per se but combine information from attributes such as color, orientation, and motion in order to solve complex image-processing problems (e.g., color as a cue to visual structure) (Gegenfurtner, 1997; Lennie, 1999; Shapley & Hawken, 2002). 

In principle, the results presented here could be used to support the classic version of color perception: The chromatic pathway creates a chromatic map (possibly mediated through double-opponent cells), and the contrast pathway creates another map of the world. The contrast pathway could be derived in a number of ways: for instance, from an unequal weighting of cone signals that create a continuum of contrast-sensitive cells that respond both to color and to luminance information (Hurlbert, 2003). The two maps could then be combined at some later (binding) stage of processing to create a colored visual perception of the world. 

There are, however, a number of other ways in which separate responses to chromatic and contrast information could arise. One possibility is that the chromatic signal is constructed at a later stage than the contrast signal. Liu and Wandell (2005) recently used fMRI techniques to show that color signals encoded in human motion-selective cortex (MT+) respond equally well to 10 Hz and 1 Hz modulation, and those in the ventral occipital cortex (VO) respond best to low-frequency modulation. One speculative idea is that the 2nd-order response is processed through MT+, and the chromatic response is processed through VO. This interpretation is consistent with the idea of two different types of color signals, that the contrast response is faster than the chromatic response, and that there would achromatic responses in VO. 

Ioannides, Johnston, and Griffin (2006) put forward a quantitative model that describes how the chromatic response could potentially be constructed from processes that respond to contrast and higher order statistics in the visual image. In their approach, the visual system needs to represent a visual image described by F(x,y). This image can be decomposed by a Taylor series expansion of a fundamental term followed by a series of higher order derivatives. The early stages of visual physiology are capable of presenting the higher order derivatives but must estimate (or construct) the fundamental term. The fundamental term in the Taylor series expansion would correspond to what I have been calling the chromatic signal. 

An account in which a chromatic response is derived from contrast (and higher order) response is compelling because it gives a functional purpose to the separate pathways proposed in the model. For instance, the slower processing of the chromatic response in the contrast asynchrony could arise because of the extra processing needed to calculate the fundamental term. If this is correct, then relatively slow color response in area VO could be the result of the neural calculation of color and brightness from the response to contrast (and higher order) information. This approach is also consistent with findings that show a central location for color contrast effects (Shevell & Wei, 2000) and with the idea that processes that respond to different sources of chromatic information do so to serve different perceptual functions (Gegenfurtner & Hawken, 1996; Gegenfurtner & Kiper, 2003; Koida & Komatsu, 2007; Thiele, Dobkins, & Albright, 1999). 

Yet another reason for the visual system to separate chromatic and contrast information is implied by statistical regularities in the environment. Frazor and Geisler (2006) have shown, at least for achromatic information, that there is little correlation between local contrast and local luminance information from one fixation to the next. They suggest that efficient processing implies contrast gain control mechanisms that operate largely independently of luminance gain control mechanisms; Mante et al. (2005) have found physiological evidence for such independent processes in the LGN. The model presented here is consistent with this framework: Separate chromatic and contrast responses allow for an efficient representation of statistically independent aspects of the scene. Indeed, it may be that the chromatic response an contrast responses are separable in the same manner that orientation and contrast responses are separable, as was recently demonstrated by Geisler, Albrecht, and Crane (2007). The Geisler et al. results are particularly intriguing because not only were luminance and contrast separable, but the luminance response had a substantially slower time course than the contrast response. 

Do the two pathways simply represent the response of different spatial frequency channels?

In the model presented here, the contrast pathway is derived from of a simple comparison between two points: I have not specified any of the spatial parameters. One possible objection to the current model is that it could be recast in terms of spatial frequency channels: The contrast (antiphase) response is the result of a high spatial frequency channel, and the chromatic (in phase) response is the result of a low spatial frequency channel. 

While I believe that eventually the model will have to be expanded and integrated into a more sophisticated spatial channel/texture theory, there are a number of issues that make it unclear how such an expansion would be accomplished in a straightforward manner. First, the most obvious interpretation of the contrast asynchrony would suggest that the spatial frequency channels that primarily respond to the center would need to be slower than those that response to the edges. However, the response to chromatic information is slower than the response to contrast (see Figure 7). A slow, low spatial frequency response would contradict evidence that low spatial frequency channels have a faster temporal response than high spatial frequency channels (Watson, 1986). Second, Figures 9 and 10 show that perceptual asynchronies can occur with a blurred image or with thin lines. Thus, it is possible that the contrast response takes place simultaneously over many different spatial scales. Third, the contrast response can be inhibited by thin lines. Shapiro et al. (2005, Figure 11a) showed that when thin lines are added to the edges of modulating rectangles, the rectangles appear to modulate in phase instead of in antiphase. This behavior is similar to the “contrast blocking” effect of texture described by Hurlbert and Wolf (2004). 

It is unlikely therefore that there will be a simple description of the contrast asynchrony in terms of spatial frequency channels. Nonetheless, ongoing experiments in my laboratory are intended to give insight into the spatial parameters controlling both single-field and two-field contrast asynchronies. We have created a single-field contrast asynchrony in which the apparent motion shifts directions depending on the frequency content of the flanking edges. The results suggest that contrast asynchronies can be created simultaneously at different spatial scales. 

Concluding comments

Hundreds of studies have shown that color contrast is a fundamental source of information for color vision: Color contrast is important for visual adaptation (see D'Zmura & Singer, 1999; Zaidi, 1999) and for color constancy (see Brown, 2003; Hulrbert & Wolf, 2004), and can create dramatic changes in color appearance (Chevreul, 1839/1854/1967). Nonetheless, contrast has often been marginalized in many descriptions of color vision. 

Whittle (2003) notes that relatively recent comprehensive reviews of color vision (Abromov & Gordon, 1994; Kaiser & Boynton, 1996) mention contrast only as a “minor side-effect.” As Whittle points out, this situation is not without reason; although color contrast has been shown in the laboratory to be an important aspect of perception, the colors of everyday objects do not appear to depend on their relationship to the color of the background: “If you move a coloured object above a variegated background, you will usually see little change in its colour.” As a result, many of the important aspects of color contrast—such as Von Kries adaptation—have been “discovered, forgotten, re-discovered, proved and disproved in different contexts.” Until color theory recognizes the basic fact that the visual representation of color is not a single-valued function, we will continue to repeat this cycle. 

In this paper, I have developed a quantitative model that explicitly represents a separate contrast pathway. One implication of the model is that the visual system always has access to both chromatic/luminance information and contrast information. Questions concerning how context influences color perception can be recast into questions concerning when (and how) one class of signal predominates over the other, and how these two classes of signal are integrated into perception of a visual scene. In this view, then, long-standing issues of color perception (such as chromatic discrimination, color appearance, and color constancy) can be considered in terms of weighted combinations of separate color and color contrast maps, each of which extracts different sources of information from the environment. 

Supplementary Materials

Movie 1 - Supplementary File 

Movie 4 - Supplementary File 

Movie 7 - Supplementary File 

Movie 9 - Supplementary File 

Movie 10 - Supplementary File 

Acknowledgments

The author thanks Sherri Geller for her editorial assistance. Justin Charles and Anthony D'Antona were in the laboratory and participated in discussions concerning many of the ideas in the manuscript. 

Commercial relationships: none. 

Corresponding author: Arthur G. Shapiro. 

Email: [email protected]. 

Address: Department of Psychology and Program in Neuroscience, Bucknell University, Lewisburg, PA 17837, USA.