Interactions between color and luminance in the perception of orientation

Abstract

At the early stages of visual processing in humans and other primates, chromatic signals are carried to primary visual cortex (V1) via two chromatic channels and a third achromatic (luminance) channel. The sensitivities of the channels define the three cardinal axes of color space. A long-standing though controversial hypothesis is that the cortical pathways for color and form perception maintain this early segregation with the luminance channel dominating form perception and the chromatic channels driving color perception. Here we show that a simple interaction between orientation channels (the tilt illusion) is influenced by both chromatic and luminance mechanisms. We measured the effect of oriented surround gratings upon the perceived orientation of a test grating as a function of the axes of color space along which the gratings were modulated. We found that the effect of a surround stimulus on the perceived orientation of the test is largest when both are modulated along the same axis of color space, regardless of whether that is a cardinal axis. These results show that color and orientation are intimately coupled in visual processing. Further, they suggest that the cardinal chromatic axes have no special status at the level(s) of visual cortex at which the tilt illusion is mediated.

Introduction

Anatomical and electrophysiological evidence suggests that chromatic and achromatic signals are carried in distinct divisions of the retino-geniculo-cortical pathway (Hubel & Wiesel, 1966; De Valois, Abramov, & Jacobs, 1966; Derrington, Krauskopf, & Lennie, 1984; Hendry & Yoshioka, 1994; Martin, White, Goodchild, Wilder, & Sefton, 1997; De Valois, Cottaris, Elfar, Mahon, & Wilson, 2000). The signals in the two chromatic channels correspond to modulation of the response of the S cones and the difference between L- and M-cone activation. The achromatic (luminance) channel is derived from additive combination of cone signals. The chromatic sensitivities of the channels define the cardinal axes of a three-dimensional color space (Derrington et al., 1984), illustrated in Figure 1. Livingstone and Hubel (1984, 1987, 1988) linked this early segregation to neurochemical compartmentalization of primate primary visual cortex (V1), such that neurons in cytochrome oxidase-rich regions (blobs) are specialized for color processing. Interblob regions are dominated by neurons with high orientation selectivity but poor color specificity. Livingstone and Hubel (1984, 1987, 1988) proposed that the chromatic channels provide the inputs to mechanisms for color perception, but contribute little to early mechanisms for orientation processing. However, more recent physiological evidence shows that in macaque V1 a significant proportion of cells respond maximally to combined modulation of color and luminance (Thorell, De Valois, & Albrecht, 1984; Lennie, Krauskopf, & Sclar, 1990; Leventhal, Thompson, Liu, Zhou, & Ault, 1995; De Valois et al., 2000), including cells showing a high degree of orientation-selectivity (Johnson, Hawken, & Shapley, 2001). This raises the possibility that color and luminance interactions are a feature of early visual processing. 

Figure 1

 

Chromatic sensitivities of the channels defining Derrington-Krauskopf-Lennie space (Derrington et al., 1984).

The chromatic selectivity of orientation processing in the human visual system has been investigated psychophysically by measuring spatial and temporal interactions in orientation perception (Livingstone & Hubel, 1987; Flanagan, Cavanagh, & Favreau, 1990). It is well known that perceived orientation is affected by the simultaneous presence of an oriented surround: the tilt illusion (TI), illustrated in Figure 2 (Gibson & Radner, 1937). For small inducer-test angles, the perceived orientation of the test is repelled away from that of the inducer, with the maximum effect occurring for an orientation difference of around 15° between inducer and test. Adaptation to an oriented stimulus also affects subsequent orientation perception (Gibson & Radner, 1937), a phenomenon known as the tilt aftereffect (TAE). The magnitude and direction of the TAE and TI show a very similar dependence on the relative orientation of inducing and test stimuli (Gibson & Radner, 1937; Wenderoth & Johnstone, 1987; Clifford, Wenderoth, & Spehar, 2000). 

Figure 2

 

The tilt illusion. A vertical test patch appears repelled in orientation away from a surround oriented at 15°. The magnitude of the tilt illusion is taken as half the difference between the orientation of subjective vertical for ±15° surrounds.

At isoluminance, large TAEs have been found (Flanagan et al., 1990), whereas the TI has been reported to disappear (Livingstone & Hubel, 1987). The latter result has been taken as evidence of a functional separation of color and form processing in human vision (Livingstone & Hubel, 1987). These reported differences are surprising in the light of the otherwise similar phenomenology of the TAE and TI. Here we measured the chromatic selectivity of the TI to quantify the degree of interaction between color and orientation processing in human vision. The results show strong interaction between chromatic and form processing at the psychophysical locus of the tilt illusion. 

Methods

Three of the authors (C.C., B.S., and S.S.) and one experienced observer naïve to the purposes of the study (T.W.) served as subjects. All had normal or corrected-to-normal vision. Stimuli were generated using Matlab software to drive a VSG 2/5 graphics board (Cambridge Research Systems, Rochester, Kent, UK) and presented on a 21″ Sony Trinitron GM 520 monitor with the white point for color space calculations set at CIE chromaticity coordinates (.280 and .303) and a luminance of 66.0 cd/m⁻². 

Prior to the tilt illusion experiments, isoluminant L-M and S-cone isolating axes were determined separately for each subject using a minimum motion technique (Anstis & Cavanagh, 1983). Detection thresholds were measured in each of the three cardinal directions of color space (Derrington et al., 1984). The stimulus was a vertical 1.0 cycle/deg sinusoidal grating in a circular aperture with a diameter subtending 15.0 deg of visual angle. 

For the tilt illusion experiments, each test stimulus consisted of a 1.0 cycle/deg sinusoidal grating in a circular aperture with a diameter subtending 3.0 deg of visual angle. The surround stimulus (also 1.0 cycle/deg) was presented in an annulus with inner and outer diameters of 3.0° and 15.0°, respectively, concentric with the test stimulus. Stimuli were presented in a raised cosine temporal window, such that the stimulus was present at full contrast for 200 ms, and ramping on and off took 100 ms each. Test and surround stimuli were presented at the same multiple of detection threshold. Subjects viewed the screen from a distance of 55 cm. The testing cubicle was dark, and its walls were covered with matt black material to remove any external references to vertical. A chin-rest was used to prevent head movements. 

The experiments followed a forced-choice procedure, such that subjects were required to report via a response box whether the test stimulus appeared tilted clockwise or anti-clockwise from subjective vertical. The subjects’ previous responses were used to determine the physical orientation of subsequent test stimuli according to an adaptive psychophysical procedure under computer control (Kontsevich & Tyler, 1999). In this way, the orientation of subjective verticality was determined in 60 trials for each subject for each stimulus configuration. To control for any biases in perceived vertical that a subject may have, the magnitude of the tilt illusion for a surround orientation of 15° was taken as one half the difference in perceived vertical between interleaved trials in which the surround orientation was 15° and −15°. 

Results

We measured the effect of a surround grating oriented at 15º to the vertical upon the perceived orientation of a central test grating. When both test and surround were modulated along the same axis of color space, we found that some subjects experienced large TIs for subjectively isoluminant stimuli at contrasts 5–20 × detection threshold, whereas others experienced none. At contrasts 30–40 × detection threshold, each subject showed TIs of approximately equal magnitude for the three cardinal axes (Figure 3). 

Figure 3

 

Magnitude of the tilt illusion as a function of stimulus contrast. Test and surround stimuli were modulated along the S-cone isolating (blue triangles), L-M (red squares), and luminance (black circles) directions of color space. Error bars in all figures are ±1 SEM.

When the modulation axis of the surround was varied with that of the test fixed (with contrast equated at 30 × or 40 × detection threshold), significant TIs were consistently observed whether the test stimuli were modulated along cardinal or non-cardinal axes. Each non-cardinal stimulus was constructed to have equal projections along two of the cardinal axes while being orthogonal to the third (Figure 4). 

Figure 4

 

Schematic of the formation of non-cardinal stimuli. Each pair of cardinal components can be combined to produce two distinct stimuli depending on their relative phase.

The maximum TI always occurred when test and surround were modulated along the same axis, whether modulation was along cardinal (Figure 5) or non-cardinal (Figure 6) axes in color space. The interaction between test and inducing color was highly significant (p < .001) for subjects B.S. and T.W. in all conditions. The results for the other two subjects followed a similar pattern (data not shown), though significance levels were generally lower for subjects C.C. and S.S. (Table 1). For any given observer, the magnitude of the illusion was similar for cardinal and non-cardinal stimuli. Selectivity for cardinal and non-cardinal directions in the isoluminant plane was also found for the smaller attractive effects in perceived orientation induced by a 75º surround (Figure 7), but we did not study this effect in detail. 

Figure 5

 

Magnitude of the tilt illusion for test and inducing stimuli modulated along the same or orthogonal directions in color space for subjects B.S. (top row) and T.W. (bottom row). Test and surround stimuli were modulated along each of the three cardinal directions in color space and presented at 40 × detection threshold for B.S. and 30 × detection threshold for T.W. Modulations were along the luminance and L-M axes (left), the luminance and S-cone isolating axes (middle), and the L-M and S-cone axes (right). The color of the test stimulus is labeled on the x-axis. Lines are labeled with the color of the surround.

Figure 6

 

Magnitude of the tilt illusion for test and inducing stimuli modulated along non-cardinal directions in color space for subjects B.S. (top row) and T.W. (bottom row). Each non-cardinal stimulus was constructed so as to have equal projections along two of the cardinal axes while being orthogonal to the third. Modulations were in the plane of color space containing the luminance and L-M axes (left), the luminance and S-cone isolating axes (middle), and the isoluminant plane (right). The two non-cardinal stimuli in each plane of color space differed only in the relative phase of the two projections onto the cardinal axes, and would thus be indistinguishable solely from the responses of mechanisms tuned to cardinal directions of color space.

Figure 7

 

Magnitude of the tilt illusion for a 75°-inducing stimulus averaged across all four subjects. Test and inducing stimuli were modulated along the S-cone isolating and L-M axes of color space (at 30 × detection threshold for S.S. and T.W. and 40 × for C.C. and B.S.). Negative values indicate that the test appeared to be attracted in orientation toward that of the inducer. The interaction between test and inducing chromatic axis was highly significant (p < .005).

Table 1

 

Significance of color-specific interactions between test and surround. Data from four subjects are shown for cardinal and non-cardinal stimuli modulated in each of the three planes defined by pairs of cardinal chromatic axes.

Cardinal			Non-Cardinal
Subject	LMS/L-M	LMS/S	L-M/S	LMS/L-M	LMS/S	sL-M/S
B.S.	.001	.001	.001	.001	.001	.001
T.W.	.001	.001	.001	.001	.001	.001
C.C.	.001	.001	.005	.001	.005	.05
S.S.	.005	.001	.001	.05	n/s	.01

We next quantified the chromatic selectivity of the TI by varying the color of the surround for a fixed test stimulus. For cardinal (Figure 8) and non-cardinal test stimuli (Figure 9), the TI was largest for inducers modulated along the same chromatic axis as the test. For surround stimuli modulated along chromatic axes away from that of the test, the illusion fell to a baseline level between 30% to 70% of its maximum value but always remained above zero. The range of surround colors for which the illusion was stronger than this baseline level was generally restricted to modulation axes within 45° of the test direction in color space. The chromatic bandwidth (half width at half height) of the color-specific component of the best-fitting circular normal functions to the data in Figures 8 and 9 ranged from 15.2° to 44.4° with a mean of 25.1°. The mean bandwidths for cardinal and non-cardinal test stimuli were very similar: 23.9° and 26.2°, respectively. 

Figure 8

 

Magnitude of the tilt illusion for cardinal test stimuli as a function of the direction of modulation in color space of the surround stimulus. Top. Tilt illusion for S-cone isolating test stimulus at 40 × detection threshold for C.C. for inducers modulated in the isoluminant plane (left) and in the plane of color space (right) containing the luminance and S-cone isolating axes. Bottom .Tilt illusion for test stimulus modulated along the L-M axis at 40 × detection threshold for subject B.S. for inducers modulated (left) in the isoluminant plane and (right) in the plane of color space containing the luminance and L-M axes. Solid curves show the best-fitting circular normal functions (see Clifford, 2002). The bandwidth (half width at half height) of the best-fitting functions was 21.5°, 20.3°, 32.4°, and 21.3°.

Figure 9

 

Magnitude of the tilt illusion for non-cardinal test stimuli as a function of the direction of modulation in color space of the surround stimulus. Top. Tilt illusion for non-cardinal S + (L+M+S) test stimulus modulated at 40 × detection threshold for C.C. for inducers modulated (left) in the plane of color space containing the luminance and S-cone isolating axes (right) in the orthogonal plane containing the white point. Bottom. Tilt illusion for non-cardinal (L-M) + (L+M+S) test stimulus modulated at 40 × detection threshold for subject B.S. for inducers modulated in the plane of color space (left) containing the luminance and L-M axes and in the orthogonal plane (right) containing the white point. Solid curves show the best-fitting circular normal functions. The bandwidth (half width at half height) of the best-fitting functions was 22.4°, 15.2°, 22.9°, and 44.4°.

Interaction between non-orthogonal color vectors was still present when a gap of up to 1.0° of visual angle was introduced between test and surround (Figure 10), demonstrating that the chromatic specificity of the effect was not related to any difficulty in segregating test and surround. 

Figure 10

 

Magnitude of the tilt illusion as a function of the width of the gap between test and inducing stimuli for subject B.S. Stimuli were modulated at 40 × detection threshold along cardinal (left) and noncardinal (right) axes. Test and inducer were modulated along either the same axis (solid symbols) or orthogonal axes (open symbols). Red circles show the magnitude of the TI for L-M (left) or (L-M) + (L+M+S) test stimuli (right). These are the same test stimuli as for the data for subject B.S. shown in Figures 8 and 9. Blue squares show the magnitude of the TI for test stimuli that were S-cone isolating (left) or modulated along the (L-M) − (L+M+S) axis (right). Interactions were significant at all gap sizes for cardinal and non-cardinal modulations (p < .05) except for the 1.0° gap with non-cardinal stimuli (p = .07).

Discussion

The reported absence of the TI at isoluminance has been taken to indicate a functional separation of color and form processing in human vision (Livingstone & Hubel, 1987). Here we found that while some subjects showed a reduced TI for isoluminant stimuli at low contrast, the loss of tilt induction at isoluminance is not a general result. At high contrasts, the TI shows considerable color specificity to non-cardinal directions, regardless of whether the modulation is restricted to the isoluminant plane or includes luminance variation. These data suggest that the TI is mediated by neural mechanisms subsequent to the combination of signals from the chromatic and achromatic channels. We have no explanation for the discrepancy between our results and those of Livingstone and Hubel (1987), but note that that study used a somewhat different stimulus configuration involving high frequency rectangular wave gratings (their Figure 29). 

Chromatic Selectivity of the TI and TAE

Flanagan et al. (1990) found that for an adapting stimulus modulated along a cardinal axis of color space the TAE is maximal when the test is modulated along the same chromatic axis and is near minimum for orthogonal axes. For non-cardinal colors, the TAE is smaller than for cardinal adapting colors, and the maximum TAE does not always occur when the test is the same color as the adaptor. This suggests that the color preferences of orientation-selective mechanisms in human vision can be characterized principally by the three cardinal axes of the color space proposed by Derrington et al. (1984). 

Here we found the TI was maximal when the inducer was modulated along the same chromatic axis as the test. The results for the TI presented here thus differ from those for the TAE (Flanagan et al., 1990) in showing that the TI can be just as large for non-cardinal as for cardinal chromatic directions. This suggests that the cardinal chromatic axes have no special status at the level of visual processing at which the TI is mediated. 

The similar angular dependence of the TAE and TI suggests mediation of the two effects by a common mechanism (Wenderoth & Johnstone, 1987). However, the difference in their chromatic dependency indicates a degree of independence. We suggest that the color-specific component of the TI involves the operation of lateral interactions at the cortical level, and thus reflects the diversity of chromatic tuning found in visual cortex (Thorell et al., 1984; Lennie et al., 1990; De Valois et al., 2000; Johnson et al., 2001; Kiper, Fenstemaker, & Gegenfurtner, 1997; Gegenfurtner, Kiper, & Levitt, 1997). 

In contrast, the color-specific component of the TAE might arise from depression of thalamocortical afferent activity. Mechanisms of input-specific synaptic depression have been proposed to operate in primary visual cortex, in addition to adaptive mechanisms that reduce responsiveness to all inputs (Abbott, Varela, Sen, & Nelson, 1997). Given that the chromatic tuning of lateral geniculate nucleus (LGN) neurons clusters around the two chromatic axes of color space, any contribution of input-specific adaptation to cortical color vision would be expected to have its principal effect on the perception of cardinal stimuli. Brain-imaging data suggest that selective adaptation to color contrast occurs in V1 (Engel & Furmanski, 2001). We speculate that V1 is also the site of the intracortical interactions and synaptic depression proposed to underlie the color-specific components of the TI and TAE. 

For inducing angles of ±15°, a repulsive TI was consistently obtained even for surround stimuli modulated along chromatic axes orthogonal to that of the test. This shows that substantial color-insensitive interactions underlie the TI in addition to its color-specific component. Data from the TAE (Flanagan et al., 1990), selective adaptation to color contrast (Engel & Furmanski, 2001), and adaptation-induced shifts in perceived spatial frequency (Hardy & De Valois, 2002) show that these effects also involve a combination of color-specific and color-insensitive components. The color-insensitive component may reflect the operation of higher-level visual mechanisms that code for form in a cue-invariant manner (Hardy & De Valois, 2002). 

Non-Cardinal Mechanisms

Previous studies have also found evidence for non-cardinal chromatic mechanisms in human vision. For isoluminant stimuli, psychophysical adaptation to modulations along non-cardinal directions of color space has its maximum effect on the detection (Krauskopf, Williams, & Heeley, 1982; Krauskopf, Williams, Mandler, & Brown, 1986), discrimination (Krauskopf & Gegenfurtner, 1992), and appearance (Webster & Mollon, 1991) of test stimuli modulated along similar directions. Interactions between color and luminance signals are evident from experiments on color appearance (Webster & Mollon, 1991), contrast detection (Gegenfurtner & Kiper, 1992; Gur & Akri, 1992), and texture segmentation (Li & Lennie, 1997). This specificity for non-cardinal directions requires either mechanisms tuned to non-cardinal directions of color space (Krauskopf et al., 1986) or interactions between cardinal mechanisms that depend on the relative phase of modulations along the axes of color space. Such mechanisms might serve a functional purpose in the efficient coding of chromatic information (Zaidi & Shapiro, 1993; Atick, Li, & Redlich, 1993; Clifford et al., 2000). 

Chromatic Bandwidth

The narrow tuning of the color-specific component of the TI could be the result of cortical mechanisms that sharpen chromatic bandwidth. Neurons in the LGN (Derrington et al., 1984) and a significant proportion of those in V1 (Lennie et al., 1990; De Valois et al., 2000) show cosine chromatic tuning, with a bandwidth of 60°, consistent with linear summation of cone inputs. Broadband linear mechanisms tuned to a range of chromatic directions have also been shown to account for the effects of noise masking on psychophysical color detection (D’Zmura & Knoblauch, 1998). However, Goda and Fujii (2001) found that the discrimination of color distributions in multi-colored textures was best accounted for by narrowly tuned channels with a bandwidth of about 40°. The chromatic bandwidths measured here for the color-specific component of the TI (range: 15.2° – 44.4°; mean: 25.1°) tend to be slightly narrower than those inferred by Goda and Fujii (2001). These chromatic bandwidths are similar to those reported for a subset of cells in V2 (Kiper et al., 1997), an area also known to contain cells responsive to contours defined by non-luminance cues (von der Heydt, Peterhans, & Baumgartner, 1984). 

Coupling of Color and Orientation Processing

Our data provide support for the hypothesis that the processing of color and orientation is intimately coupled (Flanagan et al., 1990; Lennie, 1998). It is widely agreed that the TI can be accounted for by lateral interactions between neurons tuned to similar orientations (Blakemore & Tobin, 1972; Wenderoth & Johnstone, 1987; Clifford et al., 2000). Similar interactions between neurons tuned to different colors might also underlie surround effects on perceived contrast (Chubb, Sperling, & Solomon, 1989; Singer & D’Zmura, 1994). The color-specific component of the TI could result from orientation-specific interactions within a population of neurons selective for color and orientation. Nonlinear combination of color- and orientation-specific interactions would also generate the orientation-specificity of center-surround interactions in perceived contrast (Solomon, Sperling, & Chubb, 1993) as well as providing a possible substrate for chromatic aftereffects contingent on orientation (McCollough, 1965). 

Acknowledgments

This research was supported by a grant from the University of Sydney New Staff Support Scheme. We are grateful to John Ross and Paul McGraw for comments on a draft version of this manuscript, and to Jason Forte and Justin Harris for helpful discussions. Commercial 

Relationships: None.