Orientation-selective chromatic mechanisms in human visual cortex

Abstract

We used functional magnetic resonance imaging (fMRI) at 3T in human participants to trace the chromatic selectivity of orientation processing through functionally defined regions of visual cortex. Our aim was to identify mechanisms that respond to chromatically defined orientation and to establish whether they are tuned specifically to color or operate in an essentially cue-invariant manner. Using an annular test region surrounded inside and out by an inducing stimulus, we found evidence of sensitivity to orientation defined by red–green (L–M) or blue–yellow (S-cone isolating) chromatic modulations across retinotopic visual cortex and of joint selectivity for color and orientation. The likely mechanisms underlying this selectivity are discussed in terms of orientation-specific lateral interactions and spatial summation within the receptive field.

Introduction

At the early stages of visual processing in humans and other primates, chromatic signals are carried to primary visual cortex (V1) via two opponent chromatic channels and a third, luminance channel (Derrington, Krauskopf, & Lennie, 1984). It has been suggested that the cortical pathways for color and form perception maintain this early segregation with the luminance channel dominating form perception and the chromatic channels driving color perception (Felleman & Van Essen, 1991; Livingstone & Hubel, 1988; Zeki, 1978). However, the existence of single neurons selective for both color and orientation in areas V1, V2, and V3 of non-human primates (Gegenfurtner, Kiper, & Fenstemaker, 1996; Gegenfurtner, Kiper, & Levitt, 1997; Johnson, Hawken, & Shapley, 2001, 2008; Lennie, Krauskopf, & Sclar, 1990; Leventhal, Thompson, Liu, Zhou, & Ault, 1995; McClurkin, Optican, Richmond, & Gawne, 1991; Thorell, De Valois, & Albrecht, 1984) argues against a strongly modularized cortical architecture for the processing of these attributes. 

Psychophysically, the chromatic selectivity of orientation processing has been investigated via surround modulation and adaptation; effects known as the tilt illusion and tilt aftereffect, respectively (Gibson & Radner, 1937). In these effects, the perceived orientation of a test pattern is biased in a way that depends systematically on the difference in orientation between test and inducer. Both effects show significant selectivity for the color/luminance congruence of the test and inducer such that the magnitude of the illusion for a chromatic test and luminance inducer (or vice versa) drops to around 50% of the effect, compared to when test and inducer are congruent in color (Clifford, Pearson, Forte, & Spehar, 2003; Clifford, Spehar, Solomon, Martin, & Zaidi, 2003; Flanagan, Cavanagh, & Favreau, 1990). Comparison of same-eye and opposite-eye presentations of test and surround indicates that the monocular component of the tilt illusion is entirely color-selective while the binocular component shows only weak color selectivity (Forte & Clifford, 2005). These results suggest that color and orientation processing interact at monocular stages of visual processing whereas binocular visual mechanisms code for form in a manner that is largely insensitive to the cues carrying the orientation signals. 

The lack of color selectivity of the binocular mechanisms underlying the tilt illusion is consistent with studies showing little interocular transfer of the McCollough effect (Coltheart, 1973; Lovegrove & Over, 1973; MacKay & MacKay, 1973). The McCollough effect can be induced by adaptation to orthogonal square-wave gratings of different colors (e.g., red–black and green–black). The subsequent appearance of an achromatic (white–black) grating depends upon its orientation, with the white strips appearing colored with the complementary hue of the adaptor of the same orientation (McCollough, 1965). This orientation-contingent color aftereffect has been attributed to adaptation of neurons selective for both color and orientation (Vidyasagar, 1976). That the effect occurs only if the adapting stimulus is presented to the same eye(s) as the test grating indicates a difference between processing at monocular and binocular levels. One possibility is that monocular orientation-selective mechanisms are also highly color-selective whereas binocular ones are far less so (Forte & Clifford, 2005). Alternatively, it may be that the binocular mechanisms are less adaptable; however, the existence of strong tilt aftereffects under conditions of dichoptic presentation argues against this (Paradiso, Charney, & Freeman, 1989). 

In human visual cortex, recent evidence from fMRI studies using adaptation (Engel, 2005) and multivariate pattern classification (Seymour, Clifford, Logothetis, & Bartels, 2010; Sumner, Anderson, Sylvester, Haynes, & Rees, 2008) suggests conjoint tuning of color and orientation. Engel (2005) used horizontally and vertically oriented grating stimuli defined by either a light–dark (L + M) or a red–green (L–M) modulation. The responses of V1 and a range of extrastriate areas (V2, V3v, V3d/V3A/V7, V4/V8) were analyzed following adaptation to a parallel or orthogonal grating of the same or different color. All areas showed evidence of adaptation that was jointly selective to color and orientation. 

The stimuli used by Engel (2005) were designed to provide maximal stimulation to the red–green and luminance channels. In contrast, Sumner et al. (2008) used grating stimuli defined by modulations designed to isolate the red–green and luminance channels, rather than to provide maximal stimulation to either. In addition, they used an S-cone isolating stimulus to achieve selective stimulation of the blue–yellow opponent channel. Following the methods of Haynes and Rees (2005), Sumner et al. (2008) used multivariate pattern analysis to establish the ability of areas V1, V2, and V3 to discriminate the orientation of gratings tilted ±45° from vertical. They found that each of these regions of interest performed significantly above chance at classifying orientation not only for luminance stimuli (Haynes & Rees, 2005; Kamitani & Tong, 2005) but also for the two stimuli isolating the respective chromatic channels. Furthermore, they found that training with stimuli of one color and testing with stimuli of another reduced classification performance. Based on this imperfect generalization of orientation classification across stimulus color, they concluded that areas V1–V3 were not only sensitive to orientation defined by isoluminant chromatic modulation but that they were also selective to particular combinations of color and orientation. 

Seymour et al. (2010) provided further evidence of conjoint coding of color and orientation. They used multivariate pattern analysis to discriminate between blocks of different conjunctions of two colors and two forms. Each stimulus block consisted of alternations between red spirals of one sense (clockwise or anti-clockwise) and green spirals of the opposite sense. Seymour et al. (2010) found that, even though all blocks contained the same two forms and the same two colors, the pattern of activity in each of their regions of interest (V1, V2, V3, V3A/B, hV4) allowed the different conjunctions to be discriminated significantly above chance. 

As techniques applied to fMRI, it has been suggested that both adaptation and multivariate pattern classification are subject to interpretational difficulties (Bartels, Logothetis, & Moutoussis, 2008); for instance, classifiers can be very sensitive to confounding stimulus features and adaptation is thought to be sensitive to upstream processing. Despite these caveats, the convergence of the different techniques is reassuring and the consensus from fMRI studies using these techniques appears to be that color and orientation are coded in conjunction throughout the visual cortex (Engel, 2005; Seymour et al., 2010; Sumner et al., 2008). 

Given that extrastriate cortex is generally held to be essentially binocular, with the vast majority of neurons receiving input originating from both eyes (Gonzalez & Perez, 1998), the evidence from neuroimaging stands in contrast to psychophysical data indicating that orientation processing only shows strong color selectivity at monocular levels (Forte & Clifford, 2005). 

The apparent difference in chromatic selectivity evident between behavioral and neuroimaging studies of orientation processing in human visual cortex motivated us here to carry out functional imaging using a stimulus configuration similar to that employed in psychophysical investigations of the tilt illusion. Stimuli used to measure the tilt illusion typically consist of a test grating presented in a circular region centered at fixation surrounded by an annulus containing an inducing grating (Schwartz, Hsu, & Dayan, 2007). Although this configuration has been used in a previous fMRI study of contextual modulation, areas V1–V3 could not be distinguished consistently due to the foveal presentation of the test (Williams, Singh, & Smith, 2003). Since delineation of the borders between retinotopic visual areas is easier in the periphery than in the fovea (Schira, Tyler, Breakspear, & Spehar, 2009), we chose here to use an annular test region surrounded inside and out by the inducing stimulus (see also McDonald, Seymour, Schira, Spehar, & Clifford, 2009; Zenger-Landolt & Heeger, 2003). We then measured the fMRI BOLD response to test gratings designed to isolate the luminance (L + M), red–green (L–M), and blue–yellow (S–(L + M)) pathways in the presence of parallel and orthogonal inducers. 

Methods

Experimental procedures

Three female and five male participants took part in all experiments. In Experiment 2, one male was substituted, as the original participant was unavailable at the time of scanning. All were experienced psychophysical participants and had normal or corrected-to-normal vision. Each participant had at least twenty four runs. Eight runs were required for Experiment 1 (6 experimental runs, 2 localizer runs), six runs for Experiments 2 and 3, and at least four runs were used to establish the retinotopic areas. 

Scanner and experimental setup

A Philips 3T scanner with a whole-head coil was used to conduct the MRI. Anatomical images were collected using a turbo field-echo protocol for enhanced gray–white contrast and consisted of whole-head scans in the axial and sagittal planes (voxel size = 1 mm isotropic) and a high-resolution partial-head coronal scan (voxel size = 0.75 mm isotropic) to recover maximum detail in the occipital lobes. Functional images were collected using a T2* sensitive, field-echo echo-planar imaging (boustrophedon) pulse sequence (TR = 3 s, TE = 30 ms, flip angle = 90°, FOV = 70.5 × 192 × 192 mm³, matrix = 128 × 128, voxel size = 1.5 mm isotropic). Images were acquired in 47 ascending interleaved slices in the coronal plane covering the occipital lobes. 

Stimuli were displayed on a Philips Liquid Crystal Display (LCD) monitor with a display resolution of 1024 × 768 pixels that was positioned behind the bore. Participants viewed the monitor from a distance of 158 cm via a mirror mounted on the head coil, resulting in a viewing angle of 12.6° × 9.5°. For 2 participants in Experiment 2, due to technical reasons, instead of using the LCD monitor, a Digital Light Projector DLP was used to project the stimuli onto a screen at the head of the magnet bore. Stimulus size was kept constant and the same correction procedures were applied for the DLP as for the LCD monitor. 

Stimuli were presented using PsychToolbox 3.0.8 (Brainard, 1997; Pelli, 1997). Behavioral responses were indicated via a LU400-PAIR Lumina response pad (Cedrus, San Pedro, CA, USA). Except where otherwise specified, analyses were performed using SPM5 (http://www.fil.ion.ucl.ac.uk/spm) on Matlab 7.5. 

Stimulus and task

In each of three experiments, we compared the magnitude of the BOLD signal in response to stimulus blocks where the test and inducer were parallel (but 90° out of phase spatially) with blocks where test and inducer were perpendicular (Figure 1A). Stimuli were sinusoidal gratings (1 cycle/°) presented in 15-s blocks counterbalanced in order (Figure 1B). In the first experiment, test and inducer were both achromatic or both isoluminant (L–M isolating). In the second experiment, test and inducer were both S-cone isolating gratings. In the third experiment, the test was achromatic and the inducer L–M isolating, or vice versa. 

Figure 1

 

Experimental stimuli. (A) The two different stimulus types used in the experiments, orthogonal test–inducer and parallel test–inducer. All conditions also have a fixation marker. (B) Schematic example of the stimulus orientations used in a typical block, in this case achromatic stimuli with orthogonal test–inducer.

We used a stimulus configuration with an annular test region 2–3° in radius surrounded inside and out by the inducing stimulus (McDonald et al., 2009; Zenger-Landolt & Heeger, 2003). To ensure that segmentation processes were not engaged differentially in the parallel and orthogonal conditions, as may have been the case in previous fMRI studies using abutting gratings (McDonald et al., 2009; Williams et al., 2003), black lines 0.06° in diameter were used to mark the inside and outside of the test annulus. 

Each stimulus block contained a succession of gratings presented for 0.75 s at 20 different orientations. Parallel and orthogonal blocks differed in the relative orientation of the test and inducing gratings but not the distribution of absolute orientations to avoid potential biases introduced by differences in response as a function of absolute spatial orientation (Furmanski & Engel, 2000; Sasaki et al., 2006). We used a 90° shift in spatial phase between test and surround gratings in the parallel condition to isolate orientation-specific as distinct from phase-specific interactions. Relative spatial phase is not a meaningful parameter when test and surround are not parallel, so we did not want to confound the effects of orientation and spatial phase by using in-phase test and surround in the parallel condition. Attention and eye movements were controlled by requiring observers to perform a demanding dimming task at fixation. Throughout all blocks, participants were required to indicate with a button press whenever there was a brief luminance decrement of the fixation marker presented at the center of the display. 

Color calibration

Stimuli were calibrated in situ for the LCD monitor and mirror arrangement, using measurements obtained with a PR-655 SpectraScan spectrophotometer (by Photo Research). The monitor's luminance was linearized between 0 and 255. Isoluminance along the L–M (red–green) isolating axis of DKL color space (Derrington et al., 1984) was established separately for each participant prior to scanning using minimum motion (Anstis & Cavanagh, 1983) and minimum flicker techniques under viewing conditions matched to those of the scanner. Isoluminance was then confirmed in the scanner for each participant using only the minimum flicker technique. Cone contrasts for the stimuli used in Experiments 1 and 3 are given in Table 1. 

Table 1

 

Cone contrast values for achromatic stimuli and L–M isolating stimuli, for the subjective equiluminance point of participant JSM. The background stimuli had the CIE xy coordinates 0.31, 0.34 and a luminance (Y) of 13.5 cd/m².

Stimulus colour	L-cone contrast	M-cone contrast	S-cone contrast
Light/Dark	0.98	0.98	1.00
Red/Green	0.09	0.09	0.0

In Experiment 2, stimuli consisted of gratings that were modulated along the S-cone (violet–yellow) isolating axis of DKL color space. We took care to ensure the changes in both chromaticity and luminance of the screen with increasing R, G, and B values were taken into account when generating the experimental stimuli. The CIE (xyY) coordinates measured for 16 values during calibration were interpolated to 255 values using the best-fitting spline, and these were used to calculate the luminance and chromaticity for each combination of R, G, and B intensity values. 

To create a specific stimulus, we converted the representation of color from DKL space into CIE xyY coordinates, using Stockman and Sharpe's (2000) 2° cone fundamentals. The xyY coordinates that are closest to these values are searched for in the interpolated LUT to find the corresponding R, G, and B intensity values. 

For each participant, subjective isoluminance for the S-cone isolating axis was established separately prior to scanning using minimum motion (Anstis & Cavanagh, 1983) and minimum flicker techniques, under viewing conditions matched to those of the scanner. Cone contrasts for the stimuli used in Experiment 2 are given in Table 2. 

Table 2

 

Cone contrast values for S-cone isolating stimuli, for the subjective equiluminance point of participant JSM. The background stimuli had the CIE xy coordinates 0.30, 0.34 and a luminance (Y) of 6.8 cd/m².

Stimulus colour	L-cone contrast	M-cone contrast	S-cone contrast
Blue/Yellow	0.0	0.0	0.42

Analysis procedures

Preprocessing

A mean anatomical image was formed for each participant by combining the axial and sagittal whole-head scans and the coronal partial-head scan. Before averaging, each anatomical image was inhomogeneity corrected (Manjón et al., 2007), coregistered, and resampled to a voxel resolution of 0.75 mm (isotropic) where necessary. Each participant's mean anatomical image was then segmented using the automatic routines of mrGray (Teo, Sapiro, & Wandell, 1997) and ITKGray (Yushkevich et al., 2006, http://white.stanford.edu/software) followed by careful hand editing. Functional images were corrected for differences in slice timing with reference to the middle slice. Between and within run participant movement was estimated and corrected by applying the movement parameters and reslicing using 4th degree B-spline interpolation. The images were also placed into register with the world space of the participant's mean anatomical image by applying coregistration parameters to each image's affine transformation matrix. 

Regions of interest

For each participant, localizer scans using rotating wedge and expanding ring stimuli were used to define regions of interest (ROIs) corresponding to the retinotopic visual areas V1, V2, V3, V3A/B, and hV4 (Figure 2) following the definitions of Larsson and Heeger (2006). We did not attempt to separate areas V3A and V3B, instead defining a single region of interest encompassing both that we term V3A/B. We defined hV4 as a hemifield representation of the contralateral visual field sharing a border with the ventral part of V3 and sharing the foveal representation of V1, V2, and V3. 

Figure 2

 

Sample maps of functionally defined retinotopic areas for the left hemisphere of participant CWGC. (A, B) Flattened maps of visual cortex overlaid with phase maps of the response to the wedge and ring stimuli, respectively. Beneath these maps is a schematic of the stimulus and a color map showing the area of the visual field to which each color in the phase maps corresponds. (C) Resulting delineation of the individual retinotopic regions.

In addition, for each participant an inclusive mask was defined comprising those voxels that responded significantly more (p < 0.05, uncorrected) to blocks of the test stimulus presented in isolation than to blank (fixation-only) blocks (Figure 3). The intersection of this mask with each ROI was then used to define a masked region of interest (mROI)—those voxels in each visual area responding significantly to stimulation within the test annulus. 

Figure 3

 

Flattened map of left occipital cortex of participant CWGC, overlaid with a heat map showing those voxels that respond significantly to stimulation within the test annulus.

Comparison of conditions using percentage signal change

For each participant, we extracted fMRI responses by averaging data from all the voxels within each mROI (see Figure 4). For each experiment, we then averaged the signal across all blocks of the same type. The fMRI response in each condition was calculated as the percentage signal change (PSC) from fixation: 

$\mathrm{P}\mathrm{S}\mathrm{C}={100}^{*}(t-b)/b,$

(1)

where t is the mean signal value across the block (offset by 2 TRs to allow for hemodynamic delay) and b is the baseline response to the blank, fixation-only blocks. The PSC was then averaged, for each participant in each mROI, across runs. 

Figure 4

 

(A) A sagittal slice of the right hemisphere from a functional EPI of participant KJS. (B) A raw time course for the localized region (mROI) of V1, for participant CWGC. The x-axis is volume number, the y-axis is the average voxel value, and the blue line indicates average voxel value across time. Gray regions indicate the fixation-only condition; light blue, green, rose, and yellow correspond to orthogonal achromatic test and inducer, parallel achromatic test and inducer, orthogonal L–M modulated test and inducer, and parallel L–M modulated test and inducer, respectively.

For each participant in each mROI, we defined an index of the orientation selectivity of contextual modulation (m) as 

$m={100}^{*}(\mathrm{P}\mathrm{S}{\mathrm{C}}_{\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{h}}-\mathrm{P}\mathrm{S}{\mathrm{C}}_{\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{a}})/(\mathrm{P}\mathrm{S}{\mathrm{C}}_{\mathrm{o}\mathrm{r}\mathrm{t}\mathrm{h}}+\mathrm{P}\mathrm{S}{\mathrm{C}}_{\mathrm{p}\mathrm{a}\mathrm{r}\mathrm{a}}),$

(2)

where PSC_orth is the percent signal change for the condition where the test and inducing gratings are orthogonal and PSC_para is the percent signal change for the condition where the test and inducing gratings are parallel. 

Results

When test and inducer gratings were both L–M isolating, lower BOLD activation was observed across the early retinotopic areas of visual cortex in response to gratings with parallel versus orthogonal inducers (Figure 5A). Paired t-tests confirmed that this difference was significant as early as V1 (V1: t ₇ = 3.79, p < 0.001; V2: t ₇ = 5.65, p < 0.001; V3: t ₇ = 5.23, p = 0.002; V3A/B: t ₇ = 7.79, p < 0.001; hV4: t ₇ = 4.65, p = 0.003). A similar pattern of results was evident for S-cone isolating stimuli (V1: t ₇ = 3.52, p = 0.010; V2: t ₇ = 4.63, p = 0.003; V3: t ₇ = 4.53, p = 0.003; V3A/B: t ₇ = 4.97, p = 0.002; hV4: t ₇ = 3.34, p = 0.013), as shown in Figure 6A. For achromatic gratings (Figure 7A), the response with parallel versus orthogonal inducers was significantly lower only in areas V3 and V3A/B (V3: t ₇ = 2.46, p = 0.044; V3A/B: t ₇ = 3.27, p = 0.014). 

Figure 5

 

Experimental data—test and inducer both L–M isolating. (A) Percentage signal change averaged across participants, as a function of visual area, for (left) orthogonal and (right) parallel stimulus configurations. Two asterisks indicate significant difference of p < 0.01 (paired t-test) between orthogonal and parallel conditions; one asterisk indicates significant difference of p < 0.05. (B) Comparison of orientation-selectivity index (see Equation 2) across visual areas. All error bars represent ± one standard error of the mean between participants.

Figure 6

 

Experimental data—test and inducer both S-cone isolating. The figure follows the same convention as Figure 5.

Figure 7

 

Experimental data—test and inducer both achromatic. The figure follows the same convention as Figure 5.

In response to L–M isolating, S-cone isolating, and achromatic stimuli, measured orientation selectivity increased up the visual hierarchy (Figures 5B, 6B, and 7B). Trend analysis of the orientation-selectivity index across areas V1, V2, and V3 revealed significant linear trends (L–M: F _(1,7) = 30.76, p < 0.001; S: F _(1,7) = 110.06, p < 0.001; achromatic: F _(1,7) = 10.39, p = 0.015) with no significant quadratic trend. 

The data presented in Figures 5 and 6 are evidence of orientation processing sensitive to purely chromatic modulation. However, this sensitivity does not of itself constitute sufficient evidence to conclude the existence of processing by color-selective mechanisms. Instead it could, for example, be achieved by cue-invariant mechanisms sensitive to orientation regardless of the nature of the modulation (i.e., chromatic or luminance) defining it. 

To investigate the presence of orientation processing mechanisms not only sensitive to but specifically selective for color, we carried out a third experiment in which the test was L–M isolating and the inducer achromatic, or vice versa (Figure 8). The responses to L–M test and achromatic inducer showed no significant difference between parallel and orthogonal conditions in any region of interest. For the achromatic test and L–M inducer, only V3A/B showed a significantly larger response to orthogonal versus parallel test and inducer (V3A/B: t ₇ = 3.02, p = 0.019). 

Figure 8

 

Experimental data—test and inducer modulated along different color axis. (A, B) Test stimulus is modulated along L–M isolating axis, and inducer is achromatic. (C, D) The color axes of test and inducer have been swapped; test is achromatic, and inducer is modulated along the L–M axis. Both pairs of panels follow the convention for Figure 5.

The same eight participants participated in Experiments 1 and 3. The orientation-selectivity indices from these experiments are shown together in Figure 9. Using the percentage signal change data from these experiments allowed us to look for color/luminance selectivity in orientation processing in the form of an interaction in a three-way ANOVA (2 × 2 × 2: orthogonal/parallel × color/luminance test × color/luminance inducer). Significant three-way interactions were evident in all regions of interest except V1, which narrowly escaped significance (V1: F _(1,7) = 4.60, p = 0.069; V2: F _(1,7) = 7.22, p = 0.031; V3: F _(1,7) = 5.90, p = 0.046; V3A/B: F _(1,7) = 6.49, p = 0.038; hV4: F _(1,7) = 7.81, p = 0.027). This pattern of results indicates that orientation processing throughout extrastriate visual cortex is selective for the color/luminance congruence of the test and inducer. 

Figure 9

 

Replotted experimental data for Experiments 1 and 3 (from Figures 5B, 7B, 8B, and 8D). Each panel denotes the data from a different visual area. The abscissa indicates whether the test was modulated along the L–M isolating or achromatic axis of DKL space. The ordinate indicates the orientation-selectivity index (see Equation 2). The color of the data points and their connecting lines indicates whether the inducing stimuli were modulated along the L–M isolating or achromatic axis. Note that the orientation-selectivity index tends to be greatest for test and inducer modulated along the same DKL axis; this implies that not all of the effect of changing surround orientation can be attributed to color-invariant orientation-selective mechanisms.

Discussion

The results of Experiments 1 and 2 demonstrate in humans the existence across retinotopic visual cortex of mechanisms sensitive to the orientation of gratings isolating the red–green (L–M) and blue–yellow (S–(L + M)) opponent chromatic pathways. This sensitivity may be due to mechanisms jointly selective to color and orientation, to mechanisms sensitive to orientation regardless of the cue defining it, or to both. The results of Experiment 3 confirm the existence of mechanisms jointly sensitive to orientation and color, although it should be noted that they do not rule out the existence of additional cue-invariant mechanisms of orientation processing. 

The experiments described here used oriented stimuli presented in an annular test region surrounded inside and out by a stimulus of the same or orthogonal orientation. Such simultaneous presentation of stimuli has been shown to suppress the fMRI BOLD response across human visual cortex relative to sequential presentation (Beck & Kastner, 2005, 2007; Kastner et al., 2001; Williams et al., 2003; Zenger-Landolt & Heeger, 2003). What are the mechanisms underlying the orientation-specific surround effects we observe in the BOLD signal? 

One possibility is that in the current study we are observing a correlate of orientation-specific lateral inhibition. In this case, the lower response observed in the presence of parallel oriented inducing stimuli, compared to the orthogonally oriented stimuli, would be consistent with greater inhibition from the former (McDonald et al., 2009; Williams et al., 2003). Using a similar stimulus configuration to that employed here, Zenger-Landolt and Heeger (2003) found that the reduction of perceived grating contrast in the presence of a parallel inducer was well predicted by the level of BOLD response in V1. They argued that the observed reduction of BOLD response in V1 was likely due to suppressive lateral interactions from neurons representing the inducing stimulus. 

Our experiments using purely chromatic stimuli also showed a significant orientation-specific effect in V1, consistent with the operation of orientation-specific lateral interactions between neurons representing the inducing and test stimuli. However, it is important to note that we do not claim here to have isolated only the response to the test region of the stimulus. Voxel receptive field size in V1 at the eccentricity of the test annulus (2–3°) is of the order of 0.5° (Dumoulin & Wandell, 2008; Kay, Naselaris, Prenger, & Gallant, 2008; Larsson & Heeger, 2006), considerably wider than the line used to separate test and inducer. Moreover, voxel receptive field size increases with the size of neuronal receptive fields as we move beyond V1 to higher visual areas (Kastner et al., 2001). Thus, some of the voxels in our masked regions of interest (mROIs), defined as those voxels in each visual area responding significantly to stimulation within the test annulus, inevitably show some direct response to the inducing stimulus. Hence, it is probable that summation effects within the neuronal receptive field contribute to the pattern of results we observe. 

Consider for example the situation illustrated schematically in Figure 11. The shaded circle represents the receptive field location shared by two hypothetical orientation-selective neurons, one tuned to vertical (Figures 11A and 11B) and the other to horizontal (Figures 11C and 11D). The receptive field of these neurons is centered within the test annulus but extends into the inducing region. Figure 10 shows how changing the width of the test annulus would change the relative contribution of test and inducer stimuli. In Figure 11, the spatial summation properties are illustrated schematically in the inset graphs, in the form of an inclusion field and an exclusion field (Angelucci & Bressloff, 2006). 

Figure 10

 

A hypothetical depiction of the effect of changing the width of the test annulus on the response of a voxel; the diagrams in the left-hand column indicate stimuli in the test annulus (top) and stimuli in the inducing region (bottom). The gray circle in both diagrams indicates the region of visual space that drives the response of the voxel. The graphs in the right-hand column indicate the effect of increasing the width of the test annulus. In the top graph, increasing the width of the annulus (in the absence of the inducer) increases the response of the voxel, possibly to saturation. In the bottom graph, increasing the width of the annulus (in the absence of stimulation in the test region) decreases the response of the voxel as there is less stimulation from the inducing region.

Figure 11

 

Possible explanation for modulation of orientation-selectivity index that does not require orientation-specific lateral inhibition. Format is similar to that of Figure 10. The left and right columns represent the parallel and orthogonal stimulus conditions, respectively. For the sake of illustration, the orientation of the test stimulus is vertical. The top and bottom rows represent the contribution of vertically and horizontally tuned neurons, respectively, to a voxel's BOLD response. Test annulus and inducer regions are both stimulated by vertical gratings; hence, (A) the BOLD response due to the vertically tuned neurons is high and (C) there is no BOLD response due to horizontally tuned neurons. (B, D) BOLD response due to the vertically tuned neurons.

Experimentally, neuronal inclusion fields are determined by measuring response to the preferred stimulus presented in a circular aperture centered on the receptive field as a function of the radius of the aperture. They typically show a monotonic rise in response as a function of radius that saturates beyond a certain aperture size. Exclusion fields are determined by presenting the preferred stimulus in a large annulus surrounding a blank region centered on the receptive field. Neuronal response is then measured as a function of the internal radius of the annulus. At small values of the internal radius, the response of the neuron is typically saturated but then falls monotonically as the radius of the blank region increases. If summation within the receptive field is not linear, for example due to the presence of a non-linearity in the spike generating mechanism (Ringach & Malone, 2007), then the exclusion field will not simply be the complement of the inclusion field. 

The possible non-linearity of spatial summation within the receptive fields of individual neurons has important consequences in the context of the current experiment. Consider a voxel containing a population of orientation-selective neurons as described above, some preferring vertical and others horizontal. For ease of exposition, let us fix the orientation of the test stimulus as vertical and consider the summed neuronal response within the voxel in the presence of a parallel (vertical) or orthogonal (horizontal) inducing stimulus. 

The parallel condition (Figures 11A and 11C) is the more straightforward. The vertically tuned neurons within the voxel are responding to vertically oriented stimuli extending well beyond the inclusion field radius at which they saturate, so give a maximal response (Figure 11A). There is no horizontal stimulus, so the horizontally tuned neurons give no response (Figure 11C). Thus, in the parallel condition, the summed neuronal response within the voxel thus corresponds to the maximal response of the vertically tuned neurons. 

The orthogonal condition (Figures 11B and 11D) is slightly more complicated as it elicits responses both from vertically and horizontally tuned neurons within the voxel, even though the voxel itself is centered on the test annulus that contains only a vertical stimulus. The vertically tuned neurons respond to the vertical test stimulus, but their response does not saturate if their inclusion fields extend beyond the test (Figure 11B). The horizontally tuned neurons also respond if their exclusion fields extend beyond the test as they are being stimulated by the horizontal inducer (Figure 11D). Thus, in the orthogonal condition, the summed neuronal response within the voxel consists of contributions from both the vertically and the horizontally tuned neurons. 

In the case where contribution of the horizontally tuned neurons outweighs the reduction in response of the vertically tuned neurons relative to the parallel condition, the orthogonal stimulus condition will elicit the greater response in the voxel. This is the direction of effect consistently observed in the current experiment. Moreover, the magnitude of the difference in voxel response between orthogonal and parallel conditions can be expected to vary as a function of receptive field size. As receptive field size increases as we move beyond V1 to higher visual areas (Kastner et al., 2001), so the response to the inducing stimulus of voxels centered on the test stimulus will tend to increase. Thus, we might expect to see a greater difference between orthogonal and parallel conditions as we ascend the cortical processing hierarchy. This is indeed the pattern of data that was observed, with the orientation-selectivity index across areas V1, V2, V3, and V3A/B consistently revealing significant linear trends in Experiments 1 and 2 (Figures 5B, 6B, and 7B). 

How can we reconcile evidence of significant color/luminance selectivity in orientation processing across human visual cortex from this and previous neuroimaging studies (Engel, 2005; Seymour et al., 2010; Sumner et al., 2008) with behavioral evidence for essentially cue-invariant processing at binocular levels of the visual system (Forte & Clifford, 2005)? One possibility is that, as discussed above, the chromatic selectivity of orientation processing observed in the present study may in large part be a reflection of selectivity in the responses of individual neurons. In contrast, chromatic selectivity as measured behaviorally through the tilt illusion likely reflects selectivity in lateral interactions between neurons. 

It is instructive to decompose the selectivity of lateral interactions into two components: the selectivity of the individual neurons and the specificity of their interactions. For example, consider a hypothetical example of two mutually inhibitory neurons. The only excitatory input that one neuron receives is chromatic and the only excitatory input that the other receives is from luminance signals. Thus, the neurons would be said to be selective for color and luminance, respectively, on the basis of their excitatory inputs. However, their interaction would show color–luminance selectivity in that it would only be evident when both neurons were receiving excitatory input. For lateral interactions to be as selective as individual neurons, the interactions must occur specifically between neurons sharing the same selectivity. Assuming that the lateral interactions between two neurons cannot be more selective than the neuronal responses themselves, any lack of specificity in the interactions between neurons can only serve to diminish the selectivity of those interactions below the level of the individual response selectivity. The existence of neurons at binocular levels of human visual cortex whose response is jointly selective for color and orientation but whose gain is controlled by interaction with a pool of neurons of varying or heterogeneous selectivity could then reconcile the apparent contradiction between the results of this study and previous psychophysical findings using a similar stimulus configuration (Forte & Clifford, 2005). 

Acknowledgments

Scanning was carried out in Sydney at the Prince of Wales Medical Research Institute and St Vincent's Hospital. We are grateful to Kirsten Moffat and her team for expert technical assistance. This research was funded by an Australian Research Fellowship to CC and a Project Grant from the National Health and Medical Research Council. 

Commercial relationships: none. 

Corresponding author: Colin W. G. Clifford. 

Email: [e-mail]. 

Address: Griffith Taylor Building (A19), The University of Sydney, Sydney, NSW 2006, Australia.