Neural representation of form-contingent color filling-in in the early visual cortex

Abstract

Perceptual filling-in exemplifies the constructive nature of visual processing. Color, a prominent surface property of visual objects, can appear to spread to neighboring areas that lack any color. We investigated cortical responses to a color filling-in illusion that effectively dissociates perceived color from the retinal input (van Lier, Vergeer, & Anstis, 2009). Observers adapted to a star-shaped stimulus with alternating red- and cyan-colored points to elicit a complementary afterimage. By presenting an achromatic outline that enclosed one of the two afterimage colors, perceptual filling-in of that color was induced in the unadapted central region. Visual cortical activity was monitored with fMRI, and analyzed using multivariate pattern analysis. Activity patterns in early visual areas (V1–V4) reliably distinguished between the two color-induced filled-in conditions, but only higher extrastriate visual areas showed the predicted correspondence with color perception. Activity patterns allowed for reliable generalization between filled-in colors and physical presentations of perceptually matched colors in areas V3 and V4, but not in earlier visual areas. These findings suggest that the perception of filled-in surface color likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors.

Introduction

Color perception depends on both peripheral and central processing of color information. Prolonged viewing of a colored stimulus is known to lead to a negative color afterimage, due to low-level adaptation at the level of the retina (Brindley, 1962; Craik, 1940; Zaidi, Ennis, Cao, & Lee, 2012). However, psychophysical studies suggest that purely cortical mechanisms can also determine the perception of color aftereffects (Shevell, St. Clair, & Hong, 2008; Shimojo, Kamitani, & Nishida, 2001), as suggested by the different percepts that can result from a bicolored afterimage (van Lier, Vergeer, & Anstis, 2009). Following adaptation to a red and cyan bicolored star, such as that shown in Figure 1, either one of two filled-in color percepts can be elicited in the unadapted central achromatic region, depending on the achromatic test stimulus that follows. In essence, the achromatic boundary allows for the perceptual spreading of afterimage color within the bounded region, but prevents the afterimage color outside of the boundary from filling in the central region. 

Figure 1

 

Experimental design, stimuli, and reported color values during perceptual filling-in. (A) The color filling-in experiment consisted of two possible configurations for the adapting stimulus, which led to a negative color afterimage in local regions receiving direct color stimulation. After adaptation, either a reddish or cyanish filled-in percept could be induced in the central achromatic region by presenting the appropriate test stimulus. The dashed circle region depicts the visual-field location of interest, which was functionally localized in separate fMRI runs. (B) The control experiment measured cortical responses in the central region to physically presented colors. Test stimuli were uniformly colored with matched chromaticity values obtained for the two possible filling-in percepts. (C) Group-averaged values of perceived chromaticity during filling-in, shown in cone-contrast color space (error bars were smaller than the symbols). The two outer points indicate the colors in the adapting stimulus; the arrows indicate the direction of color shift in the central region induced by color filling-in.

This form-contingent color aftereffect cannot be explained in terms of retinal adaptation alone, and instead, is consistent with cortically based models of filling-in (Francis, 2010; Grossberg & Mingolla, 1985a, 1985b; Pinna & Grossberg, 2005). These models propose that local surface features, such as color, have a tendency to spread across the retinotopic visual cortex until a luminance-based perceptual boundary is encountered. Although these models provide a viable account, the neural mechanisms that underlie filling-in are not fully understood (Dakin & Bex, 2003; De Weerd, Gattass, Desimone, & Ungerleider, 1995; Kanai, Wu, Verstraten, & Shimojo, 2006; Komatsu, 2006; Meng, Remus, & Tong, 2005; Nakayama, Shimojo, & Ramachandran, 1990; Pessoa, Thompson, & Noe, 1998; Ramachandran & Gregory, 1991; Sasaki & Watanabe, 2004; Spillmann & De Weerd, 2003; von der Heydt, Friedman, & Zhou, 2003). Moreover, it has proven challenging to isolate cortical responses to phenomenally perceived color, in a manner that is independent of the properties of a colored stimulus. The goal of the present study was to investigate the representation of perceptually filled-in color and the cortical basis of these color-form contingent interactions. 

We used functional MRI (fMRI) and multivariate pattern analysis to measure feature-selective responses in the human visual cortex (Kamitani & Tong, 2005, Tong & Pratte, 2012), focusing on cortical responses to color (Brouwer & Heeger, 2009, 2013; Parkes, Marsman, Oxley, Goulerman, & Wueger, 2009; Seymour, Clifford, & Logothetis, 2010; Sumner, Anderson, Sylvester, Haynes, & Rees, 2008). In the main experiment, observers were presented with the adapting stimulus followed by one of the two test stimuli, which induced either a filled-in percept of faint red or cyan (Figure 1). We applied fMRI decoding to retinotopic regions corresponding to the central achromatic portion of the adapting display, and tested whether activity patterns for the two filled-in color percepts could be distinguished. In the control experiment, observers viewed physically colored versions of the test stimulus, which were presented in faint cyan or red, to match the perceived colors in the filling-in experiment. This allowed us to test whether a pattern classifier trained on cortical responses to physical colors allowed for reliable prediction of the perceptually filled-in color. 

Methods

Participants

Six healthy volunteers with normal or corrected-to-normal vision (ages 26–38, three females and three males) participated in the experiment. All participants gave informed consent to participate in the fMRI experiment, which was approved by the Vanderbilt University Institute Review Board. 

Apparatus and stimuli

Visual stimuli were generated by a Macbook Pro laptop computer running Matlab (The MathWorks, Inc., Natick, MA), and projected onto a rear-projection screen using Eiki LC-X60 LCD projector (Eiki International, Inc., Rancho Santa Margarita, CA) with Navitar zoom lens (Navitar, Inc., Rochester, NY). The LCD projector was carefully calibrated using a Minolta LS110 luminance meter (Konica Minolta Sensing Americas, Inc., Ramsey, NJ) to ensure linearity of luminance output in the visual display. Color calibration of the projector was also conducted by measuring the spectral power distribution of the R, G, and B primaries at maximum output with an Ocean Optics USB4000 spectroradiometer (Ocean Optics, Dunedin, FL). 

The adapting stimulus consisted an eight-pointed star shape (diameter, 16° of visual angle), as shown in Figure 1. Four spikes were colored with reddish appearing light (l = 0.692, s = 0.90) and the other four spikes were colored with cyanish appearing light (l = 0.624, s = 1.23). The central region of the adapting stimulus (9.6° diameter) was achromatic gray (l = 0.655, s = 1.10), which was identical to the overall background. These inducing colors were adopted from the original study expressed in terms of CIE coordinates (van Lier et al., 2009), then transformed to chromaticity values based on cone-contrast color space (Figure 1C). The luminance of all regions of the adapting stimulus was fixed at 24 cd/m². Test outlines consisted of four-pointed stars that followed the boundaries of either the reddish spikes or cyanish spikes of the adapting stimulus. In the filling-in experiment, the test outlines were always presented with an achromatic gray interior. 

In the control fMRI experiment, the adapting stimulus was not presented. Instead, the four-pointed test stimulus was presented with either a faint reddish or cyanish tint. The interior of the test stimulus was uniformly colored with chromaticity values to match the observer's reported filled-in colors. These chromaticity values were obtained in a separate color-matching experiment performed outside the scanner. In both fMRI experiments, participants were instructed to maintain fixation on a central fixation dot (0.2° diameter) that remained present throughout each fMRI run. The fixation dot was presented in black and flashed at random intervals every few seconds, and participants reported the flash of the fixation dot by pressing a designated button. 

For the color-matching task, observers adjusted the hue, saturation, and light intensity of a matching patch by using six different keys (to increase or decrease each of the three parameters), until the patch appeared the same as the filled-in color that resulted from the form-contingent color aftereffect. During each trial of the matching experiment, the adapting stimulus and one of the test outlines were repeatedly presented in alternation, with the adapting stimulus presented on the left side of the screen and the matching patch presented on the right side of the screen. Observers were instructed to fixate their eyes on the fixation dot until they observed a reliable effect of color filling-in. Once the observers experienced strong color filling-in, they could saccade to the fixation dot in the middle of the matching color patch and adjust its color appearance. Observers were allowed to repeat this procedure, switching back and forth between the color filling-in stimulus and the matching stimulus, until they were satisfied with their match. Observers performed three trials of color matching for each of the two color filling-in conditions, and the averaged chromaticity values of their reports were used for the control experiment. Inside the scanner, observers performed a minimum motion procedure to set individual equiluminance values, so that the matched chromaticity values could be correctly presented with the display system at the scanner. After setting the participant's equiluminance values, observers performed the color-matching task inside the scanner to confirm the chromaticity values obtained outside the scanner. For some observers, chromaticity values for the control stimulus were modestly increased (by 2–3 times the initial reported cone-contrast value) during the fMRI control experiment to ensure that fMRI decoding of the physical stimulus color would be reliable. This control condition allowed us to test for generalization between fMRI responses to perceptually filled-in colors and real colors presented at low saturation. As long as the real colors evoked reliable fMRI responses, the precise level of saturation would not be expected to affect the pattern of generalization performance. 

The eight-pointed star spanned a diameter of 16° of visual angle. The diameter of the central achromatic portion of the star was about 9.6°, and our retinotopic region of interest was confined within 8.5° of the central circular region based on the flickering checkerboard stimulus used in the fMRI localizer runs. 

Experimental procedure and fMRI acquisition

BOLD responses were acquired using a 3.0-Tesla Intera Achieva Philips MRI scanner (Philips Healthcare, Best, The Netherlands) with an eight-channel head coil at Vanderbilt University Institute for Imaging Science. At the beginning of the MRI session, a high-resolution three-dimensional (3-D) anatomical T1-weighted scan was obtained (field of view [FOV] 256 × 256, 1 × 1 × 1 mm resolution). Functional BOLD responses were measured with a standard gradient-echo echoplanar T2*-weighted imaging sequence (repetition of time [TR] 2000 ms, echo time [TE] 35 ms, flip angle 80°; FOV 192 × 192, slice thickness 3 mm (no gap), in-plane resolution 3 × 3 mm). The imaged volume consisted of 26 slices acquired perpendicular to the calcarine sulcus, covering the entire occipital lobe as well as the posterior parietal and temporal cortices. 

Each scanning session was composed of two visual localizer runs and 20 experimental runs. The entire scanning session lasted about 2.5 to 3 hr. For the localizer runs, an on-off block design was used to localize responses to a flickering checkerboard stimulus (check size 0.4°, flicker rate 10 Hz) that covered the central region (8.5° visual angle) of the adapting and testing stimuli. Participants performed a central fixation task and had to report whenever the fixation stimulus briefly reversed in contrast polarity, which occurred at randomly chosen intervals. Each localizer run lasted 300 s, and consisted of alternating 12-s periods of stimulation and fixation rest. 

Each experimental session consisted of 10 runs of the filling-in experiment and 10 runs of the control experiment, presented in alternating blocks of runs, counterbalanced across observers. The adapting stimulus occurred in two possible configurations, such that fMRI decoding of filled-in color could not be attributable to the particular adapting stimulus or test stimulus, but rather, the correspondence between adaptor and test. Only one of two possible configurations of the adapting stimulus was used (Figure 1A) in each run, to maximize adaptation to that particular color pattern, and five runs were assigned for each configuration. Each run for the filled-in condition began with an 18-s period of viewing the adapting stimulus, 12 successive experimental stimulus blocks (18 s each), followed by a final 18-s rest period during which participants viewed the same adapting stimulus to balance the level of activity at beginning and end of the run. On each stimulus block, the adapting stimulus and one of the two achromatic test outlines were presented alternately for 1 s each for nine presentations. 

In the control experiment, no adapting stimulus was presented; instead, each run began and ended with an 18-s fixation rest period. During each of the 12 stimulus blocks, either the red- or cyan-tinted test stimulus was repeatedly presented (1 s on, 1 s off) to mimic the subjectively perceived colors that were induced in the filling-in experiment (Figure 1B). There were the two types of control runs, as the color assignment of the low saturation red- and cyan hues to the two test stimuli was manipulated across run types. 

fMRI data processing

All fMRI data were motion-corrected using automated image registration software. The motion-corrected data were preprocessed using Brain Voyager QX (version 1.9, Brain Innovation, Maastricht, The Netherlands), which included slice timing correction and linear trend removal. No spatial or temporal smoothing was applied. Rigid-body transformations were performed to align the fMRI data to the within-session T1-weighted 3-D anatomical scan, which in turn was aligned to an anatomical 3-D scan collected in the separate retinotopic mapping session. All automated alignment procedures were carefully inspected visually and subjected to manual fine-tuning to correct for any visible residual misalignment. After across-session alignment, all data underwent Talairach transformation and reinterpolation using 3 × 3 × 3 mm voxels. 

Region of interest

The retinotopic visual areas of each participant were functionally mapped in a separate scanning session using well-established methods (Engel, Glover, & Wandell, 1997; Sereno et al., 1995; Wandell, Dumoulin, & Brewer, 2007). After retinotopic areas V1, V2, V3, V3A, and hV4 in both hemispheres were identified, visually activated voxels corresponding to the central region of the adapting stimulus were selected based on statistical activation maps obtained from the visual localizer runs. Area hV4 was identified based on its hemifield representation and its anatomical location just anterior to ventral V3. We selected the most reliably activated 100 voxels from each region of interest (ROI), although the number of voxels obtained in V3A and V4 did not reach that number for some observers. For all of our fMRI decoding analyses, we confirmed that decoding performance was reliable and robust with varying numbers of voxels selected from each area. In general, decoding for filled-in afterimage colors tended to improve up to 80 voxels per ROI and showed a similar level of performance with up to 150 voxels selected from each visual area. 

Linear classifier and decoding

We used linear support vector machines to obtain a linear function to discriminate the activity patterns elicited by the two perceived filled-in colors (appearing pinkish and appearing cyanish). Specifically, we used a linear support vector machine classifier to find optimal weights and bias values to define a discriminating hyperplane to distinguish between fMRI response patterns to the two perceptual color conditions in each training data set, and evaluated classification accuracy using independent test data from separate runs (Kamitani & Tong, 2005). We used a leave-two-runs-out procedure for cross-validation to ensure proper counterbalancing, as only one of the two possible adapting stimuli appeared in each experimental run. We repeated the cross-validation procedure until each pair of runs had served as the test data set, and then calculated the average classification performance for all iterations. 

Results

Filling-in of afterimage color was vivid and robust, so that the appearance of the filled-in surface color could be readily measured by a conventional color-matching task. Participants dynamically adjusted the hue and saturation of a distinct patch until it matched the perceived color induced in the central region of the filled-in after image (see Methods). The matched chromaticity values were l = 0.648, s = 1.13 for the cyanish appearing afterimage and l = 0.661, s = 1.05 for the pinkish appearing afterimage. Plotting these reported colors in cone-contrast space (MacLeod & Boynton, 1979), along with the actual color values of the adapting stimulus, it can be seen that all data points fall along a single line (Figure 1C). These results imply the operation of a color-opponent mechanism, as the filled-in color was perceived to be opposite in hue from the relevant inducer. 

We analyzed BOLD activity patterns from early visual areas V1, V2, V3, V3A, and hV4, for the central circular portion of the visual field corresponding to the physically achromatic region of the adapting stimulus (Figure 1A and B, dashed line). Multivariate pattern analysis of BOLD responses, obtained from six participants, showed that activity patterns in V1–V4 reliably predicted the filled-in color percept. As shown in Figure 2A (gray bars), classification performance was significantly greater than chance level (50%) in all visual areas tested (V1: t[5] = 3.04, p < 0.05; V2: t[5] = 6.51, p < 0.01; V3: t[5] = 7.52, p < 0.01; V3A: t[5] = 3.13, p < 0.05; hV4: t[5] = 3.19, p < 0.05), which indicated differential color filling-in responses within the retinotopic region where no physical color had appeared. We performed a control analysis on univariate mean BOLD responses from these regions of interest and found that mean BOLD could not reliably distinguish between the two different filled-in colors (Figure 2A, white bars). Thus, we find relevant information in the patterns of visual cortical activity rather than in the overall mean BOLD amplitude. It should be noted that the classification performance observed here could not be based on the differences in the neural responses induced by the components of test stimulus, since the same adapting colors and achromatic outlines occurred in the two conditions to be decoded. Instead, it was the specific pairing between the two adapting colors and the two outlines that led to the difference in perceptually filled-in color. 

Figure 2

 

Classification accuracy with multivoxel pattern analysis (gray bars) and with decoding of univariate mean BOLD amplitudes (white bars) for filling-in condition (A), control condition (B), and generalized classification performance between the two conditions (C). Error bars represent ±1 SEM with six observers. Dashed line represents chance-level performance in classification. (*p < 0.05, **p < 0.01.)

Reliable pattern classification implies that the neural responses evoked by the two different filling-in conditions are distinct enough to be picked up by multivariate pattern analysis of fMRI BOLD signals. What then is the nature of these differential responses, and do they necessarily correspond to the perceptual experience of color in this central region? (For example, if filling-in were more vivid in one of the two conditions, then differential responses might reflect differences in vividness rather than sensitivity to color per se.) 

To address this issue, we conducted a control experiment (in the same fMRI session) in which perceptually matched versions of the filled-in colors were directly presented to the observer (Figure 1B). The chromaticity values of these physically presented colors were obtained for each observer in a separate color-matching experiment. We hypothesized that if the perceptual representation of filled-in color is present in a visual area, then the activity patterns in that area should be similar for filled-in colors and for physically matched colors. 

Results from the control experiment indicated that physical color differences between red and cyan could be reliably distinguished by fMRI pattern analysis in all early visual areas. In general, we found comparable levels of fMRI decoding performance for participants who were shown the test stimuli with chromaticity values that matched the perceptual filling-in condition and those who were shown the stimuli with somewhat greater cone-contrast, 2–3 times that of the reported match. We therefore combined the measurements from all observers for the analysis. Classification performance was significantly greater than chance level in individual visual areas V1 through V4 (Figure 2B, gray bars; V1: t[5] = 3.94, p < 0.01; V2: t[5] = 5.37, p < 0.01; V3: t[5] = 3.33, p < 0.05; V3A: t[5] = 4.48, p < 0.01; hV4: t[5] = 5.48, p < 0.01). In contrast, classification based on the mean BOLD response in each ROI did not significantly differ from chance-level performance (Figure 2B, white bars; V1: t[5] = 1.27, p = 0.12; V2: t[5] = 0.45, p = 0.12; V3: t[5] = 0.47, p = 0.33; V3A: t[5] = 0.74, p = 0.25; hV4: t[5] = 0.28, p = 0.39), again indicating that color-specific information resides in the cortical activity patterns and not in the mean BOLD response. 

To evaluate generalization performance across filled-in and physically viewed colors, we trained linear classifiers with fMRI data obtained from the afterimage experiment and tested the classifiers with fMRI data obtained in the control experiment, and vice versa. The results of this generalization analysis revealed a reliable correspondence between cortical responses to filled-in colors and responses to actual colors in extrastriate visual areas V3 and V4 (Figure 2C, gray bars; V3: t[5] = 3.57, p < 0.05; V4: t[5] = 5.19, p < 0.01). However, visual areas V1 and V2 failed to show reliable generalization in their responses to physical and perceptually filled-in colors (V1: t[5] = 1.79, p = 0.07; V2: t[5] = 1.29, p = 0.13). These results imply that the cortical representation of filled-in surface color is evident at higher levels of the pathway in extrastriate visual areas such as V4. Such representations of perceived color may be distinct from more basic responses to color at earlier stages of processing. For example, previous fMRI studies have found evidence of sensitivity to the perceptual similarity of colors as well as some degree of categorical clustering in area V4, but no such effects in V1 (Brouwer & Heeger, 2009, 2013). Our findings bolster the view that higher visual areas, such as V4, are important for representing the subjective experience of color, as reliable information about perceived color is found in this region, even when an achromatic stimulus is responsible for inducing these experiences. 

Discussion

We observed distinct patterns of visual cortical activity for the color percepts that arose from this form-contingent color filling-in illusion (van Lier et al., 2009). The color-form interactions required for the dynamic filling-in of afterimage color presumably depend on long-range spatial interactions at a cortical level. We found that the activity patterns for filled-in colors corresponded well with those evoked by physical, appearance-matched colors in visual areas V3 and V4, but failed to observe an equivalent correspondence in visual areas V1 and V2. These results suggest that the perception of form-contingent color filling-in likely requires more extensive processing by extrastriate visual areas, in order for the neural representation of surface color to become aligned with perceptually matched real colors. Our findings provide compelling new evidence to support the view that color representations in higher visual areas, such as V4, correspond better with subjective color perception than those observed at earlier processing stages, such as V1. 

It is worth considering whether the lack of generalization performance in V1 may be attributed to the smaller receptive field size of neurons in this region. Consistent with neurophysiological studies, fMRI measures of voxel-based population receptive fields (pRF) have demonstrated that pRF sizes increase as a function of eccentricity and become larger in size at progressively higher levels of the human visual pathway (Dumoulin & Wandell, 2008, see also Yoshor, Bosking, Ghose, & Maunsell, 2007). While we cannot rule out the potential contributions of receptive field size to V1 performance, we did find that fMRI decoding applied to the central portion of V1, quite far from the color inducers, revealed the presence of reliable response patterns that distinguished between the two color filling-in conditions. If receptive field size, pRF size, or retinotopic location led to a critical limiting factor, then decoding of color filling-in should have proven unsuccessful in V1. These long-range effects of color filling-in could arise from intracortical interactions within V1 or top-down feedback of visually specific information to this region; long-range effects of filling-in have been previously reported in the primary visual cortex (Huang & Paradiso, 2008; Komatsu, 2006; Meng et al., 2005; Sasaki & Watanabe, 2004). 

Our findings provide support for the proposal that activity in extrastriate visual areas, such as V4, may better reflect color perception than activity in V1. It has been suggested that responses in V4 are more stable across changes in illumination, consistent with the predictions of color constancy (Kusunoki, Moutoussis, & Zeki, 2006; Walsh, Carden, Butler, & Kulikowski, 1993; Wild, Butler, Carden, & Kulikowski, 1985; Zeki, 1983). Moreover, recent fMRI studies have suggested that activity in V4 corresponds better with perceptual color space than that of V1 (Brouwer & Heeger, 2009, 2013). Whereas these previous studies measured responses to physically presented colors to test for a correspondence with perception, the present study evaluated cortical responses to phenomenally induced colors at a visual location where no color stimulus was presented. Given that either of the two possible color percepts could be elicited following adaptation to a common bicolored stimulus, our procedure controls for low-level stimulus-driven responses to isolate activity associated with phenomenal color appearance. 

Our findings are also relevant to investigations of the neural mechanisms of filling-in. We found that activity patterns in the unstimulated region of all early visual areas, including V1, led to differential responses for the two filled-in colors. This suggests that some type of filling-in of information is occurring even at the earliest stages of the visual cortical hierarchy, although admittedly, we found a better correspondence with color perception at higher stages of visual processing. Investigations of surface brightness filling-in has indicated some positive results of filling-in of activity in V1 (Haynes, Lotto, & Rees, 2004; Huang & Paradiso, 2008; Pereverzeva & Murray, 2008; Rossi, Rittenhouse, & Paradiso, 1996) while other studies have reported the absence of filling-in or positive effects only in higher extrastriate areas (Cornelissen, Wade, Vladusich, Dougherty, & Wandell, 2006; Perna, Tosetti, Montanaro, & Morrone, 2005; Roe, Lu, & Hung, 2005). The role of primary visual cortex in perceptual filling-in, however, has been reliably shown with other filling-in phenomena including the perception of visual phantoms (Meng et al., 2005), subjective contour perception (Kok, Bains, van Mourik, Norris, & de Lange, 2016; Lee & Nguyen, 2001), and filling-in at the blind spot (Komatsu, Kinoshita, & Murakami, 2002). In the primary visual cortex, however, neural responses to a filled-in surface are known to occur later in time than the responses to an actual contour stimulus (Huang & Paradiso, 2008; Lee & Nguyen, 2001; Supèr, Spekreijse, & Lamme, 2001); this has been taken to suggest that filling-in responses are mediated by feedback from higher visual areas, such as V4 (Supèr & Lamme, 2007). 

The neural basis of color filling-in has also received some attention, under conditions of neon color spreading (Sasaki & Watanabe, 2004) and Troxler's fading (Hsieh & Tse, 2010). Sasaki and Watanabe (2004) induced neon color spreading by presenting pac-men Kanisza figures. This study found enhanced responses in V1 regions associated with the perception of a filled-in color surface, but did not directly test whether this V1 activity reflected any color-specific information. Hsieh and Tse (2010) used fMRI pattern analysis to test for evidence of color mixing under conditions of Troxler's fading. Participants maintained prolonged fixation while peripheral blue target patches gradually faded against a red background, which ultimately led to a blended color percept of a purplish background. The fMRI pattern analysis suggested that activity in V1 was biased towards purple following Troxler's fading of the targets, consistent with a color-mixing effect. However, Troxler's fading is associated with low-level adaptation beginning as early as the retina, which may account for the earlier site of color-specific filling-in found in their study. In the present study, the negative afterimage of the bicolored star likely involved retinal adaptation, but the final color percept depended on form-contingent interactions that would require higher level cortical processing. It is worth noting that neurophysiological studies of border ownership have found that a majority of neurons in V2 and V4 are sensitive to which side of an achromatic figure they happen to fall on, whereas a minority of V1 neurons exhibit such sensitivity (Zhou, Friedman, & von der Heydt, 2000). Similar effects of border ownership have been observed in color-selective neurons in V2 as well as V1 (Friedman, Zhou, & von der Heydt, 2003). Thus, it is plausible that the integration of signals from color- and border-tuned neurons at downstream sites could be responsible for the form-contingent color filling-in effects observed here. 

Other research on V4's role in perceptual filling-in and form-based processing may also be of relevance for understanding why we observed a reliable correspondence between filled-in and actual colors in area V4, but not in V1. V4 neurons not only have larger receptive field sizes, they also can integrate visual information over a larger spatial pool of neurons. Such integrative processing in V4 may be critical for creating sensitivity to visual form across diverse stimulus components and/or feature domains (Roe et al., 2012). Further research suggests that V4 is important for figure-ground processing and surface completion. For example, figure-ground perception can be significantly impaired by lesions applied to area V4 in awake-behaving monkeys (Supèr & Lamme, 2007). The timing of figure-selective responses in areas V4 and V1 is also consistent with the notion of feedback propagation of visual information from higher to lower visual areas (Poort et al., 2012). V4 may have an important role in representing information about larger visual surfaces, as its responses are not simply dictated by the edges of a visual stimulus. In a study of illusory Kanizsa figures, Cox et al. (2013) found that the response of V4 neurons was elevated only when their receptive field was centered on the illusory surface rather than on the inducing elements. These results suggest that a considerable proportion of V4 neurons may be more strongly tuned to surface properties rather than to the edge properties of a stimulus. These aspects of surface and form-related processing may have contributed to the greater correspondence we observed in area V4, in terms of its response patterns to illusory filled-in colors and perceptually matched real colors. 

Finally, the present study demonstrates how fMRI pattern analysis is sufficiently sensitive to investigate more subtle aspects of color perception. Although the accuracy of color decoding was fairly modest in our study (∼60% accuracy for individual visual areas), when compared to the decoding accuracy typically observed for saturated colors (e.g., Brouwer & Heeger, 2009; Parkes et al., 2009), it is notable that we were able to discriminate between hues of very low saturation (±0.5% L-cone contrast). It has been shown that the primary visual cortex is sensitive to chromatic contrast, such that, L − M signals (i.e., red–green equiluminant stimuli) can induce stronger fMRI responses than luminance (L + M) signals with equivalent cone contrast (Engel, Zhang, & Wandell, 1997). Although the chromatic contrast of the matched colors in our control condition was just above the threshold contrast needed to observe reliable mean BOLD responses in early fMRI studies (Engel et al., 1997), our multivariate pattern analysis proved effective at detecting differences in the visual content represented by BOLD activity patterns. Moreover, we were able to discriminate color-specific responses to stimuli that differed only in the phenomenal color percept that they evoked. These findings suggest that fMRI approaches involving multivariate pattern analysis (Tong & Pratte, 2012) or forward modeling (Brouwer & Heeger, 2009, 2013) may serve as promising tools for future investigations of the neural bases of human color perception. 

Acknowledgments

This research was supported by National Eye Institute grant R01 EY017082 to FT. Additional technical support was provided by center grant P30 EY008126 awarded to the Vanderbilt Vision Research Center. 

Commercial relationships: none. 

Corresponding author: Sang Wook Hong. 

Email: . 

Address: Department of Psychology, Florida Atlantic University, Boca Raton, FL, USA.