Luminance contours can gate afterimage colors and “real” colors

Abstract

Abstract

Abstract:

Abstract  It has long been known that colored images may elicit afterimages in complementary colors. We have already shown (Van Lier, Vergeer, & Anstis, 2009) that one and the same adapting image may result in different afterimage colors, depending on the test contours presented after the colored image. The color of the afterimage depends on two adapting colors, those both inside and outside the test. Here, we further explore this phenomenon and show that the color-contour interactions shown for afterimage colors also occur for “real” colors. We argue that similar mechanisms apply for both types of stimulation.

Introduction

Color and luminance interact in many ways in the human visual system. One example is Daw's (1962) discovery that colored afterimages were much more visible if they were superimposed on a congruent luminance-defined test pattern. This was beautifully demonstrated in Sadowski's well-known “Spanish castle illusion” that can be seen on his website at www.johnsadowski.com/big_spanish_castle.php. Sadowski split a color photo of a castle into its luminance and chrominance components. After adapting to the negative chrominance picture, observers viewed a grey-scale positive, which they mistook for a color photo until they moved their eyes and realized that the “colors” in the photo were afterimages projected onto the achromatic test field. We extended these effects to show that afterimage colors spread out and average spatially within test contours but not across them (Van Lier et al., 2009). In particular, we showed that one and the same adapting pattern can induce multiple, differently colored afterimages at test locations that were not adapted to color (Movie 1A ). In addition, Van Lier et al. (2009 ) showed that, when the colored star was followed by the two outlines one after the other, the color of the afterimage switches accordingly (Movie 1B ). 

Van Lier et al. (2009) further showed that the afterimage colors also depended on the position of the adapting color fields, both on the color previously presented within the outlined area and on the color previously presented outside the outlined area (Movie 1A and 1B ). The color inside the outlined area leads to a complementary colored afterimage. However, the color outside the outlined area also leads to an afterimage inside the outlined area, but with a color similar to its original color. The latter effect is the result of contrast induction of the afterimage across the outline (Anstis, Rogers, & Henry, 1978). 

In Van Lier et al. (2009), we tested afterimages following colors inside and outside the subsequent contour with the strongest effects for the colors inside the subsequent contours. In addition to that, we also tested two-color stimuli using complementary colors (inside and outside the contours as in Movie 1A and 1B-->) and with colors that are orthogonally positioned in color space (using color settings according the Teufel-Wehrhahn color ring; Teufel & Wehrhahn, 2004). The results show clear effects revealing that the induced afterimage colors due to the colors inside and outside the subsequent contours tend to mix inside that contour. 

In the past decades, there have been many studies on color spreading within luminance contours. Cole, Hine, and McIlhagga (1993) have identified two mechanisms receiving predominantly L- and M-cone signals—a red/green (RG) detection mechanism and a luminance (LUM) mechanism (reviewed by Eskew, McLellan, & Giulianini, 1999). These mechanisms have been shown to interact in pedestal studies. Thus, Hilz and Cavonius (1970) and Hilz, Huppmann, and Cavonius (1974) showed that wavelength discrimination could be improved five- to seven-fold by introducing a luminance difference between the adjacent bars of a 4 cycles per degree (cpd) colored square-wave grating. The spatial phase between the RG test and LUM pedestal were important. Montag (1997) found that the detection/discrimination of an RG sine-wave grating was facilitated two-fold when thin dark reticle lines were placed at the zero crossings of the sine-wave grating but not when aligned with the peaks and troughs of the grating. He argued that the lines produced an effective “containment of neural integration of color,” similar to the gap effect of Boynton, Hayhoe, and Macleod (1977). Gowdy, Stromeyer, and Kronauer (1999) found strong phase dependence when both the LUM and RG gratings were low-frequency square waves, with facilitation vanishing at a relative phase of 90°. They attribute the large RG facilitation to three processes: (a) color is spatially demarcated by the luminance edges, (b) the color is then effectively integrated between the luminance edges, and finally (c) the color difference is compared across the luminance edges. 

In this paper, we shall use colored plaid stimuli to show that (a) our afterimage effects (Van Lier et al., 2009) are not limited to two-color displays, (b) test areas can also be defined by second-order contours instead of luminance contours, and (c) similar effects of color averaging occur not only for afterimages but also for “real” colors. The apparent colors of the afterimage were measured using a matching method. We argue that our effects result from spatial spreading and averaging of colors within, but not across, test contours within a surface. They also depend upon color contrast induction between surfaces. 

 

Movie 1A.

 

(Van Lier et al., 2009). A partly colored eight-pointed star is followed by two differently oriented four-pointed stars. The four-pointed star outline that includes the previously colored green points appears to be filled in with a red afterimage color, whereas the subsequently presented four-pointed star outline that includes the red points appears to be filled in with a green afterimage color. That is, the perceived colors of the afterimage depend on the geometry of black/white test contours.

 

Movie 1B.

 

(Van Lier et al., 2009). The colored star is followed by two different outlines, and the color switches accordingly.

Experiment 1: Spatial averaging of afterimage colors

Our first experiment showed that one and the same adapting four-color pattern could produce afterimages of different colors, depending upon the achromatic test contours. The adapting stimulus was a colored plaid made by transparently superimposing two square-wave gratings: a vertical blue/yellow grating and a horizontal red/cyan grating (Movie 2). This colored plaid consisted of four squares, repetitively arranged and colored purple (red + blue), orange (red + yellow), ultramarine (cyan + blue), and lime (cyan + yellow). On normal viewing of this plaid, observers typically report seeing these four combination colors and do not perceptually segregate them back into their constituent blue/yellow and red/cyan gratings. The u′ v′ values of these squares and their constituent gratings are shown in Figure 1. Their luminances are shown in Table 1. The luminance of each plaid square equaled the average of the overlapping gratings that formed it. 

 

Movie 2.

 

Adapting pattern is a four-color plaid made by superimposing a vertical blue/yellow grating and a horizontal red/cyan grating. Following adaptation to the plaid, achromatic test gratings are fixated. Result: The vertical grating shows a yellow/blue afterimage, and the horizontal grating shows a cyan/pink afterimage.

Figure 1

 

Large diamond, circle, triangle, and spindle show CIE 1976 coordinates of colors in the adapting gratings, and corresponding small symbols show subjective matches to the resulting afterimages. Lines connect all the symbols to the neutral point. Square white symbols show the four colors of the adapting plaid. Separate results from Experiment 2 are plotted as four crosses. Result: Afterimages were approximately complementary to the gratings that were not seen, not to the plaids that were seen. Conclusion: Observers spatially averaged afterimage colors within, but not across, the thin black lines or texture edges that formed the test contours.

Table 1

 

Luminance values of colors used in Experiment 1.

Contained in	Color	cd/m2
Plaid	Ultramarine	116
Plaid	Purple	105
Plaid	Lime	179
Plaid	Orange	130
Gratings	Cyan	148
Gratings	Red	118
Gratings	Blue	110
Gratings	Yellow	155

If observers adapt to this plaid and then view a white test field, most of the time they simply report a negative afterimage complementary to the adapting colors. But when the test field in Movie 2 contained thin, black, vertical lines congruent with the vertical edges of the adapting plaid, their reported afterimage was a yellow/blue vertical grating. Conversely, when the test field contained similarly congruent, thin, black, horizontal lines, the afterimage consisted of a cyan/red horizontal grating. No afterimages appropriate (i.e., complementary) to the purple, orange, ultramarine, or lime colors in the adapting plaid were ever reported. These observations suggest that the visual system spatially averages the afterimage colors together, within but not across the test contours. 

Individual differences in color naming

We asked six naïve observers to name the colors in what we thought of as red/cyan and blue/yellow afterimages. For the horizontal stripes, the red was variously called “red” or “pink,” and the cyan was variously called “blue,” “teal,” “blue-green,” or “green.” For the vertical stripes, the blue was variously called “blue,” “pink,” or “white” (and was also called the hardest color to see), and the yellow was variously called “yellow” or “green.” In sum, individual differences produced some scatter, but there was little overlap between the terms used to describe the supposedly red/cyan versus the blue/yellow afterimages. In addition, the quantitative results that follow in Figures 1, 2, and 3 show rather consistent color settings across individuals. 

Figure 2

 

Same conventions as Figure 1. Adapting gratings were now green/magenta and red/cyan. Except for the adapting green, the afterimages were close to complementary for the colors in the adapting gratings. They were not complementary to the plaid colors (square symbols).

Figure 3

 

Same conventions as Figure 1. Afterimages were again more nearly complementary to the gratings than to the plaid.

Method

Participants were five college students aged 19 to 24, tested for normal color vision with the Isihara Color plates. They received course credit for their participation. Each square-wave grating was an 18° × 18° array of 18 horizontal or vertical bars. Bars were 1° wide (0.5 cpd). Stimuli were programmed in Adobe Director 11 and presented on a 24-inch iMac. A “topping-up” adaptation regime was used. The adapting plaid was presented for 1 second, followed by the horizontally lined test display for 1 second, the adapting plaid again for 1 second, and finally the vertically lined test display for 1 second. This cycle repeated until a final response was given. For collecting data, the adapting plaid had a small 2° × 2° grey square at its center. The test fields had a similar 2° × 2° square at their centers, which was divided into two adjacent, color-adjustable 2° × 1° bars. When the test lines were horizontal (or vertical), these two bars were automatically set to be two horizontal bars, one above the other (or two vertical bars side by side). These central grey squares are not shown in Movie 2. 

During each trial, observers picked a test bar by striking keys 1, 2, 3, or 4, then moved the mouse over a colored color-picker wheel (see panel D in Movie 7.) An eye-dropper program colored the adjustable test bar with whatever color chosen on the color-wheel. When the observer had set the apparent color of the four test bars to match his/her afterimage, he/she struck the space bar, which recorded all four settings for later analysis. We used three plaids, made from three different pairs of colored adapting gratings, whose coordinates (CIE 1976) are plotted in Figures 1, 2, and 3, together with the colors that the observers chose as subjective matches to their afterimages. 

Results and discussion

In Figure 1, the four colors of the original gratings are shown as a large diamond (red), circle (yellow), triangle (cyan), and spindle (blue). The mean colors that the observers selected to match their resulting afterimages are shown as matching small symbols (diamond, circle, triangle, spindle). Lines are drawn joining each symbol to the neutral point. If the afterimages were exactly complementary to the adapting gratings, the lines from the adapting color through the neutral point to the afterimage color would be straight. This was approximately, but not exactly, true. The four colors of the adapting plaid are shown as square symbols, namely purple (red + blue), orange (red + yellow), lime (cyan + yellow), and ultramarine (cyan + blue). (The plus-symbols show the results for Experiment 2 described later.) 

The relative saturation of the afterimages is given by the ratio of the line lengths of the afterimage color divided by the adapting grating color. These ratios show that the afterimages had 35% of the adapting red, 22% of the adapting yellow, 32% of the adapting cyan, and 11% of the adapting blue. The afterimage colors were approximately complementary, not to the plaid colors to which they adapted, but to the grating colors that they never consciously saw. 

Figures 2 and 3 show similar results when the adapting plaids were green/magenta and red/cyan in Figure 2, and blue/yellow and red/cyan in Figure 3. In each case, the afterimages were more complementary to the gratings that they did not see than to the plaids that they did see. These results are consistent with the idea that, for these displays, the visual system averaged the afterimage color within, but not across, the achromatic black/white test contours. 

Note that, occasionally, when having just a white test field (without additional test lines), observers might also see afterimages of color plaids that appear to be organized in vertical or horizontal grids, a phenomenon that seems related to Shimojo, Kamitani, and Nishida (2001) who also found different afterimages on a uniform test field. This spontaneous grid afterimage might resemble rivalry between afterimage percepts (as if they were real colors). Adding vertical or horizontal contours may then bias the percepts in one direction or the other. 

Experiment 2: Afterimages followed by second-order test contours

Experiment 1 showed that the thin black lines in the test fields interacted strongly with the adapting colors. In Van Lier et al. (2009), we suggested that afterimage filling-in is driven by test contours, whether these are defined by luminance or not, and we found that illusory contours exerted a similar, although somewhat weaker, effect on afterimage spreading. Experiment 2 follows up on this idea and shows that the test contours could be second-order edges defined by moving regions of texture. 

The participants, adapting colors, and procedure were the same as in Experiment 1. The only difference was that the test field now comprised moving strips of dense black/white random dots. These test edges, defined only by motion, were second-order with no luminance differences across them. Note that any static luminance-based visual channels would be blind to these motion edges, which could be sensed only by neural motion detectors such as Reichardt (1961) units. These strips were either vertical with alternate strips moving up and down or horizontal with alternate strips moving left and right. The test strips were 1° wide moving at a speed of 1°/s. Movie 3 demonstrates this display. (The movie has only a few bars of grating in it to cut down on the file size.) 

 

Movie 3.

 

Second-order test bars made of moving achromatic random dots also induce the colored afterimages.

Results

The results are plotted as plus signs (crosses) in Figure 1. As mentioned, Figure 1 shows that adaptation to a single colored plaid can give two sets of afterimages, depending on the test conditions. The afterimage results from Experiment 1, produced by thin, black, test contours, are shown as diamond, circle, spindle, and triangle symbols. Now, the results from Experiment 2, when the test contours were second-order edges produced by moving textures, are shown in the same figure as plus signs. 

Discussion

Both test conditions produced afterimages of very similar hues and saturations, so the spatial averaging of the afterimages was basically the same whether the test contours were first-order or second-order. Apparently, both test conditions were equally effective. To further illustrate the relative robustness of the afterimage filling-in, we present a few other display variations here. Note, however, that the effectiveness of the filling phenomena also depends on various aspects such as the quality of the presentation facilities (e.g., the monitor) and the ability of the observer to keep fixated on a single point. The first additional example demonstrates that the patterns need not be striped. Movie 4 shows a checkerboard produced by transparently overlaying a red/green checkerboard on a blue/yellow checkerboard. The two checkerboards are in spatial counterphase, such that the corners of one checkerboard lie on the centers of the squares in the other checkerboard. This produces a secondary checkerboard, with squares half the size of the two component checks and colored magenta (red + blue), cyan (green + blue), orange (red + yellow), and lime (green + yellow). The two sets of black test contours in the movie are congruent with the two large component checks and yield respectively a yellow/blue afterimage and a pink/green afterimage. (The pink and yellow are more salient than the blue and green afterimages, perhaps because of a suboptimal choice of adapting colors.) 

 

Movie 4.

 

The afterimages look either blue/yellow or pink/green depending upon the position of the outline test squares.

 

Movie 5.

 

Same as Movie 4 , except that the test lines are now broken. Colored afterimages are still seen, although better for the midlines than for the corners.

The next example (Movie 5) shows that breaking up the test outlines by deleting either their corners or their midlines still yields afterimages, although these are now weaker. Such demonstrations show that spatial averaging of afterimages is robust and can be triggered by second-order or subjective test contours. Although the afterimages themselves are doubtless of retinal origin, their spatial averaging must happen at some neural level after the formation of illusory contours (see also Van Lier et al., 2009) and also after motion segregation has taken place, possibly at or after medial temporal (MT) area. Feedback connections to earlier areas may play a role as well in the formation of these afterimages. 

Spatial spreading of real colors

We argued that what is true for afterimages should be true for real colors, provided that they share some of the properties of afterimages, namely low saturation, blurred edges, and retinal stabilization. We were able to demonstrate some degree of spatial spreading by using the 16 colors provided by Teufel and Wehrhahn (2004). These colors are equiluminous and equally spaced in color space. 

Figure 4A shows a plaid made by transparently superimposing gratings using Teufel colors. Note that the colored plaid is physically identical in Figure 4A, 4B, and 4C. However, the black vertical lines superimposed on the plaid as shown in Figure 4B make it subjectively resemble the vertical colored grating contained in 4A, and the horizontal lines superimposed on the plaid as in Figure 4C make it look like the horizontal colored grating contained in 4A. Thus, what is true of afterimages can also be true for blurred, desaturated real colors; the visual system tends to average both colors between the contours. 

Figure 4

 

A, B, and C are identical colored plaids, but due to spatial averaging between the contours, either homogenous vertical yellow and blue bars (b) or horizontal green and pink bars (c) are perceived. Refer to the text.

The real colors in Figure 4 are slightly less effective than afterimage colors, perhaps because afterimages are stuck on fixed locations of the retina, whereas real colors can be shifted on the retina by head or eye movements. Saccadic eye movements across the test contours can enhance the color segregation, whereas saccades along the test contours can reduce the segregation and introduce cross talk. Thus, horizontal saccades across the vertical test lines in Figure 4B improves the color segregation and strengthens the percept of the vertical purple/lime grating, whereas vertical saccades down the same vertical test lines can lead to cross-talk intrusion of the otherwise invisible red/green horizontal grating. This intrusion is caused by adaptation; a small vertical saccade can put cyan horizontal bars onto a retinal region that previously saw red horizontal bars, enhancing their visibility. Such retinal shifts produced by head and eye movements can degrade the perceptual segregation of real colors but not of afterimage colors, which cannot move across the retina. In Movie 6, the display of Figure 4 is briefly flashed, which makes the influence of saccades less likely. The first frame of Movie 6 acts as a freeze-frame for continuous viewing. Running the movie flashes up the stimuli repetitively as an aid to avoiding eye movements. 

Experiment 3: Real colors

Method

The participants were the same as before. We used more robust, saturated colors to collect data. However, we found that spatial averaging was less effective for these strong colors, so we had to use strong test contours made by combining first- and second-order edges. Our stimulus was a complex plaid containing color, luminance, and movement (Movie 7). It consisted of two superimposed sinusoidal gratings, a blue/yellow vertical grating and a red/green horizontal grating, with a small fixation point in the middle of the plaid. When viewed with the naked eye, both gratings in the plaid were of about equal salience. But superimposed on the plaid was a set of black, horizontal, parallel lines running along the borders between red and green. In addition, the horizontal red and green bars were covered with strips of random-dot texture that moved to the left and right at 1°/s. These achromatic lines and textures were congruent with the red/green grating, and we predicted that the visual system would spatially average within, but not across, these contours. If true, this would neurally blur out the blue/yellow components and strengthen the red/green horizontal components of the plaid. In the second condition, the colored plaid was the same, but now the achromatic lines and moving textures were aligned vertically, congruent with the blue/yellow vertical component. 

 

Movie 6.

 

The panels in Figure 4 are briefly flashed as to reduce the influence of eye movements.

 

Movie 7.

 

Stimulus display for Experiment 3. (A) The basic colored plaid consisted of a blue/yellow vertical sinusoidal grating superimposed on a red/green horizontal grating. (B) The plaid was covered by stationary black horizontal lines plus strips of random-dot textures that moved left and right. Result: This looked like a red/green horizontal grating. (C) Same as (B) but with vertical lines and moving textures. Result: This looked like a blue/yellow vertical grating. Observers adjusted the color of small matching squares E and F to match the perceived color of the corresponding plaid regions (G + B, G + Y, R + B, R + Y) surrounding the fixation spot. They set these by selecting one of the squares with a keypress, then adjusting its color by hovering a mouse over the color-picker wheel D, moving the mouse around, and clicking down when they found a subjective color match. Some arbitrary hypothetical settings are shown. Results are shown in Figure 5.

In practice, in order to counter any preferences for vertical versus horizontal in either the computer or the human observers, on half the trials the plaid was rotated through a right angle so the blue/yellow grating was horizontal and the red/green grating was vertical. In analyzing the data, we lumped together the erect and rotated conditions, combining all trials that enhanced the blue/yellow components into one pot, and combining all trials that enhanced the red/green components into a second pot. 

Adjacent to this moving colored test surface were four small adjustable squares. The observer could adjust the color of each square independently using a color-picker program until he/she was satisfied that they matched the physical appearance of the four corresponding regions of the test plaid as closely as possible. Pressing the space bar printed out the results for later analysis offline. If the observer had seen the plaid veridically with no illusions, he/she would have set the four matching squares to colors representing the sums of (red + yellow), (red + blue), (green + yellow), and (green + blue), and the four settings would have been exactly the same, whatever the orientation of the test contours. However, if the observer was subjectively averaging out the colors within the achromatic contours and textured regions that were aligned with the red/green grating, as we predicted, then this process would wash out the blue and yellow components, and the observer would have set the four squares to red, red, green, and green. Conversely, if the achromatic cues were aligned with the blue/yellow grating, then we predicted that the observer would set the four squares to blue, blue, yellow, and yellow, subjectively averaging or canceling out the red and green components. Thus, the degree of color shift in the results will be a marker for the amount of spatial averaging. 

Results

Results are shown in Figure 5. Figure 5 plots both the actual stimulus colors and the selected matching colors within a CIE 1976 color triangle. The four orange squares show the stimulus colors at points where the maxima of the red/green and blue/yellow gratings represent the sums of (R + Y), (R + B), (G + B), and (G + Y). These squares are labeled with the appropriate capital letters. The blue points show the same stimuli as they appeared to the observers. Thus, RY labels the hue of the stimulus where a red and yellow stripe intersect. The point labeled Ry is the observers' matching setting for this RY stimulus, when the black overlying contours were horizontal and congruent with the red/green grating, thereby (we predicted) enhancing red at the expense of yellow. The point labeled Yr is the observers' matching setting for the same stimulus hue when the black overlying contours were vertical and congruent with the blue/yellow grating, thereby enhancing yellow at the expense of red. The length of the arrows from RY to Ry and to Yr show how far the black contours “pushed” the appearance of the crossed gratings, so the longer the arrows, the greater the illusion. 

Figure 5

 

In this CIE 1976 plot, the four orange squares show the stimulus hues. Thus RY (top-right orange point) shows where the Red of the horizontal red/green grating was superimposed on the Yellow of the vertical blue/yellow grating; likewise for GY, GB, and RB. The blue points show how the RY stimulus was shifted toward Yellow (Yr) when vertical test lines enhanced the B/Y grating, or toward Red (Ry) when horizontal test lines enhanced the R/G grating.

If there was no illusion, the solid squares would be superimposed on the open squares. If the illusion was complete, then there would be a complete averaging of colors within the black contours, and the arrows from each of the four open squares would reach out and touch each other with the arrowheads touching at the hues of the separate R, G, Y, and B hues in the component gratings. In fact, we estimated the strength of the spatial averaging by taking the total perimeter around the four orange square symbols and dividing it into the total length of all the arrows. The quotient (Arrow lengths)/Perimeter was 0.44, showing that the amount of spatial averaging (mean of 5 Ss) was 44% of the possible maximum. The most labile color, namely the one that could be subjectively pushed the furthest away from its actual value by spatial averaging, was yellow (63% of the maximum possible), followed by blue (52%), red (33%), and green (26%). 

 

Movie 8.

 

The colored plaid is followed by horizontal and vertical test lines.

General discussion

Overall, our results show that both real colors and afterimage colors can undergo spatial averaging within, but not across, contours. In addition, as we have previously shown (Van Lier et al., 2009), afterimage effects of colors at both sides of the contours may average between contours. (This comprises a merging of the negative afterimage of the colors inside the contours and the positive afterimage of the colors outside the contours.) This merging appears to happen after the colored image has been presented. Notice that, just like in the star stimulus in Movie 1B (Van Lier et al., 2009), the afterimage color switches at the moment the direction of the test lines switches from horizontal to vertical or vice versa (Movie 8). 

Note that afterimage colors do not depend on the spatial spreading of real colors in the adapting color plaid. To illustrate this, consider Movies 9A and 9B, based on color plaids similar to the ones in Figure 4. Whereas the color impression of the color plaid in Figure 4A depends on the orientation of the superimposed lines (vertical versus horizontal), the afterimage colors appear to be unaffected by the initial color impression. 

For afterimages, the test contours can either be first-order luminance contours, illusory contours, or second-order contours defined by texture or movement. “Real” colors are often more saturated and perhaps contain more luminance variation compared to afterimage colors, so spatial averaging is harder to demonstrate and often requires strong superimposed contours that are defined by both first-order luminance and second-order motion. Nevertheless, we found spatial averaging that showed similar tendencies for real and afterimage colors. 

We should distinguish between the neural site of the afterimages themselves, which are probably in the retina, and of the spatial averaging which happens much later, after the point where motion segregates different regions, namely in or after the cortical area MT. 

Francis (2009, 2010) has suggested that Van Lier et al.'s (2009) afterimage filling-in phenomena, although new, can be modeled by earlier ideas (presented in Francis & Ericson, 2004; Francis & Schoonveld, 2005) and by Francis' new simulations of the existing boundary contour system/feature contour system model of visual perception (Grossberg, 2003; Grossberg & Mingolla, 1985a, 1985b), although Kim and Francis (2011) have recently shown that the model cannot explain all phenomena related to the afterimage filling-in effect. Feitosa-Santana, D'Antona, and Shevell (2011) have recently found, as we have, that illusory contours can bound the reach of color filling-in, and Hamburger, Prior, Sarris, and Spillmann (2006) and Hamburger, Geremek, and Spillmann (2012) have studied filling-in of real colors and afterimage colors respectively. A neural representation of a contour may first exist at a retinal level or a cortical level; in either case, the contour exists at a perceptual level and bounds color filling-in. 

In the past decades, various reviews appeared on filling-in phenomena dealing with effects of color and luminance (Anstis, 2010; Komatsu, 2006; Pessoa & de Weerd, 2003). It is known that color and luminance are handled very differently by the visual system. Both receive their inputs from cone signals, but the luminance carries the sum of the L- and M-cone signals, while the opponent pathways that represent color carry the differences between cone signals. Color has much lower spatial resolution than luminance (Granger & Heurtley, 1973; Mullen, 1985; Noorlander & Koenderink, 1983; Poirson & Wandell, 1993; Sekiguchi, Williams, & Brainard, 1993a, 1993b). This makes one wonder why colors do not spill across luminance borders, creating a fuzzy colored halo across contours. Luminance contours are rendered by L- and M-cones, and the process of chroma integration seems to stop, or at least is severely discouraged, beyond these contours. Thus, the hue of an object does appear to terminate at its edge and does not seem to spill into the space beyond (Kaiser & Boynton, 1996). In general, perceived color is not only determined by the local stimulus but also by the context. Surface color is controlled by flanking edge signals, which also serve to inhibit the intrusion of signals from neighboring surfaces. Kanai, Wu, Verstraten, and Shimojo (2006) conclude that color filling-in can be governed by a host of visual cues outside the realm of first-order color and brightness via their impact on perceptual surface segmentation and segregation. Gregory and Heard (1979) hypothesize a “border locking” process which locks together edges defined by different visual properties such as luminance, texture, and color. 

 

Movie 9A.

 

In Movie 9A, a color plaid with horizontal superimposed lines alternates with a test display with horizontal lines and a test display with vertical lines.

 

Movie 9B.

 

In Movie 9B, the same color plaid, now with superimposed vertical lines, alternates with the same test displays. Although the color impressions of the color plaids are very different, the afterimage colors in the test displays appear to be the same.

Color filling-in happens in several special situations. Colors fill in across the natural blind spot and across scotomata acquired through trauma or disease. Colors also fill in with retinally stabilized stimuli and, to a lesser extent, with very blurred stimuli (Krauskopf, 1973). Peripherally viewed stimuli also fill in during prolonged fixation (Troxler, 1804), and recently Simons et al. (2006) have shown that blurred photographs perceptually fade and disappear from view, especially if their contrast is abruptly reduced during fixation. Von der Heydt, Friedman, and Zhou (2003) suggest that all these phenomena share a common neural process (Cohen & Grossberg, 1984; Gerrits & Vendrik, 1970; Paradiso & Nakayama, 1991; Walls, 1954). All these authors suggest that color signals spread in all directions except across borders formed by contour activity. The process has been compared to physical diffusion, with contours acting as diffusion barriers for the color and brightness signals. Therefore, these signals tend to fill the regions between the contours evenly, like water in the space between embankments. 

Von der Heydt, Friedman, and Zhou (2003) have explored a number of variations on this basic model, reviewed the literature, and looked for neural correlates of filling-in in cortical areas V1 and V2. They found that illusory perception of filling-in under steady fixation could be related to a gradual decay of color border signal, but they found no evidence for surface filling-in at the level of neuronal signal. They concluded that the visual system computes surface color from orientation-selective border responses. It should be noted that the neural site of color and brightness filling-in are still under debate; Paradiso et al. (2006) found that V1 neurons responded in a manner consistent with lightness perception and the spatial and temporal properties of induction. They concluded that lightness appears to be computed slowly on the basis of edge and context information. Komatsu (2006) concludes that neuronal activities in early visual cortical areas are involved in filling-in; however, Cornelissen, Wade, Vladusich, Dougherty, and Wandell (2006), using FMRI, found no evidence for such filling-in within V1. Grossberg (2003) has analyzed filling-in processes with his FAÇADE model, and Francis and Schoonveld (2005), and Van Horn and Francis (2007) have applied FAÇADE to their studies of the aftereffects produced by adapting to gratings (Mackay, 1957). Grossberg and Mingolla (1985a, 1985b) suggest that imperfections in the retina, such as veins and blind spots, break up edges, but these breaks are perceptually compensated by filling-in processes that can themselves lead to illusory percepts. They distinguish between a boundary contour process, which defines edges and fills in gaps such as in Kanisza's illusory square, and a feature contour process, which triggers a diffusive filling-in of featural qualities, such as color or brightness, within boundaries determined by completed boundary contours. Our results are much closer to their feature contour process. 

Although the discussed studies diverge on specific issues, they all suggest that filling-in proceeds in the first cortical stages in the visual cortex. Our results with afterimages suggest that retinal afterimage signals are processed in these cortical areas in a similar way as “real” colors. 

Acknowledgments

Stuart Anstis was supported by a grant from the University of California, San Diego, Department of Psychology. Thanks to Enjoli Gomez, Emily Dowgialo, Lijun Yang, Lisa Yen, and Amanda Zora for assistance in collecting and analyzing the data. 

Commercial relationships: none. 

Corresponding author: Stuart Anstis. 

Email: [e-mail]. 

Address: University of California, San Diego, CA, USA.