Dynamic brightness induction causes flicker adaptation, but only along the edges: Evidence against the neural filling-in of brightness

Abstract

Abstract

Abstract:

Abstract  Is brightness represented in a point-for-point neural map that is filled in from the response of small, contrast-sensitive edge detector cells? We tested for the presence of this filled-in map by adapting to illusory flicker caused by a dynamic brightness-induction stimulus. Thereafter flicker sensitivity was reduced when our test region was the same size as the induced region, but not for smaller, inset regions. This suggests induced brightness is represented by either small edge-selective cells with no filling-in stage, or by contrast-sensitive spatial filters at many different scales, but not by a population of filled-in neurons arranged in a point-for-point map.

Introduction

The perceived luminance, or brightness ¹, of a surface depends on the luminance of the surfaces that surround it. This is known as brightness induction. One particularly well-known example of brightness induction is brightness contrast, in which the luminance of the surrounding surfaces induces a shift in the brightness of the center surface such that the apparent contrast is increased. Thus, a medium-gray square on a bright background appears darker than a square with the same luminance on a dark background (Figure 1a). Since the gray squares are physically identical (Figure 1b), the response of photoreceptors in the retina must be the same. Since the appearance of the squares differs, however, there must be an area in the brain where that difference is represented. Even before information leaves the retina it is recoded in terms of contrast, rather than point-for-point luminance. While this would predict that the square's edges would appear different as a function of their surround (Figure 1c), it does not necessarily explain why the whole square appears to change, because early receptive fields are much too small for cells responding to the center of the square to be influenced by the surround (induction can be clearly seen for figures where the induced region has an extent of 14°; Cornelissen, Wade, Vladusich, Dougherty, & Wandell, 2006). 

Figure 1

 

(a) An example of brightness induction: The two small gray rectangles are physically identical, but the one on a white background appears darker. (b) The luminance of a horizontal cut through (a). (c) A small, contrast-sensitive spatial filter, and the response of applying that filter to the luminance profile in (b). Note that the only responses (positive or negative) are near the contrast edges. (d) A larger filter, applied to the luminance profile in (b). Because of the increased filter width, the responses now extend significantly beyond the contrast edges, predicting a shift in appearance across the entire surfaces of the small gray rectangles.

Figure 2

 

Diagram of the paradigm used in Experiment 1 (and with small modifications, in Experiments 2 and 3). Each large rectangle represents a single screen shown to the subject, with the duration that screen was visible listed below it. The experiment started with the screen on the far left and proceeded rightward. The arrows show loops in the time line, causing groups of screens to be repeated multiple times (such as the pretrial flickers, which were shown a total of 120 times in the adapt trials, or 0 times in the no-adapt trials.)

It has been theorized that the change in appearance across the whole induced region is due to a neural filling-in process. Cells that respond to the contrast edges propagate information to the cells explicitly representing the interior of the induced region, and the appearance of the interior region is entirely due to the activity of those cells. In other words, at some postretinal stage in the visual system there exists a point-for-point, retinotopic map of the brightness across the entire visual field. This is generated by analyzing all the contrast edges, as measured by relatively small contrast-sensitive filters, determining the brightness on either side of those edges, and then finally propagating that brightness via a spread of activation from edges outward until the whole point-for-point representation is complete. 

The alternative is that there is no explicit “filling-in,” and that instead, the brain implicitly interprets the brightness at all points from the neurons that were activated. Brightness models without a point-for-point explicit brightness map have mostly relied on a multiscale spatial filtering approach, where contrast is analyzed by a set of receptive fields spanning many octaves of spatial frequency. If the larger receptive fields encompass the entire induced region and part of the surround, then their response depends on both the luminance of the center and surround (Figure 1d). The highly successful ODOG (oriented difference of Gaussians) (Blakeslee & McCourt, 1999, 2001, 2004) and FLODOG (frequency-specific locally normalized ODOG)(Robinson, Hammon, & de Sa, 2007) brightness perception models make use of such large-scale contrast-sensitive filters, though they are agnostic as to how (or where) those large-scale filters are implemented. The multiscale approach has generated a lot of interest, and there exists a diverse set of recent models (e.g., Barkan, Spitzer, & Einav, 2008; Dakin & Bex, 2003; Otazu, Vanrell, & Alejandro Parraga, 2008) which differ primarily in how the different scales (and orientations) are combined and weighted, thus allowing them to account for more complex brightness illusions. None of them require an explicit filling-in stage. 

One of the best known models of the neural filling-in theory is Grossberg's FACADE (Form-And-Color-And-DEpth) model (e.g., Grossberg & Todorovic, 1988), which can account for a range of brightness illusions, including brightness induction. The filling-in model differs from the multiscale model in several ways. Consider a small, uniformly colored rectangle that grows over time. As it gets bigger, more neurons are required to represent it in the filled-in map. Meanwhile, the neurons that represented its filled-in brightness when it was smaller now just represent the brightness of the interior of the rectangle. Compare this to the multiscale model where neurons respond to contrast at different scales. As the rectangle grows, the population that represents it shifts from neurons that respond to small spatial frequencies to those selective for large spatial frequencies. 

Other authors have suggested the local edge responses themselves represent the whole brightness of the induced region. This has been called the symbolic filling-in theory, since the visual system infers the appearance of the induced square based on the low-level input, but does not then re-represent that appearance in an “analog,” or point-for-point retinotopic map (Komatsu, 2006). We are unaware of any concrete model of this precise idea, but the general idea has been suggested by several authors (e.g., Dennett, 1992). More broadly, however, multiscale models might be considered a modern evolution of this approach, because there is no explicit brightness re-representation in a point-for-point map; the edge response (i.e., contrast) at multiple scales is the final representation. We think there is still a useful distinction to be made, however, by emphasizing that symbolic filling-in uses local (high frequency) edges to represent the scene, rather than edges at a wide range of scales. 

Because the neural and symbolic filling-in theories depend on high spatial frequencies, they are well suited to explaining lightness illusions where thin edges play a critical role, such as the Cornsweet illusion (e.g., Grossberg & Todorovic, 1988) or the Koffka rings (e.g., Wist & Susen, 1973). Multiscale filtering has difficulty with these illusions because thin edges contribute little energy to the image. Conversely, these multiscale theories do an excellent job of explaining illusions like grating induction (McCourt, 1982), where a physically uniform region takes on a nonuniform brightness. Filling-in theories predict that these variations in brightness would be largely or entirely smoothed out. Thus, on the basis of visual illusions it is difficult to rule out any one theory. 

Most of the research on this issue has focused on whether there is a retinotopic, point-for-point filled-in map of brightness. Rossi and Paradiso (1996) have argued that temporal limits on the perception of brightness induction reflect a point-for-point representation that takes a long time to fill in. They showed that temporally modulating the surround only induced a change in the brightness of a center region when the modulation rate was relatively slow (<5 Hz). They found that even slower modulation was necessary for larger induced regions. This is consistent with the neural filling-in theory, where the propagation, or filling-in, of brightness takes time, which is dependent on the distance that the signal must travel from contrast edges. While this result is certainly compatible with filling-in, it may instead be due to some as-yet-unknown factor, as two recent studies (discussed below) have shown that induction is visible at much higher speeds than Rossi and Paradiso's (1996) results suggest. 

Blakeslee and McCourt (2008) investigated the time course of brightness induction using the grating induction illusion (McCourt, 1982). Subjects viewed a sinusoidal grating that was modulated in counterphase. This induced a modulating grating 180° out of phase in a medium-gray test stripe that bisected the modulating grating. Using a novel motion detection paradigm, they found little to no change in phase between the induced grating and the inducing grating as the modulation rate increased from 2 Hz to 24 Hz. This suggests that induction sufficient to drive a motion percept occurs at the same speed as the perception of the inducing grating. While this is not particularly compatible with Rossi and Paradiso's theory of slow filling-in, it is compatible with filling-in, so long as filling-in is almost instantaneous. 

We (Robinson & de Sa, 2008) have also shown that brightness induction appears to happen much quicker than suggested by Rossi and Paradiso (1996), even when using induction stimuli closely modeled on Rossi and Paradiso's experiments. Instead of measuring induction strength while the surround was constantly modulated, however, we used brief, static presentations, followed by a noise mask. Induction was clearly visible and quite strong even when the brief presentation was as short as 58 ms, and there was no indication that the spatial scale of the induced region had any influence on the temporal limits of induction. Kaneko and Murakami (2012) have shown similar results for presentations as short as 10 ms. The results of these two papers differ from Rossi and Paradiso's data across several dimensions, but are still compatible with filling-in, so long as it happens on a very fast time-scale. 

Some psychophysics paradigms unrelated to brightness induction also support the hypothesis that brightness spreads from contrast edges at a relatively slow speed. Paradiso and Nakayama (1991) studied brightness percepts elicited by briefly presenting a large white disk and then masking it with a smaller pattern, such as a black circle with a white outline. At short interstimulus intervals the mask made the white disk appear as though it had a dark hole in the center, suggesting that the addition of a contrast edge blocked the completion of a slow filling-in process. 

Paradiso and Hahn (1996) showed that steadily decreasing or increasing the luminance of a disk led to a slightly delayed change in the perceived luminance at the center of the disk. This lag is suggestive of a relatively slow filling-in process, though it is also compatible with the multiscale model if the response of larger receptive fields in later visual areas are built from the response of smaller receptive fields in early visual areas. So long as people are conscious of the response in both early and later areas, the later areas would necessarily lag in time. 

fMRI experiments in humans have produced mixed evidence for the filling-in of brightness information from contrast edges. Cornelissen et al., (2006) measured the retinotopic response to viewing a static 14° wide gray disk while the surround luminance was modulated. In both V1 and V2 they found a very strong response in the voxels corresponding to the retinotopic location of the contrast edge between the center and surround. As the retinotopic distance from the edge increased, the BOLD signal decreased, but quite slowly, suggesting a very spatially broad response to edges, broader than would be expected due to the blurring caused by the BOLD point-spread function. While this could be the signature of filling-in, they argued that it is actually only evidence of a spatially diffuse response to edges. A broad edge response was also found in the surround when the center disk modulated in luminance and the surround was held constant, even though in this condition the surround appears constant in brightness. Other studies that purport to have found fMRI evidence for filling-in may well have been measuring this broad edge response instead (e.g., Boyaci, Fang, Murray, & Kersten, 2007). 

Pereverzeva and Murray (2008) extended Cornelissen's experiment with an interesting twist—they tested induction at different luminances for the central, static disk. When its luminance was low, little illusory change was seen in the central disk from modulating the surround, and they found little increase in V1 BOLD over baseline. When the central disk had a higher luminance, the brightness illusion was clear and quite strong, and they also found a larger V1 BOLD signal across the whole disk. This suggests the BOLD signal in V1 corresponds at least in part to brightness. Like Cornelissen (2006) they measured BOLD as a function of distance from the contrast edge, and found a similar falloff in signal change as the distance from the edge increased. Puzzlingly, however, this was true even in the low-luminance condition, where no brightness induction was seen, again raising the question of whether the BOLD signal corresponds to brightness as the authors suggest, or just a broad edge response, as Cornelissen argued. In addition, it is difficult to eliminate the possibility that the broad edge response found by both studies is actually an fMRI artifact. 

Single unit recordings have produced some evidence for filling-in, but at radically different speeds. Recordings in cat areas V1 and V2 suggest that a signal does propagate from contrast edges into the center of uniform regions (Hung, Ramsden, & Roe, 2007). Based on temporal correlation of spikes from pairs of neurons, they estimated that filling-in within V1 travels between 1300°/s–2400°/s, and as fast as 4000°/s from V1 to V2, although aspects of their data led the authors to suggest the spreading activation in V1 was unrelated to brightness. In contrast, Huang and Paradiso (2008) have found evidence of much slower filling-in in monkey V1. They found many cells fired much earlier to a contrast border than to the interior of a large uniform region, and from this estimated that filling-in travels at about 270°/s. The wide difference between these two experiments may be due to the different measures used (spike-timing correlation vs. change in mean firing rate), or the difference in species. 

In the present work we introduce a new psychophysical paradigm for testing the neural filling-in hypothesis using flicker adaptation. This work builds on a study we recently published (Robinson & de Sa, 2012) measuring how flicker adaptation depends on the spatial arrangement of adapting and test regions (see also Anstis, 2013). When subjects viewed a high contrast flickering disk for a minute or more, sensitivity was drastically reduced for a low-contrast disk presented at the same retinal location, as would be expected from previous work on flicker adaptation (see Schieting & Spillman, 1987, for a brief review). What made our work novel is that we also measured flicker adaptation in the interior, by using smaller test disks significantly inset from the adapter's edges. The amount of adaptation was much smaller, but still significant. We wondered if this small but significant adaptation was the signature of a neural filling-in mechanism, since the inset disk would be “filled in” by neurons that had been desensitized² by filling in the larger adapting disk's flicker. To test this, however, we needed a stimulus that would only adapt the proposed filling-in neurons. Our current study does this by generating illusory flicker via brightness induction from actual flicker in the surround. 

We show that induction does cause strong flicker adaptation, suggesting that it isolates a population of neurons that represent the change in appearance of the induced region. This only occurs when the edges of the flickering inducer are aligned with the edges of the test region, and does not occur when the test region is shrunk (that is, inset) a few degrees. This does not support the neural filling-in theory; instead it suggests that the population of cells that are adapted are either small edge-selective cells (predicted by the symbolic filling-in theory) or spatial frequency-selective cells (as predicted by the multiscale theory). 

Experiment 1: Adapt to induction, detect polarity

In this experiment we measured the strength of induction-derived flicker adaptation for two conditions: when the test region is exactly aligned with the inner edge of the inducer, and when the test region is significantly inset relative to the edges of the inducer (Figure 3). According to neural filling-in, we expected two populations of neurons to be adapted: (a) edge-sensitive neurons responding to the contrast modulation along the border between the inducer and the inducing region, and (b) filled-in neurons across the entire induced region. Sensitivity should be reduced in the aligned condition due to populations (a) and (b). In the inset condition the edges of the inducer and the test region do not overlap, but there could still be a sensitivity reduction due to population (b). Thus, if there is neural filling-in, we would expect to find adaptation in both the aligned and inset conditions. 

Figure 3

 

Schematic diagrams of the adapting and test stimuli in an aligned condition (a), and in an inset condition (b). (Note: The illustrations are not to scale). The dashed rectangle represents the size and location of the test flicker.

Methods

Subjects

Two moderately psychophysically experienced subjects participated. Both were naive to the purpose of the experiment. 

Apparatus

Stimuli were presented on a 21-in. NEC FE2111SB CRT driven by an ATI RADEON 7000 VE video card at a refresh rate of 85 Hz. Display luminance was linearized using a 256-entry color lookup table that drove a 10-bit DAC. A Cambridge Research Systems ColorCal colorimeter was used to select the appropriate lookup table values. The lookup table was apportioned so that the full resolution of the DAC was available for luminance values between 37 cd/m² and 65 cd/m² (in steps of 0.15 cd/m²). The remaining entries covered the rest of the 0 to 102 cd/m² range in much coarser increments, allowing the display of high-contrast inducers and relatively low-contrast test stimuli. 

A chinrest was used to maintain a viewing distance of 72 cm. Stimuli were generated and displayed using Matlab running the Psychophysics Toolbox, version 2.54 (Brainard, 1997; Pelli, 1997) on a Windows XP computer. The experiment was run in a dark room and subjects adapted to the light level for 3 min before collecting data. The same apparatus was used in all experiments. 

Stimuli and procedure

Our paradigm is outlined in Figure 2. Contrast thresholds were measured in two different kinds of sessions: adapt and no-adapt. In the adapt session subjects first viewed a fixation dot centered in a gray rectangle (50.62 cd/m²) surrounded by a larger rectangle whose luminance was modulated by a 3.3 Hz square-wave (153 ms per frame) between 0 and 102 cd/m². The luminance change of the surround induced a change in the brightness of the physically static center. Subjects adapted to 120 cycles of flicker. This pretrial flicker was meant to build up a high level of adaptation before any measurements were made. Subjects were instructed to fixate the dot at the center of the screen during this and all subsequent parts of the experiment. After the pretrial flickers the measurement trials started. 

A trial in an adapt session consisted of the following screens. First, a gray screen (50.62 cd/m²) was shown for 306 ms, serving as a very brief intertrial break, and then a fixation dot was drawn, which remained on the screen for the rest of the trial. After an additional 200 ms delay, subjects viewed 20 full-contrast adapting flickers, which were identical to the pretrial flickers in all ways except in total number. These flickers served to keep subjects in an adapted state. Then three test flickers were shown. The test flickers consisted of a gray screen for 153 ms, followed by a gray rectangle that was either a small increment or small decrement relative to the background luminance, for 153 ms. On any given trial the test flickers were all either increments or decrements, and the subject's task was to report this using the keyboard after the third flicker was shown (we refer to this as the polarity task). The fixation dot disappeared after the third flicker to indicate a response was required. Subjects were encouraged to respond quickly but were given up to 5 s to respond before the trial timed out. 

To measure the effect of adaptation, we compared contrast thresholds for the adapt sessions to no-adapt sessions. In the no-adapt sessions, no pretrial flickers were shown, and a single “adapting flicker” was shown instead of 20 adapting flickers. We included one flicker to make sure that any masking effects, such as meta-contrast masking, were present in both the adapt and no-adapt sessions. In the no-adapt sessions, we collected 50 trials per session; in the adapt sessions, we collected just 36 (each trial took much longer to complete and we did not want to overly fatigue subjects). 

To measure contrast thresholds, we adjusted the strength of the test flickers using a variable stepsize staircase. The staircase stepsize started at 0.75 cd/m² (1.5% Weber contrast) and was reduced by 10% every time the subject answered incorrectly on a trial, but was not allowed to decrease below the minimum resolution of the DAC. After correct trials the flicker strength was reduced by one stepsize; after incorrect trials it was increased by two stepsizes. If the staircase reached zero flicker contrast, the stepsize was also reduced by 10% and the staircase was reset to the minimum possible flicker greater than 0% contrast. After every four incorrect responses a single easy trial was introduced to keep the subjects' attention and motivation from decreasing due to frustration. The response on the easy trial, however, did not influence the location of the staircase on subsequent trials. These parameters were found to produce good coverage of the psychometric curve during pilot tests. 

Pilot tests suggested that the adapt and no-adapt sessions produced rather different contrast thresholds. In order to minimize the number of noninformative trials, we started the staircases at different contrasts (in the no-adapt sessions 5.8% contrast, or ±2.94 cd/m² relative to the 50.62 cd/m² background; in the adapt sessions 13.3% contrast). These contrasts were significantly above threshold, and subjects completed about 10 trials before failing to detect a significant number of test flickers. Pilot work suggested, however, that the initial contrast had little effect on the final measured contrast thresholds. 

We tested two conditions (aligned and inset, see Figure 3), with two different sized surrounds. For the 10°-wide surround the inner gray square (the induced adapting region) was 4° wide. In the aligned condition the test region was also 4° wide; in the inset condition it was just 2°. Since only the size of the adapting and testing regions are of theoretical importance, we will refer to these two conditions as 4/4 and 4/2 (size of adapting region/size of test region), respectively, and omit the size of the surround. We also tested a larger scale with a surround of 14°, with a 7/7 aligned condition and a 7/2 inset condition. For each configuration we ran an average of eight sessions, split between adapt and no-adapt sessions. 

Analysis

We fit a Weibull curve to the data for each condition to estimate the percent contrast necessary to elicit 75% correct on the polarity task. A bootstrap analysis using the psignifit package (Wichmann & Hill, 2001b) was used to estimate the 95% confidence intervals of this threshold, using 9,000 bootstrap tests per curve, and allowing a lapse rate of up to 5% (Wichmann & Hill, 2001a). 

Subjects tended to make an occasional mistake on the first few trials of a session, somewhat independent of condition, so we discarded the first three trials of each session to reduce noise. 

Results

The percent contrast needed to reach a 75% correct performance threshold for each condition is shown in Figure 4. To measure the decrease in sensitivity due to adaptation, we compared the contrast threshold between adapt and no-adapt sessions. For the aligned conditions the difference was quite large for both subjects, showing that adapting to induced flicker can significantly reduce the visibility of low-contrast flickering regions. For the inset conditions, however, the difference between the adapt and no-adapt sessions is quite small, with no consistent trend across subjects, and is generally no larger than the 95% confidence intervals. This suggests that adapting to induced flicker does not reduce sensitivity to low-contrast flicker when the edges of the test region are inset from those of the inducer. 

Figure 4

 

Experiment 1. The results of adapting to induced flicker for two subjects. The rectangles on the left show schematic illustrations of the stimuli presented (Note: The illustrations are not to scale). Any reduction in sensitivity due to adapting is indicated by the adapt bars being further to the right than the no-adapt bars. Error bars denote 95% confidence intervals.

This seems at odds with the theory that brightness information is neurally filled in from contrast edges. If there is a filled-in population we would expect those neurons to also show adaptation and therefore reduce flicker sensitivity across the entire induced flicker region, even when using an inset test region (as indeed it is, after adapting to real flicker; Robinson & de Sa, 2012). 

Experiment 2: Adapt to induction, detect cycle count

Experiment 2 replicated Experiment 1 using a different task. This was motivated by the informal observation during Experiment 1 that the general presence of flicker could be detected on some trials even when the polarity of that flicker was unclear. Thus, we wanted to determine if there is something special about polarity judgments. In our new task, subjects counted the number of cycles of test flicker on each trial. We also included larger stimuli to ensure that our results still held when the inset test region was larger than 2°. 

Methods

Two psychophysically experienced subjects who were naive to the purpose of the experiment participated. One author (AR) also participated. The paradigm remains similar to that sketched in Figure 2, and is identical to that used in Experiment 1 except as discussed below. 

In this experiment we varied the number of cycles of test flicker at the end of each trial (1, 2, or 3), and asked the subject to indicate how many were shown, a three-alternative forced-choice judgment (3AFC). The fixation dot always disappeared 6 ms × 153 ms = 918 ms after the final adapt flicker so that it would not serve as a cue to actual number of flickers shown. 

We implemented two other small changes to ensure that the difference between no-adapt and adapt sessions did not depend on extraneous factors. First, we were concerned that performance on adapt trials might have been lowered because there was no cue immediately proceeding the test flickers, and thus performance could decrease if subjects' attention wandered during adaptation. Therefore, we had the fixation dot change on the final cycle of the adapt flickers from white to black. In contrast, no-adapt sessions had only one adapt flicker instead of 20, so there was little reason to worry that attention might drift. Note, however, that we implemented the fixation dot change in both adapt and no-adapt sessions to keep the stimulus properties equal. 

The other issue we addressed was that in the no-adapt trials there is only 306 ms + 200 ms = 506 ms of delay after the subject's response (see Figure 2). Since the no-adapt trials had a single adapt flicker (to equalize any masking, as discussed in Experiment 1 methods), it is possible that a small amount of adaptation might build up over the course of the 50 trials. Thus, in the no-adapt sessions we introduced an additional delay before the single adapt flicker showing the fixation dot for 1.3 s instead of just 200 ms. We did not, however, increase the delay in the adapt sessions. 

We elected to collect 50 trials in both the adapt and no-adapt sessions, since in Experiment 1 subjects had not complained of any fatigue. The adapt and no-adapt sessions were also made more similar by starting their adaptive staircases at 13.3% Weber contrast. 

We tested four different configurations of inducing regions and test rectangles: 4/4 and 4/2 conditions, both with 10° surrounds, and 8/8 and 8/4 conditions, with a surround of 15°. For each configuration we ran an average of nine sessions, split between adapt and no-adapt sessions. 

We determined the contrast threshold necessary for 66% correct performance (halfway between chance and perfect performance for a 3AFC response), using the same psignifit package settings as in Experiment 1. 

Results

The results are shown in Figure 5. All subjects show much higher contrast thresholds after adaptation in the aligned condition. The effect is somewhat weaker for subject BC, and his data are somewhat more noisy, especially in the 4/4 condition, such that the 95% confidence intervals almost overlap. This could plausibly be due to extraneous eye movements, since the receptive fields near the fovea are smaller than those in the periphery, which would mean that occasional small eye movements would be more detrimental to edge-based adaptation. This hypothesis is in agreement with the observation that all subjects show wider 95% confidence intervals for the 4/4 adapt condition than the 8/8 adapt condition. 

Figure 5

 

Experiment 2. The result of adapting to induced flicker when the task is to count the number of test flickers. The rectangles on the left show schematic illustrations of the stimuli presented (Note: The illustrations are not to scale). Any reduction in sensitivity due to adapting is indicated by the adapt bars being further to the right than the no-adapt bars. Error bars denote 95% confidence intervals.

In contrast to the aligned condition, the data from the inset conditions show no consistent evidence of adaptation, with overlapping 95% confidence intervals for all of the subjects, for both the 4/2 and 8/4 configurations. This replicates the findings from the first experiment, and shows that this result is not limited to the polarity task. 

Experiment 3: Jittering edges and slower flicker

Experiments 1 and 2 found no evidence of a pool of filled-in neurons that represent brightness. What if these neurons exist, however, but our flickering inducer fails to stimulate them sufficiently to cause a measurable amount of adaptation? In Experiment 3 we addressed two possible reasons this might occur. 

First, filling-in is thought to be a two-stage process: Initially edge units respond to local contrast, and then this information is propagated to units representing uniform areas. If the edge units themselves become sufficiently adapted, however, they might be less likely to significantly modulate the response of the filled-in neurons, preventing those cells from adapting. Subjectively, however, the brightness of the induced region did modulate during adaptation, weakening this hypothesis. Even so, this effect could have reduced the amount of inset adaptation so that it was difficult to detect. To address this, we randomly modulated the location of the inner and outer edges of the inducing region, so that there would be significantly less local edge adaptation, while still modulating the supposedly filled-in neurons. 

Second, it is possible that the 3.3 Hz modulation rate of the inducer was just too fast for filling-in to complete. If so, one would expect significantly less adaptation, particularly in the center of the filled-in region. De Valois, Webster, De Valois, and Lingelbach, (1986) found that the strength of induction does drop precipitously between 2 Hz and 4 Hz, but for most subjects induction did not disappear completely even at 8 Hz modulation rates. If the weakening of induction is related to the slow speed of filling-in, as argued by Rossi and Paradiso (1996), then this could explain why we could not measure any adaptation for the filled-in cells. Note that the slow speed of induction (and therefore filling-in) is under some dispute, however: Blakeslee and McCourt (2008) have suggested that induction is nearly instantaneous, and we (Robinson & de Sa, 2008) have argued that a single 58 ms exposure is enough time for induction to occur (at 3.3 Hz each frame of the inducer is shown for 153 ms). Nonetheless, it is true that prolonged viewing of modulation at this “high” speed would tend to reduce the apparent strength of induction, which could reduce the potential for adaptation. To this end we also explored if a ∼1 Hz (506 ms per frame) modulating inducer would reveal greater adaptation. Even by the slowest estimates, 1 Hz is slow enough that filling-in should complete, and furthermore, 1 Hz modulation causes stronger induction than 3.3 Hz. 

Methods

We used the same three subjects as in Experiment 2. Our paradigm differs from Experiment 1 in the following ways. First, after every full cycle (two frames) of the pre-adapt or adapt flickers, the inner and outer edges of the inducing region were moved within a 1° range (in step sizes of 1/32 of a degree). For our 6/2 configuration this meant that the inner edge of the inducing region could be anywhere between 6° and 7° in width. The size of the outer inducing edge (13°) was always expanded by the same amount as the inner edge, which loosely preserved the degrees of visual angle subtended by the inducing region. Note that this jitter made it meaningless to test an aligned configuration, so our only configuration was inset. 

We changed the no-adapt sessions to have exactly the same temporal properties as the adapt sessions to make them equally fatiguing, since fatigue could potentially reduce contrast sensitivity (note that this would actually tend to increase the difference between adapt and no-adapt sessions). To this end we kept the number of flickers the same between sessions, but changed the location of the flickers in the no-adapt sessions so that the flicker would contribute minimally to any visual adaptation at the target location. In no-adapt sessions all but one of the pre-adapt and test flickers were shown in a 4.15° × 32° rectangle abutting the top edge of the screen. These top-flickers covered on average the same amount of visual angle as the flickering region centered around the test rectangle in the adapt sessions. They did not induce any change in the brightness of the rest of the screen. 

Thus, in no-adapt sessions, subjects first viewed 120 flickers occurring at the top of the screen while fixating at the center of the screen. Then, each trial was preceded by 19 top-flickers, followed by a single regular-induced flicker surrounding the test patch, and then finally the test flicker was shown. In adapt sessions no top-flickers were shown; all adapting flickers were centered on the fixation point, as in Experiment 1. 

As in Experiment 2 we had the fixation dot change from white to black on the final cycle of the adapt flickers to cue subjects that the test was about to start. We returned to the polarity task used in Experiment 1, but used just one cycle of test flicker on all trials. This protected against the possibility that by the time three test flickers had been shown the filled-in neurons had already unadapted significantly. 

We collected data for 3.3 Hz and 1 Hz flicker speeds. Experimental sessions took dramatically longer in the 1 Hz condition, so in order to maximize the amount of useful data, we adjusted the staircase procedure to start just a little above the contrast needed for 100% performance. A brief pilot experiment determined that a single set of parameters achieved this for all of our subjects: 3% initial contrast, with an initial stepsize of 0.5% contrast. The same parameters were used in the 3.3 Hz condition. We collected 50 trials per session in the 3.3 Hz condition and 30 trials per session in the 1 Hz condition. We collected an average of four sessions for each speed (split between adapt and no-adapt sessions). 

Results

We determined the contrast threshold necessary for 75% performance, using the same psignifit package settings as in Experiment 1. The contrast thresholds are shown in Figure 6. In both the 1 Hz and 3.3 Hz conditions there was no evidence of increased contrast thresholds in the adapt condition. These data replicate the general finding from the first two experiments that adapting to induced flicker does not cause flicker adaptation in the interior of the induced region. Thus, the lack of adaptation does not appear to be due to confounds from overly rapid flicker in the inducer, or because adaptation to edge contrast prevented adaptation from occurring in the supposedly filled-in neurons. 

Figure 6

 

Experiment 3. The result of adapting to induced flicker when the location of the inducing rectangle's edges are jittered in time. The rectangles on the left show schematic illustrations of the stimuli presented (Note: The illustrations are not to scale). The top set of bars shows the results when subjects adapted to 3.3 Hz flicker (the same speed used in the previous two experiments), and the bottom pair of bars shows the results for 1 Hz. Any reduction in sensitivity due to adapting is indicated by the adapt bars being further to the right than the no-adapt bars. Error bars denote 95% confidence intervals.

General discussion

We have shown that adapting to induced flicker can cause significant reductions in contrast sensitivity to real flicker, but only in the situation where the edges of the inducer are aligned with the edges of the test region. When the test region is inset, it appears that there is no reduction in sensitivity. 

We know that there are at least some cells that would respond to the physical modulation of luminance within a uniform region, such as photoreceptors in the retina, temporal contrast mechanisms, or nonzero sum³ contrast-sensitive cells. Our previous work (Robinson & de Sa, 2012) showed that these cells can be adapted by luminance flicker, using the same basic paradigm as the three experiments reported here. Our results here suggest that the same population of cells does not respond to induced brightness changes. Even though the entire rectangle appears to flicker during induction, it seems that the cells that signal or represent this change in appearance are those with receptive fields intersecting the contrast edge of the flickering region. Perhaps this is because contrast across an edge is much more informative with respect to the material properties of a region than the response of cells that only signal relative changes in the center. For instance, the contrast ratio across an edge is preserved in a global illumination change. 

Our results are not consistent with the theory that brightness information “fills in” from edge-selective neurons into a point-for-point representation of brightness across the entire visual field. In particular, if such a filled-in representation exists, we would expect that it, too, is susceptible to induced flicker adaptation, just like the cells intersecting the contrast edge, and that this adaptation would have some effect on contrast sensitivity in regions inset from the edges. 

There are some caveats, however, to our conclusion that brightness does not fill in. It is possible that brightness is represented in a filled-in population, but for some reason that population is not susceptible to flicker adaptation. This would be quite surprising, since adaptation is a relatively ubiquitous phenomenon, which is found throughout the visual system (Webster, 2011), but of course it cannot be ruled out. Perhaps adaptation to real flicker serves a useful purpose early in the visual system, such as gain control, but would be of no use to “filled-in” cells? 

There is a large literature showing that adapting to high contrast luminance flicker reduces sensitivity to low contrast flicker, but it is not necessarily clear why this occurs. While visual adaptation phenomena were originally explained in terms of neural fatigue, many of those results have since been recast as a change in tuning properties to increase sensitivity to common stimuli (Webster, 2011). Thus, high contrast flicker might result in gain control adjustments that make low contrast flicker difficult to see. This would predict that discriminations around the contrast level of the adapting stimuli should actually improve after adaptation, which to our knowledge has not been tested. For spatial contrast (gratings), however, there is no evidence of increased discrimination ability at the adapting contrast (Barlow, MacLeod, & van Meeteren, 1976). Indeed, flicker adaptation in particular may still be best explained as simple desensitization (for which neural fatigue is but one mechanistic explanation). Under strict fixation, a high-contrast flickering spot in the periphery will lose contrast, and then disappear entirely (Schieting & Spillmann, 1987). Adapting to near-threshold flicker also causes reduced sensitivity, such that ever-increasing contrast is required to keep the adapting dot visible over time (Anstis, 1996), which is difficult to explain with gain control. Thus, while it is certainly open to debate, we think it is more probable that flicker adaptation causes general desensitization of mechanisms sensitive to that flicker, leaving little reason to expect that it would be limited to just the early areas of the visual system. If we are wrong, however, at the very least our results suggest that brightness perception has the unusual property of not adapting, which is still an interesting finding. 

Another more plausible objection is that even though we adapted the supposedly filled-in neurons, we did not adapt the edge-selective neurons that were internal to the filled-in region. Thus, in the inset condition, the unadapted edge-selective neurons were able to signal for the edges of the inset flickering rectangle. While the adaptation of the filled-in neurons may have occurred, this edge signal could have been present with no reduction in strength, and perhaps this is the signal that subjects used to govern their response. It is impossible to rule this out, though we have three arguments for why it seems implausible. First, if filling-in is an important component of representing brightness, then we would expect that interfering with it would cause some reduction in sensitivity—otherwise why would the visual system need to fill in at all? Second, if filling-in neurons were adapted, but edge neurons were not, then one might expect some rather strange percepts in the inset condition—such as flickering edges with no center, sort of like a wireframe rectangle. At least subjectively, we did not observe any such percepts. Finally, and most convincingly, if the edge signal alone is sufficient for full sensitivity, the experiments reported in Robinson and de Sa (2012) should not have found a reduction in sensitivity for inset test regions after adapting to real flicker. 

What about other theories of brightness representation? Our results are compatible with the symbolic theory of filling-in. Adapting the edge-selective cells that code for filled-in regions reduces contrast sensitivity to other filled-in regions that are represented by those same cells (as in the aligned condition). An inset region would stimulate a different set of edge cells, and therefore would not be subject to adaptation from real or induced flicker. 

Our results are also compatible with the multiscale theory. Adapting to a flickering inducer will affect all of the receptive fields at the appropriate scales to detect the contrast across the edges (Figure 7a), and also the much larger receptive fields that code for the contrast of the center relative to the surround. These larger receptive fields would be poorly suited for signaling the presence of a smaller test rectangle, since they would be heavily adapted, and furthermore the test rectangle stimulates a much smaller part of their positive (or negative) centers. Meanwhile, an intermediate-sized receptive field, whose center just covers the smaller test rectangle, is ideally suited for signaling its presence (Figure 7e), and would not be adapted from the flickering inducer at all. 

Figure 7

 

(a) Adapting stimuli from the present study and (b) Robinson and de Sa (2012), and the multiscale model response assuming zero sum filters. Note that the set of filters stimulated are the same. (c and d) The same stimuli, but with nonzero sum filters, causing (d) to differ from (b). (e) The test stimulus is the same for both studies, with detection depending on the smaller scale filter, whether zero sum or not.

It is worth noting that neither symbolic filling-in nor the multiscale theory as currently formulated can account for our earlier finding (Robinson & de Sa, 2012) that real flicker adapts inset test regions. The multiscale model fails because the smaller receptive fields would not be stimulated by the adapter (Figure 7b), leaving them spared to respond fully to the inset test (Figure 7e). There is, however, a relatively straightforward modification to the multiscale model, which is to use filters that are slightly unbalanced, that is, nonzero sum (the positive and negative portions of the filter do not sum to zero). This means a small part of their response is due to the mean luminance, and not just the contrast between center and surround. This would actually make the model more in line with neurophysiology evidence (e.g., Croner & Kaplan, 1995). If nonzero sum filters were used, the adapter in our 2012 experiment would also cause some adaptation in the small scale filters (Figure 7d), and a reduction of sensitivity would be seen for inset test regions that stimulate those filters. Using nonzero sum filters would not change the predictions for the present paper; only the large-scale filters would be stimulated by the adapter since there is no actual luminance change in the induced region (compare Figure 7a and c). 

Meanwhile there is no obvious modification to the symbolic filling-in theory that would likewise allow it to account for our 2012 data. To be fair, however, we have only tested this proposed modification to the FLODOG model briefly, so we do not know if it would change its ability to predict a wide range of illusions. Encouragingly, Barkan et al. (2008) reported that their multiscale model works with nonzero sum filters, although they did not present any data nor discuss how their results change. This seems like a worthy direction for future research on multiscale models. 

The multiscale approach also offers a compelling resolution of seemingly conflicting findings in the literature as to whether a brightness change has the same effect in the visual system as an actual luminance change. Cornsweet and Teller (1965) argued that only an actual luminance change can influence sensitivity. Detecting small luminance increments is easiest on a black background. Placing the increment on a luminance pedestal reduces sensitivity; the higher the luminance of the pedestal, the greater the reduction. They found, however, that if the pedestal's brightness is increased by means of induction, there is no loss in sensitivity. On the other side of the debate, McCourt and Kingdom (1996) demonstrated that detecting low-contrast luminance gratings could be facilitated by adding an illusory grating to the test field by means of grating induction (adding a real grating has the same effect). Our study also suggests that luminance and brightness can have the same effect, because induced flicker caused real flicker adaptation, when aligned properly. Our results highlight in particular the importance of the spatial relationship between the luminance and brightness stimuli. 

Cornsweet and Teller's (1965) findings can be explained by the multiscale theory because the test increment was significantly inset from the outer edge of the pedestal. Thus, an induced brightness change would have no effect on a spatial filter tuned to the size of the test region. An actual luminance pedestal, however, would stimulate the surround of this spatial filter, reducing sensitivity. Note that the symbolic filling-in theory can potentially explain Cornsweet and Teller's finding as well, but has difficulty explaining the grating induction stimulus used by McCourt and Kingdom (1996), where nonuniform lightness is induced into a uniform region, or how this induced grating could combine additively with an actual grating. In the multiscale model induced and physical brightness are the product of the same neural units, so it has no difficulty predicting McCourt and Kingdom's results. 

Our results suggest that it would be interesting to revise Cornsweet and Teller's (1965) study, varying the spatial parameters; when the test increment is significantly less inset, we would predict that a brightness pedestal would indeed reduce sensitivity. 

Converging evidence supporting the importance of spatial alignment comes from a recent study of chromatic afterimages by Kim and Francis (2011). They extended the illusion introduced by Van Lier, Vergeer, and Anstis (2009) that demonstrates what appears to be color averaging of chromatic afterimages within achromatic borders. Kim and Francis varied the alignment of the achromatic borders (lines) so that they were inset (and outset) relative to the afterimage. According to filling-in theories the lines block the spreading of filled-in color, so changing their location should change the size of the chromatic afterimage. Instead, the spread of color often extended past the lines suggesting that they only block “filling-in” when the lines are aligned with the edge of the chromatic afterimage. The authors point out that since color spreading was still seen, their results still support filling-in generally, though in a very different form from current models. We propose a different explanation: The lines serve to draw attention to or enhance the response of cells aligned with the chromatic contrast due to the afterimage. This makes it easier to locate the edge of what is a fairly noisy signal (the chromatic afterimages are quite faint and have indistinct edges due to small eye movements during adaptation). In any case, their results and ours highlight the importance of studying spatial relationships and alignment to better understand the visual representation of lightness, brightness, and color. 

Conclusions

Many studies have been published on the issue of how brightness is represented in early visual areas. While there is no consensus, our results provide converging evidence from a new paradigm that a point-for-point, filled-in representation is unlikely. While we doubt this debate will be resolved soon, we suggest that it would be productive to consider alternatives to the point-for-point filled-in theory, and that it is time to start designing experiments that can differentiate between other possible methods of representing brightness, such as those described in this paper. 

Acknowledgments

We would like to thank Don MacLeod for his help designing these experiments, and Paul Hammon, Stuart Anstis, and Karen Dobkins for helpful discussions about this work in general. This work was supported by the National Science Foundation under NSF CAREER Grant IIS-0133996 to VR de Sa and NSF grant SBE-0542013 and SMA-1041755 to the Temporal Dynamics of Learning Center. AE Robinson was supported by NSF Grant DGE-0333451 to GW Cottrell/VR de Sa, NSF grant SBE-0542013 to the Temporal Dynamics of Learning Center, and NIH grant EY01711. 

Commercial relationships: none. 

Corresponding author: Alan E. Robinson. 

Email: [e-mail]. 

Address: Department of Psychology, University of California, San Diego, CA, USA. 

Footnotes

¹   Usage in the literature sometimes differs; here we will use the term brightness exclusively to mean perceived luminance.

Footnotes

²   This desensitization might be due to neural fatigue, contrast adaptation, or some other factor. Our approach does not depend on the reason adaptation happens, only that it changes the response properties of mechanisms sensitive to flicker (see Conclusions for further discussion).

Footnotes

³   Neurons where the negative and positive regions are not perfectly balanced.