Monocular signals in human lateral geniculate nucleus reflect the Craik–Cornsweet–O'Brien effect

Abstract

The human visual system has a remarkable ability to accurately estimate the relative brightness of adjacent objects despite large variations in illumination. However, the lightness of two identical equiluminant gray regions can appear quite different when a light–dark luminance transition falls between them. This illusory brightness “filling-in” phenomenon, the Craik–Cornsweet–O'Brien (CCOB) illusion, exposes fundamental assumptions made by the visual system in estimating lightness, but its neural basis remains unclear. While the responses of high-level visual cortex can be correlated with perception of the CCOB, simple computational models suggest that the effect may originate from a much lower level, possibly subcortical. Here, we used high spatial resolution functional magnetic resonance imaging to show that the CCOB illusion is strongly correlated with signals recorded from the human lateral geniculate nucleus. Moreover, presenting the light and dark luminance transitions that induce the CCOB effect separately to each eye abolishes the illusion, suggesting that it depends on eye-specific signals. Our observations suggest that the CCOB effect arises from signals in populations of monocular neurons very early in the human geniculostriate visual pathway.

Introduction

A large class of brightness illusions has been categorized as “filling-in” phenomena. The Craik–Cornsweet–O'Brien illusion (Cornsweet, 1970; Craik, 1966; O'Brien, 1958) is a particularly striking example in which the perceived lightness of a region of uniform luminance can be profoundly altered by the presence of a luminance gradient along all or part of the border enclosing the region (Figure 1). In the example shown, central regions of the forehead and beret are of identical luminance but appear very different. It has been widely assumed that such computations of relative brightness are a high-level mechanism carried out at a cortical level (Boyaci, Fang, Murray, & Kersten, 2007; Huang, MacEvoy, & Paradiso, 2002; Pereverzeva & Murray, 2008; Perna, Tosetti, Montanaro, & Morrone, 2005; Roe, Lu, & Hung, 2005; Rossi & Paradiso, 1999; Rossi, Rittenhouse, & Paradiso, 1996), consistent with the observations that high-level interpretation of a scene, such as the perceived curvature, orientation, and depth of a surface can influence the magnitude of the effect (Knill & Kersten, 1991; Purves, Shimpi, & Lotto, 1999). 

Figure 1

 

Example of the Craik–Cornsweet–O'Brien illusion. In this example, (A) a high-contrast black and white image of Che Guevara has been filtered with a center–surround, Laplacian-of-Gaussian (LoG) filter to produce a series of light–dark transitions at the locations of edges in the original image. In (B) the CCOB image, regions corresponding to the forehead and hair are of identical luminance (see inset below illustrating the luminance profile for a horizontal section through the figure—marked by the dashed white line on either side of the image) but continue to induce a strong but clearly illusory sense of relative lightness and darkness.

Although the context in which the light–dark border is presented can enhance the CCOB effect, the illusion can be driven purely by border information in the absence of any coherent high-level information (Burr & Morrone, 1994; Cohen & Grossberg, 1984; Davidson & Whiteside, 1971; Gerrits & Vendrik, 1970; Grossberg & Todorovic, 1988; Maddess, Davey, Srinivasan, & James, 1998; Paradiso & Nakayama, 1991; Rudd & Arrington, 2001; Figure 2A). 

Figure 2

 

Stimuli and power spectra. Sample stimuli for the four experimental conditions in Experiment 1: (A) Craik–Cornsweet–O'Brien illusion—“CCOB,” (B) CCOB with low SFs phase scrambled—“scramLo,” (C) CCOB with high SFs phase scrambled—“scramHi,” (D) similar pattern with physical change in luminance matched to the perceived change in luminance for CCOB—“physical” match (see Methods section). See also Figures A1A–A1C for enlarged images. For (A) the CCOB stimulus, regions adjacent to the dark portion of the contour appear darker than those adjacent to the light portion of the contour. This illusory percept is abolished when the (B) low SFs are phase scrambled but preserved when only the (C) high SFs are phase scrambled. Luminance profiles (for a horizontal section indicated by the dashed black line) found beneath each example demonstrate that the mean luminance across regions is constant for stimuli A–C, whereas physical changes in luminance occur for condition D. Each stimulus type has been color coded for correspondence with the power spectra shown in panel E. (E) The power spectra for conditions A–C were indistinguishable and had ∼7.5× the energy compared to condition D (varied slightly between participants depending on their matching contrast), where energy for a signal x is ∣F(x)∣² and F is the Fourier transform. The shaded regions highlight the range of SFs that were phase scrambled in the scramLo condition (red) and scramHi condition (green).

Several low-level models of this phenomenon (Grossberg, 1994; Pessoa, Mingolla, & Neumann, 1995) share a common conceptual similarity, using significant image features to infer a brightness value that is then propagated across space (at a cortical level) to “fill in” regions of homogenous luminance. However, recent psychophysical observations (Dakin & Bex, 2003) challenge this orthodoxy by showing that phase scrambling low—but not high—spatial frequencies (SFs) in the image destroys the CCOB illusion (see Figures 2B and 2C). This is important since it shows that introducing large amounts of luminance fluctuation into previously uniform areas does not greatly affect the illusion, even though such a manipulation would be catastrophic for models relying on “brightness propagation” between boundaries. 

Sensitivity of the CCOB to the spatial frequency structure of the image led Dakin and Bex (2003) to propose that the mechanism responsible for the CCOB illusion operates by amplification of the weak low SF structure of the image (to bring the image statistics into line with natural scenes, in which low SFs are greatly over-represented), rather than via propagation of a neural signal across space. Specifically, the model works by filtering an incoming image with a bank of SF-tuned filters, and then iteratively reweighting and summing the filter outputs to reconstruct an image with as-close-to-natural statistics as possible. This approach predicts the CCOB and its variants including the missing fundamental illusion¹ (Kingdom & Simmons, 1998), as well other filling-in illusions such as White's effect (White, 1979). Critically, this model works optimally with isotropic (non-orientation-tuned) mechanisms. This led us to hypothesize that neural signatures of brightness filling-in may be seen at very early, possibly subcortical, stages of visual processing. 

Although single unit electrophysiology has established a key role for early cortical visual areas (notably V1 and V2) in brightness perception in macaque monkeys (Huang et al., 2002; Kinoshita & Komatsu, 2001; Roe et al., 2005) and cats (MacEvoy, Kim, & Paradiso, 1998; Rossi & Paradiso, 1999; Rossi et al., 1996), there is considerably less work on brightness-correlated responses in the LGN (Rossi & Paradiso, 1999; Valberg, Lee, Tigwell, & Creutzfeldt, 1985). Modulating the luminance of the far-surround on neurons centrally stimulated with either a uniform luminance patch or drifting bars results in significant modulation of the responses of more than half of LGN neurons (Rossi & Paradiso, 1999). Furthermore, changes in steady-state illumination of the surround of LGN neurons can facilitate or suppress responses in a manner consistent with simultaneous brightness contrast (Valberg et al., 1985). However, such experiments have essentially characterized the influence of surrounding luminance on LGN response to conventional stimuli, using conditions not designed to produce clear illusory shifts in brightness. In this sense such approaches may not be optimal for studying the neural analogues of brightness perception in humans. 

In humans, cortical responses correlated with perceived brightness are seen as early as V1 (Boyaci et al., 2007; Pereverzeva & Murray, 2008), as well as in higher areas of the dorsal visual stream (Boucard, van Es, Maguire, & Cornelissen, 2005; Cornelissen, Wade, Vladusich, Dougherty, & Wandell, 2006; Perna et al., 2005). However, no investigation of the LGN has been undertaken, despite the evidence from animal electrophysiology reviewed above. Therefore, we set out to investigate whether responses early in the human retino-striate visual pathway correlate with perceived brightness. In four linked experiments we provide converging evidence to suggest that signals correlated with perceived brightness arise from populations of monocular neurons in LGN and primary visual cortex. 

Experiment 1: BOLD signals recorded from the human LGN correlate with perceived brightness

All procedures for Experiments 1– 3 were approved by the local ethics committee, and informed consent was obtained from all participants. 

Methods

Participants and stimuli

Nine healthy volunteers, aged 25 to 39 years, with normal visual acuity took part. 

Participants fixated centrally while viewing annular stimuli that subtended 4–16° of visual angle (i.e., extending from 2 to 8 deg eccentricity from the fovea in each hemisphere). For the main experiment there were four stimulus conditions: (1) “CCOB” (Figure 2A), (2) CCOB with low SFs scrambled—“scramLo” (Figure 2B), (3) CCOB with high SFs scrambled—“scramHi” (Figure 2C), and (4) physical luminance match (Figure 2D). See Figures A1A– A1C for enlarged images. All stimuli were polarity reversed at 1 Hz. This rate is optimal for inducing maximal modulations in V1 responses correlated with illusory changes in perceived brightness (Rossi & Paradiso, 1999). 

Stimuli were generated by convolving white noise patterns with a 2D Laplacian-of-Gaussian filter, and then thresholding (at mean response level) to generate a spatially broad-band black-and-white “blob” image. These patterns had a fundamental frequency of 1.2 blobs/deg (i.e., 0.6 c/deg). Blob images were either contrast-adjusted, to form the physical luminance stimuli (see details below), or were refiltered with a Laplacian-of-Gaussian (yielding a peak SF of 120 c/image, equivalent to 7.5 c/deg for our stimuli) and contrast-scaled to the maximum range to produce the CCOB stimuli. Stimuli for the scramLo and scramHi conditions were generated by phase-scrambling either the low or high SF structure, respectively (see Figure 2E for the range of SFs scrambled). The SF ranges employed were selected so that the RMS contrast of both ranges was matched and so that the sum of the two ranges yielded the original signal. Note that we used SF ranges with a shallow roll-off to avoid “ringing” artifacts that arise from the use of sharp cut-off filters. Phase-scrambled conditions were matched in RMS contrast to the CCOB condition and had indistinguishable power spectra ( Figure 2E). 

For the physical luminance match control condition (condition D), each subject performed a psychophysical experiment prior to scanning to determine the physical change in luminance that best matched the perceived change in luminance for the CCOB illusion. The CCOB stimulus was presented alongside an image of a similar pattern but unfiltered, i.e., defined by physical modulations in luminance. Participants were asked to adjust the contrast of this latter image until it best matched the perceived contrast of the adjacent CCOB image. The initial contrast of the physical match image was reset at the beginning of every trial to a value randomly assigned between ±14% in steps of 1%. Contrast could be incremented or decremented (using a key press) in steps of 0.1% to achieve the closest match. This procedure was repeated 16 times and the mean contrast value then used to generate images for condition D (contrast settings ranged from 4.2% to 5.4% across participants). The RMS contrast of the physical match was on average 13% of that for stimuli from conditions A–C (note that the power for the physical match varied slightly according to the matching contrast set by each participant). 

All images used in the fMRI and psychophysics experiments were presented on a gamma corrected display. To ensure steady central fixation was maintained throughout, participants were instructed to monitor and count how many times the fixation cross changed color, which occurred at irregular intervals. Fixation was monitored throughout using an ASL504 LRO infrared video-based MRI compatible eye tracker (Applied Science Laboratory, Bedford, MA) sampling at 60 Hz, with a spatial resolution of 0.5°. 

Paradigm

In a block design paradigm each of the four conditions were presented twice per experimental run, with each condition occurring in the first and second halves of the run in a randomized order. Block order was randomized between runs and between participants. Each block lasted 31 s (10 volumes), interleaved with a fixation baseline for 18.6 s (6 volumes). An additional 5 volumes were acquired at the start of each experimental run to achieve steady-state magnetization. Each participant performed 6 runs of the main experiment (133 volumes per run) and also underwent an “LGN localizer” (Haynes, Deichmann, & Rees, 2005) to functionally locate voxels in the right and left LGN, as well as standard retinotopic mapping procedures to localize V1, V2, and V3 (Sereno et al., 1995; see below for details). 

Localizing the LGN

We functionally identified the LGN independently in each of our nine participants by contrasting contralateral with ipsilateral hemifield stimulation (Haynes et al., 2005; Figure 3A). Participants fixated centrally while passively viewing blocks of flashing black and white checkerboards (8 Hz), which stimulated either the right or left hemisphere for 27.9 s (9 volumes), interleaved with a fixation baseline for 21.7 s (7 volumes). Each participant performed 2 runs of 133 volumes each (4 repetitions of each hemifield + 5 initial dummy scans that were discarded) immediately following the main experiment. Checkerboard wedges extended between 2° and 8° from fixation and overlapped the visual area stimulated by the annular images used in the main experiment (Figure 2). A central fixation task was performed to ensure steady fixation throughout. 

Figure 3

 

BOLD signals recorded from the LGN. (A) LGN localizer—Voxels within an anatomically defined region of the posterior thalamus that responded more strongly to contralateral versus ipsilateral checkerboard stimuli were used to identify the right and left LGN, respectively (see Methods section). (B) BOLD signal changes measured in bilateral LGN for each of the four experimental conditions in Experiment 1. Mean signal change (compared to fixation baseline) across all nine participants is plotted for each of the four conditions. Error bars—SEM for the group. Our three experimental conditions (A–C) elicited significant differences in LGN activity (F(2,16) = 3.959, p = 0.040). Critically, there was a significant reduction in activity comparing the CCOB condition with the scramLo condition, for which the CCOB effect does not exist (t(8) = 3.092, p = 0.015). However, activity evoked by the scramHi condition, for which the CCOB effect persists, did not differ significantly from that evoked by the CCOB condition (t(8) = 1.190, p = 0.268). Thus, there was a strong correlation between signal strength and perception of the CCOB illusion. These differences cannot be explained by differences in stimulus power, as both scrambled conditions had power spectra indistinguishable from that for the CCOB (Figure 2E). Furthermore, the physical match condition evoked comparable activity to that for the CCOB (t(8) = 0.835, p = 0.428), despite a significant reduction in energy (condition D had ∼13% of the energy in conditions A–C).

Localizing primary visual areas

Retinotopic areas V1–V3 were also identified for each individual in a second scanning session using standard procedures (Sereno et al., 1995; see Figure A2). Participants fixated centrally while passively viewing flashing (8 Hz) black and white checkerboards, which stimulated either the horizontal or vertical meridian for 19.5 s (15 volumes), interleaved with a fixation baseline of 13 s (10 volumes). Each participant performed 2 runs of 255 volumes each (5 repetitions of each meridian + 5 initial dummy scans that were discarded). To ensure steady central fixation throughout, participants performed a central fixation task. 

fMRI acquisition

A 3T Siemens Allegra scanner, with standard head coil, was used to acquire all functional and structural data. For the main experiment and LGN localizer a high-resolution EPI sequence (matrix 128 × 128, field of view 192 mm, in-plane resolution 1.5 × 1.5 mm, slice thickness 1.5 mm, TE 30 ms, acquisition time per slice 102 ms, TR 3.1 ms) was used to acquire 30 slices, positioned to optimize coverage of the LGN and occipital pole. A high-resolution T1-weighted structural image (1 × 1 × 1 mm) was also acquired for every participant. 

For retinotopic mapping a standard EPI sequence (matrix 64 × 64, in-plane resolution 3 mm ², slice thickness 2 mm with 1-mm gap, TR 1.3 s) was used to acquire 20 slices oriented parallel to the calcarine sulcus and positioned to optimize coverage of the occipital cortex. 

fMRI analysis

Imaging data were analyzed using SPM2 ( http://www.fil.ion.ucl.ac.uk/SPM). The first 5 images from each experimental run were discarded and the remaining high-resolution images from the main experiment and LGN localizer were realigned and coregistered to each participant's T1 structural image. For the main experiment, data were smoothed using a 3-mm isotropic Gaussian smoothing kernel. A linear combination of regressors representing the time series for each of our 4 experimental conditions and fixation baseline was convolved with a synthetic hemodynamic response function and its temporal derivative, creating a box-car function. The general linear model, as employed by SPM2, was used to generate parameter estimates for each regressor at every voxel. 

For the LGN localizer, data were smoothed using a 2-mm isotropic Gaussian smoothing kernel. Time series representing stimulation of the right hemisphere, left hemisphere, and rest were modeled. To identify the left LGN, voxels within an anatomically defined region of the posterior thalamus that showed greater activity for right field stimulation compared to left field stimulation (p = 0.05, cluster threshold 20 voxels) were inclusively masked with those voxels that showed greater activation for right field stimulation compared to rest (masking threshold: p = 0.01, cluster threshold >20 voxels). Similarly, to identify the right LGN, voxels that showed greater activity for left field stimulation compared to right field stimulation were inclusively masked with voxels showing greater activation for left field stimulation compared to rest (Figure 3A). Mask volumes for both LGN were created and the parameter estimates averaged across all voxels within the mask, for each condition of interest. The pattern of responses for the right and left LGN was indistinguishable so data were averaged bilaterally. 

To identify V1, V2, and V3, mrGray (http://white.stanford.edu/~brian/mri) was used to segment white and gray matter and for cortical flattening. fMRI activation elicited by the meridian mapping stimuli was superimposed on the flat map of visual cortex and the boundaries between V1, V2, and V3 (dorsal, d, and ventral, v) defined by the transitions between voxels representing the horizontal and vertical meridians (see Figure A2). Mask volume images were created for V1, V2, and V3 for each participant and the fMRI signal associated with each of our experimental conditions, versus baseline, extracted. A threshold of p < 0.001 uncorrected was used to determine significance for each condition of interest in accordance with our prior hypotheses. 

Control experiment

Two participants from the main experiment also performed a control experiment to ensure that the high SF structure of the images used in the main experiment was “visible” to the human LGN. Participants viewed annular stimuli with the same spatial configuration as the main experiment but now containing just the high SF structure of our original CCOB images. Figure 2E shows the range of spatial frequencies included in this image. This stimulus was not RMS matched to the original CCOB stimulus. These “Hi-only” images were presented in 31 second blocks (10 volumes), interleaved with a fixation baseline for 18.6 s (6 volumes)—i.e., the same timings used in the main experiment. There were four blocks of “Hi-only” images per experimental run and participants performed 2 runs each. Data were analyzed in the same way as for the main experiment. 

Results and discussion

In Experiment 1, we constructed three abstract annular stimuli whose amplitude and power spectra were physically identical but that varied in their ability to evoke the CCOB effect (Figures 2A– 2C). The “CCOB” stimulus induced a strong perception of filling-in (Figure 2A). Randomizing the phase of the low SF structure in this stimulus destroyed the CCOB effect (Figure 2B, “scramLo”). However, similar phase randomization of the high SF content led to only a slight attenuation in the perceived contrast of the illusion (Figure 2C, “scramHi”; see results for Experiment 2 for psychophysical assessment of the degree of filling-in experienced). We hypothesized that brain areas whose activity reflected perception of the illusion would show strong reductions in activity for the “scramLo” stimulus compared to the “CCOB” stimulus (even though their amplitude and power spectra were identical), but little or no reduction for the “scramHi” stimulus. We also included a control condition where physical changes in luminance were psychophysically matched (on a per-participant basis) to the perceived change in luminance induced by the CCOB stimulus. This condition was not matched for RMS contrast but provided a further control for assessing brightness-correlated responses. 

First, we functionally identified the LGN in each of the nine participants using high-field functional MRI at high spatial resolution (Figure 3A). We then measured Blood Oxygenation Level Dependent (BOLD) signals evoked by visual stimuli in the three experimental conditions (Figures 2A–2C). Consistent with our hypothesis, BOLD signals recorded from the LGN showed a strong correlation with perception of the CCOB effect (Figure 3B) with stimuli from our three experimental conditions eliciting significant differences in LGN activity (F(2,16) = 3.959, P = 0.040). These differences were accounted for by a significant reduction in activity comparing the “CCOB” stimulus, where a strong brightness filling-in effect was perceived, with the “scramLo” stimulus, for which this perceptual effect was no longer present (t(8) = 3.092, P = 0.015). In contrast, the activity evoked by the “CCOB” stimulus did not significantly differ from that evoked by the “scramHi” stimulus where a comparable brightness filling-in effect was perceived (t(8) = 1.190, P = 0.268). As the “CCOB” stimulus and both high and low frequency scrambled stimuli had almost identical power spectra (Figure 2E), the differences in LGN activity we observed cannot be accounted for by differences in stimulus power. Moreover, the perceptually matched physical-luminance control stimulus evoked activity levels statistically indistinguishable from the “CCOB” stimulus (t(8) = 0.835, P = 0.428), a finding that is again consistent with activity in the human LGN correlating with perceived brightness. 

BOLD signals recorded from retinotopically identified areas V1–V3 showed a qualitatively similar pattern of findings, although now with additional significant reductions in activity comparing activity evoked by the CCOB stimulus and the high spatial frequency scrambled stimulus (see Figure A3 for V1–V3 fMRI responses). Thus, activity in the LGN showed the best correlation with perception of the CCOB effect. 

The stimuli used in the main experiment were designed to optimize perception of the CCOB effect. Although the low SFs in this stimulus were well within the sensitivity range of LGN neurons, some of the high SF structure lay at the limits of receptive field sizes recorded in non-human primates (Kilavik, Silveira, & Kremers, 2007; Kremers, Silveira, & Kilavik, 2001; Levitt, Schumer, Sherman, Spear, & Movshon, 2001). LGN receptive field sizes are also known to increase with increasing eccentricity (Irvin, Casagrande, & Norton, 1993; Xu, Bonds, & Casagrande, 2002; Xu et al., 2001) and with low contrast stimuli (Kilavik et al., 2007; Kremers et al., 2001). Therefore, it is conceivable that the high SFs in our annular stimuli (which extended 2–8 deg from the fovea) were not within the sensitivity range of human LGN neurons. If this were the case, then our observation that randomizing the phase of the high SFs had no effect on LGN activity might be accounted for by an absence of high spatial frequency sensitivity rather than by a correlation of neuronal signals with perceived brightness. To rule out this possibility, we performed a control fMRI experiment that examined whether human LGN responded to the high spatial frequencies in the original CCOB stimulus (see above). 

We compared BOLD responses evoked by the high SF images with a uniform gray background. The “Hi-only” images evoked significant activity in the LGN (t(1) = 67.196, p < 0.001, one-tailed), as well as V1 (t(1) = 6.266, p = 0.05, one-tailed), V2 (t(1) = 9.275, p = 0.034, one-tailed), and V3 (t(1) = 8.972, p = 0.036, one-tailed; Figure A4), confirming that the human LGN does indeed respond to stimuli whose spatial frequency spectra are restricted to just the high SF content of the original CCOB image. 

Thus, we can conclude that the observations of a reduction in signal in the human LGN in association with scrambling low spatial frequencies in our CCOB stimulus were not confounded by a lack of sensitivity of the human LGN to the high spatial frequency content of our stimuli. 

Experiment 2: Effect of dichoptic viewing on CCOB illusion

The LGN contains monocular neurons segregated into eye-specific layers (Jones, 1985; Shatz, 1996). These layers remain segregated on entering striate cortex at layer IV, and only subsequently does integration of these monocular pathways occur. Our fMRI findings therefore suggest that the CCOB effect reflects activity in monocular pathways. However, as BOLD contrast fMRI signals can also reflect dendritic potentials (Logothetis, Pauls, Augath, Trinath, & Oeltermann, 2001), it remains possible that the modulations we observed in LGN reflect a feedback signal from binocular neurons in striate cortex, where processing of surface brightness has been established (Haynes, Lotto, & Rees, 2004; Huang et al., 2002; Kinoshita & Komatsu, 2001; MacEvoy et al., 1998; Roe et al., 2005; Rossi & Paradiso, 1999; Rossi et al., 1996). To distinguish the possible contribution of binocular neurons in V1 from monocular neurons in either V1 or LGN to the fMRI signals we observed, we conducted a second psychophysical experiment. 

Methods

We constructed novel dichoptically presented CCOB stimuli ( Figure 4A) where one eye viewed only the light portion of the border contour and the other eye viewed only the dark portion of the border contour. When viewed with both eyes open, the light and dark portions of the border were aligned side by side, as in the original CCOB image. 

Figure 4

 

Brightness filling-in under dichoptic viewing conditions. (A) Schematic diagram showing dichoptic presentation of the CCOB stimulus in Experiment 2: the light portion of the CCOB stimulus was presented to one eye and the dark portion to the other eye, assigned at random. The light and dark contours were aligned side by side in the binocular percept. A contrast nulling procedure was used to establish whether the CCOB illusion was present under dichoptic viewing conditions (see Methods section). The schematic also illustrates how information from the right hemifield is carried to the left LGN and vice versa—a fact we utilize when localizing the LGN. Monocular information from each eye remains segregated in this pathway and terminates in distinct layers of the LGN. (B) Throughout the contrast nulling, a random dot letter stereogram task was performed at fixation to ensure binocular fusion (see Methods section). When binocularly fused a letter T appeared to be in front of the background image (outlined in red for illustrative purposes only). (C) Mean contrast nulling values for each of the 6 participants for each condition (red triangle = group mean value). Error bars—SEM. High nulling contrasts were required to negate the strong effects of brightness filling-in experienced for both the monocular CCOB (5.2%, SD ± 0.5%) and scramHi conditions (5.1%, SD ± 0.6). However, brightness filling-in was all but abolished for the monocular scramLo condition (1.3%, SD ± 0.7). Crucially, a similar reduction in nulling contrast was found for the CCOB viewed dichoptically (2.5%, SD ± 0.8), confirming that the CCOB effect is significantly attenuated when viewed dichoptically.

To assess the magnitude of brightness filling-in for each of our experimental conditions (A–C) and to assess whether the CCOB persists under dichoptic viewing conditions, six healthy volunteers performed a contrast nulling experiment while maintaining central fixation and viewing (A) CCOB monocularly, (B) scramLo monocularly, (C) scramHi monocularly, or (D) dichoptic CCOB. All participants (24–39 years, with normal VA) gave informed consent to take part. Stimuli were presented at 120-Hz frame rate on a CRT display viewed through LCD shutter glasses (http://www.nuvision3d.com) that were synchronized to the frame refresh of the monitor using an infrared transmitter (giving an effective frame rate of 60 Hz per eye). For the monocular conditions (A–C) stimuli were randomly presented to either eye. For the dichoptic condition, only the light section of the CCOB image was presented to one eye and only the dark section to the fellow eye, assigned at random (see Figure 4A). 

An additional pedestal image—of identical pattern defined by physical luminance change—was superimposed onto the binocular percept and participants were asked to adjust the contrast of this image, using key presses that incremented/decremented in steps of 4%, 2%, or 1%, until the entire image appeared to have uniform luminance. The initial contrast value for the pedestal image was reset at the beginning of every trial (within a range of ±20% contrast, in steps of 2%) chosen at random. Participants were not informed of which condition they were viewing at any time. Sixteen trials of each condition were presented in a randomized order and the mean nulling contrast determined for each condition, for each participant. 

To ensure central fixation and steady binocular fusion throughout, and to rule out the possibility of binocular rivalry during the dichoptic condition, a random dot stereogram was simultaneously presented at fixation during all conditions (Figure 4B). At the beginning of every trial, participants were required to report the orientation of the letter T (tail pointing north, south, east, or west). Participants would not be able to perform this central task if they failed to achieve binocular fusion (all participants performed with 100% accuracy on this task). Subjects were asked to stop the trial if this 3D percept ever disappeared. All subjects reported experiencing a steady 3D percept of the central stereo image. Images for conditions A–D were identical in size, angular subtense, and luminance to those used in Experiment 1. The results are shown graphically in Figure 4C. 

Results and discussion

We reasoned that if the CCOB illusion arises from monocular signals, then separating the light and dark portions of the border that induce the effect and presenting them to each eye independently should attenuate, if not abolish, the illusion. However, if the illusion arises from binocular signals that subsequently feedback onto monocular pathways, then it should be preserved. 

For the CCOB stimulus, participants set a nulling contrast of 5.2% (SD = 0.5%), confirming that brightness filling-in was present (Figure 4C). A similar nulling contrast was required for the monocular scramHi condition (5.1%, SD = 0.6%), which was not statistically different from the monocular CCOB condition (t(5) = 0.252, p = 0.811). However, under dichoptic viewing conditions, a significantly lower nulling contrast of 2.5% (SD = 0.8%; t(5) = 6.709, P = 0.001) was required to subjectively achieve equal brightness. Similarly, brightness filling-in was all but abolished for the monocular scramLo condition (1.3%, SD = 0.7%), with a significant reduction in nulling contrast compared to the monocular CCOB (t(5) = 9.533, p < 0.001). Crucially, the nulling contrast required for the monocular scramLo was statistically indistinguishable from the dichoptic CCOB condition (t(5) = 2.481, p = 0.056), confirming that the CCOB illusion was greatly attenuated by dichoptic presentation. These findings show that monocular channels are essential for generating the CCOB effect. 

Experiment 3: The CCOB is a strictly monocular effect

It is conceivable that the fusional demands of the dichoptic CCOB image used in Experiment 2 were greater than those required to maintain perception of the central depth-defined T image. If this were the case, it is possible that the two components of the CCOB (separate light and dark portions of the border, presented dichoptically) were not perfectly fused throughout the entire contrast nulling procedure, which could have abolished the CCOB percept. To test our hypothesis that the CCOB arises from monocular signals, we therefore performed a third psychophysical experiment. Using the same experimental set-up and nulling procedure used in Experiment 2, we presented the original CCOB image to one eye and a phase-scrambled version of the stimulus (i.e., noise) to the other eye. Under these conditions, if monocular signals are driving the CCOB illusion, we hypothesized that there would be no (or only very little) noise-related degradation of the CCOB effect. Alternatively if the CCOB is primarily a binocular phenomenon the CCOB effect would be strongly reduced. 

Methods

Using the same experimental set-up as Experiment 2, six participants (five new and one that had participated in Experiment 2) viewed a CCOB stimulus presented to one eye (top left image, Figure 5) plus a phase-scrambled version of the CCOB stimulus (i.e., noise) presented to the other eye (top middle image, Figure 5). All participants (aged 27–39 years, healthy and with normal VA) gave informed consent to take part. When the noise stimulus was matched to the CCOB stimulus for RMS contrast (conferring broadly equal visibility), all participants experienced complete suppression of the noise stimulus; that is, subjects only perceived the CCOB image. Increasing the contrast of the noise quickly led to binocular rivalry. To counteract this, we generated phase-scrambled noise that was confined to the internal regions of the “blobs” in any given CCOB image (see example in middle panel of Figure 5) and set the RMS contrast to twice that of the CCOB image. Under these conditions participants performed a contrast nulling procedure, making 12 adjustments each to perceptually abolish the CCOB effect (see bottom left panel of Figure 5 for example where the CCOB is nearly abolished). 

Figure 5

 

Monocular signals drive the CCOB effect. Schematic diagram illustrating the CCOB image presented to one eye (top left image) and a phase-scrambled version, in which noise was confined to the internal regions of the “blobs” in the CCOB image, presented to the other eye (middle panel). A nulling procedure was used to abolish the CCOB effect (bottom left panel illustrates a partially nulled CCOB image). The right panel illustrates two possible binocular percepts. When the contrast level of the pedestal image was set to induce clear light–dark differences being perceived between regions of the CCOB image (either physical or illusory) subjects could not fuse the CCOB stimulus with the noise stimulus. Instead, subjects experienced strong binocular rivalry and tended to just see the image containing the CCOB stimulus—as illustrated in the top right image. However, at the point when subjects added a pedestal that perceptually nulled the CCOB effect, they tended to experience much more fusion between the CCOB image and the noise image—as illustrated in the bottom right panel. Importantly, introducing the noise image to one eye had no effect on the strength of the CCOB illusion presented to the other eye, confirming that the CCOB effect is driven primarily by monocular signals.

The same participants also performed a contrast nulling procedure for the CCOB image presented monocularly with no noise presented to the other eye (i.e., identical to condition A in Experiment 2) to ensure that a comparable nulling contrast was obtained for this group but also to provide a contrast level with which to compare any degradation of the CCOB effect due to presenting noise to the fellow eye. This latter condition (CCOB monocularly only) gave estimates of the illusion that were almost identical to our reported findings for condition A in Experiment 2: 4.8% (SD ± 0.9%). Importantly, introducing the noise image to the other eye had no effect on the strength of the CCOB illusion—nulling contrast 5.1% (SD ± 1.3%), thus showing negligible noise-related degradation (t(5) = −0.380, P = 0.719) and confirming that the CCOB effect is driven primarily by monocular signals. 

Interestingly, while performing this experiment we observed a phenomenon that would only be expected if the CCOB was strictly monocular. During the nulling procedure, while the contrast level of the pedestal image was set to induce clear light–dark perceptual differences between regions of the CCOB image (either physical or illusory), participants could not fuse the CCOB stimulus with the noise stimulus. Instead, participants experienced strong binocular rivalry or just saw the image containing the CCOB stimulus (see example in top right panel of Figure 5 where the CCOB image strongly dominates the binocular percept). However, at the point when participants added a pedestal that perceptually nulled the CCOB effect, they experienced much more fusion between the CCOB image and the noise image (see the image in the bottom right panel of Figure 5, where the CCOB and noise are depicted as largely fused). Such fusion is to be expected if rivalry is driven by perceptual brightness rather than by the physical lightness differences between surfaces. When the illusion is perceptually nulled the inter-blob regions appear more uniformly gray and thus local fusion with the noise pattern is more likely to arise. These results accord with an earlier report that brightness illusions can either prevent or promote rivalry depending on whether they, respectively, decrease or increase the perceived difference between stimuli presented to each eye (Andrews & Lotto, 2004). 

Experiment 4: Effect of region size on the CCOB illusion

In humans, there have been mixed reports of how early lightness is encoded in the visual pathway. Until recently, only two higher order areas of the dorsal visual stream had been shown to correlate with perceived brightness. In this study, primary visual cortex responded equally well to a CCOB stimulus and to a filtered (Hilbert transformed) version that did not evoke the illusion (Perna et al., 2005). However, the visual stimulus used in this study was much larger (subtending 15 deg in diameter) than in the present study and qualitatively did not evoke strong sensations of brightness filling-in (see Figure A5). Larger regions (with fixed size light–dark transitions) yield less filling-in; indeed the CCOB exhibits broad scale invariance, i.e., as region size becomes bigger one must use broader luminance transitions at the border to achieve the same size effect (Wachtler & Wehrhahn, 1997). We tested this finding using our own stimuli in order to determine if the different findings from our study and that of Perna et al. (2005) might be accounted for by differences in the size of the “filled-in” regions. To this end, we conducted a fourth psychophysical experiment where the strength of the illusion was measured using the same contrast nulling procedure used in Experiment 2 (see above) for various sizes of regions comprising the CCOB stimulus. 

Methods

Qualitatively, it appears that the size of the region enclosed by the border contour modulates the size of the CCOB effect (Figure A5). To quantify the effect of region size on brightness filling-in, we had participants perform a contrast nulling task (similar to Experiment 2) while viewing CCOB images containing one of 7 possible region sizes, ranging from 2 regions/deg to 0.088 regions/deg. Three healthy volunteers (25–32 years, with normal VA) participated. Each participant performed 8 trials for each region size, presented in a random order. All other details of the experimental set-up were identical to those for Experiment 2. 

Results and discussion

Figure 6 illustrates how the magnitude of the CCOB illusion fell as region size increased. CCOB stimuli comprised of relatively small regions, such as those used in our imaging experiment, required high nulling contrasts (Figure 6) indicating the presence of robust filling-in. However, as region size increased, the nulling contrast fell. At a region size that matched the earlier study (Perna et al., 2005), no nulling contrast was required. Thus, observers fail to show objectively the presence of any brightness filling-in for large regions. These data support our earlier contention that the source of discrepancy between our own findings and those of Perna et al. (2005) originate from these authors having presented their stimuli in a manner that did not lead to robust filling-in. More recently, and in line with our findings, neural activity in V1 has convincingly been shown to correlate with perceived brightness for stimuli that demonstrate robust changes in induced brightness (Boyaci et al., 2007; Pereverzeva & Murray, 2008). 

Figure 6

 

Region size dependence of CCOB effect. The strength of the CCOB illusion, measured using the contrast nulling technique, varies with region size. See Methods section for full details of Experiment 4 and Figure A5 for a schematic illustration of how CCOB weakens as region size increases. The graph plots the contrast nulling values for three subjects (black symbols; red triangle is group mean; error bars, SEM) plotted as a function of region size. For the region size used in the present study, a nulling contrast of 5.2% was required to counter the effects of brightness filling-in, whereas brightness filling-in was abolished well before the region size used in a previous study investigating the neural correlates of the CCOB illusion (Perna et al., 2005).

Note that the model as described by Dakin and Bex is scale-invariant (treating all SF components equally). Incorporating the visibility of different SF components would allow the model to explain the effects of region size on the CCOB. Furthermore we anticipate that this would also explain why, e.g., white noise patterns do not appear fractal, simply because the low frequency components of white noise patterns are likely to be invisible. 

Discussion

In four linked experiments (plus additional control experiments) we have demonstrated that signals arising in the earliest post-retinal stages of human monocular visual processing (LGN) correlate with the perceived brightness of our CCOB stimuli. 

Although processing associated with surface brightness is found in all layers of striate cortex (Huang et al., 2002; Kinoshita & Komatsu, 2001; MacEvoy et al., 1998; Roe et al., 2005; Rossi & Paradiso, 1999; Rossi et al., 1996), including layer IV (which receives monocular inputs from the LGN), the LGN itself has rarely been examined and only a very small subset of LGN neurons show brightness-correlated responses (Rossi & Paradiso, 1999). The apparent discrepancy between the strong modulation of population responses measured here (using BOLD contrast fMRI) in association with changes in perceived brightness induced by the CCOB illusion versus single-unit electrophysiology may reflect the stronger correlation of BOLD activity with local field potentials than with spiking activity (Logothetis et al., 2001; Viswanathan & Freeman, 2007). 

Our findings raise the question of which particular visual property is reflected in LGN responses. Although luminance remains constant across the CCOB as well as our two scrambled conditions (see luminance profiles in Figures 1A– 1C), the induced contrast between regions in our CCOB stimulus is strongly modified (approximately 5%) and falls well within the contrast sensitivity range of the human LGN measured using fMRI (Kastner et al., 2004). Neurons in the geniculostriate pathway primarily respond to luminance contrast within their receptive fields (Hubel & Wiesel, 1962) and changes in local contrast, in the absence of any change to the local luminance, modulate LGN responses (Mante, Frazor, Bonin, Geisler, & Carandini, 2005). Indeed, Cornelissen et al. (2006) raised the possibility that V1 responses correlated to changes in perceived surface lightness can be explained by long-range contrast signals at the surface border rather than to the perceived surface lightness change. However, a recent fMRI study simultaneously (inversely) modulated the influence of border contrast and perceived lightness within the same stimulus and showed that the fMRI BOLD signal in V1 correlates with perceived lightness and not border contrast (Pereverzeva & Murray, 2008). Given the similar profile of activity we observed in Experiment 1 for the LGN and V1, it is likely that the LGN responses do indeed reflect perceived lightness. However, our findings do not resolve whether this reflects processing within the LGN or arises at even earlier stages of visual processing. 

The dependence of the CCOB effect on spatial scale suggests a mechanism that involves the selective activation and/or deactivation of scale-sensitive neurons. Neurons in the dorsal, parvocellular (P) layers of the LGN (layers 3–6) have small receptive fields with low contrast gain, whereas the more ventral magnocellular (M) layers (layers 1–2) have much higher contrast gain and respond to very low contrast stimuli (Sclar, Maunsell, & Lennie, 1990; Shapley, Kaplan, & Soodak, 1981). The greater contrast sensitivity of M cells has been attributed to their larger receptive field size. Based on these neurophysiological findings, the BOLD signal changes we recorded from the LGN (which were dependent on the integrity of low SF structure) may reflect a dominant influence from M cells. This would be consistent with M cells primarily controlling cortical contrast response at low contrasts (e.g., Allison, Melzer, Ding, Bonds, & Casagrande, 2000). 

Although our behavioral and neuroimaging findings converge to provide evidence in favor of a monocular origin for the low-level component of the CCOB, they do not speak against an additional role for high-level information. Such information can modify the magnitude of the CCOB effect (Purves et al., 1999) and would likely be mediated by cortical structures. Our novel dichoptic version of the CCOB stimulus demonstrated that presenting the light and dark portions of the CCOB stimulus to each eye separately significantly attenuates brightness filling-in. These findings establish that eye-specific signals are necessary for the CCOB illusion but may not be sufficient, and indeed, some residual nulling contrast was required to negate the CCOB effect in this condition (Experiment 2). 

Our findings are consistent with the approach that visual behavior is optimized for the statistical properties of the natural environment (Corney & Lotto, 2007) and corroborate the predictions of a simple computational model (Dakin & Bex, 2003) suggesting a critical dependence of the CCOB effect on the integrity of weak low SF structure. Interestingly, the magnitude of low SF boosting required to account for the effect is consistent with the visual system effectively renormalizing the SF structure of the CCOB to conform to the statistics of natural scenes. It is proposed that this renormalization results from contrast gain control designed to optimize the response of SF-tuned neurons to natural stimulation. Our present findings suggest a central role for LGN in such gain control, a suggestion consistent with recent computational and electrophysiological studies (Mante et al., 2005). Recasting the model in terms of neural reweighting (rather than at the level of the whole image) would allow for modeling of more localized effects. 

If our results arise from reweighting of SF-tuned filter responses, what evidence is there that cortical neurons are capable of changing their tuning properties in response to natural stimuli? Recently it has been reported that SF tuning properties of most neurons in primary visual cortex adapt very quickly to the spatial frequency structure of natural scenes (Sharpee et al., 2006). This can serve to maximize information transmission by allowing neurons to optimize their responses; e.g., neurons reduce their sensitivity to low SF structure when exposed to natural scenes (that over-represent such scales) so tending to normalize their responses. It is easy to see how presentation of the CCOB (which under-represents low SFs) would lead to boosting of low SF structure within such a scheme, and this is the shift in responsivity required for the model outlined by Dakin and Bex (2003). We therefore speculate that the underlying mechanism for the changes in SF tuning observed by Sharpee et al. (2006) is through SF selective changes in response gain of neurons in the LGN and that it is essentially this gain change that we see reflected in the fMRI signal. 

Appendix A

Figures A1–A5. 

Figure A1

 

(A) “CCOB.” Sample stimulus for condition A. Original CCOB illusion. Brightness filling-in occurs between adjacent regions. (B) “scramLo.” Sample stimulus for condition B. CCOB with low spatial frequencies phase scrambled, high spatial frequencies preserved. Brightness filling-in is abolished. (C) “scramHi.” Sample stimulus for condition C. CCOB with high spatial frequencies phase scrambled, low spatial frequencies preserved. Brightness filling-in persists.

Figure A2

 

Localizing the primary visual areas. Meridian map for (A) right and (B) left visual cortices of two different participants. Red represents greater activity for horizontal meridian, green represents greater activity for vertical meridian. The border of V1 and V2 lies along the vertical midline, the border of V2 and V3 lies along the horizontal midline. (C, D). Cortical activation produced by the annular stimuli used in the main experiment represented on the same flat maps of right and left visual cortices, for the two different participants (color coding: red represents greatest activity, blue represents lowest activity).

Figure A3

 

BOLD signals for V1, V2, and V3. BOLD contrast responses in (A) V1, (B) V2, and (C) V3, averaged across all 9 participants, for all conditions compared to fixation baseline. Error bars represent the SEM for the group.

Figure A4

 

BOLD signals in response to high spatial frequencies. (A) BOLD signal changes measured in bilateral LGN in response to viewing images comprised of only the high SF component of the original CCOB stimulus, compared to fixation baseline. Mean signal change (± SEM) compared to fixation baseline across two participants is plotted. Viewing the high SFs only elicited significant LGN activity ( t(1) = 67.196, p < 0.001, one-tailed), confirming that the high SFs were indeed “visible” to LGN neurons. (B) BOLD signal changes were also significant in V1 ( t(1) = 6.266, p = 0.05, one-tailed), V2 (t(1) = 9.275, p = 0.034, one-tailed), and V3 (t(1) = 8.972, p = 0.036, one-tailed). Mean signal change (± SEM) compared to fixation baseline across two participants is plotted.

Figure A5

 

Effect of region size on strength of CCOB illusion. Illustration of the effect of region size on the CCOB illusion. For maximum effect, the image should be viewed from a distance of approximately 10 cm. It is qualitatively apparent that the strength of brightness filling-in varies from left to right across the image. The illusion is highest for small region sizes (left side of the image) and gradually attenuates as region size increases. The far right regions in this image are still much smaller than that used in a previous study investigating the neural correlates of the CCOB illusion (Perna et al., 2005), which did not find BOLD signal modulation in V1 (see also Figure 5).

Acknowledgments

We would like to thank Peter Bex, Petroc Sumner, and Bahador Bahrami for valuable advice on this manuscript. This work was supported by the Wellcome Trust. 

Commercial relationships: none. 

Corresponding author: Elaine J. Anderson. 

Email: [e-mail]. 

Address: Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London WC1N 3BG, UK. 

Footnote

Footnotes

¹  Note that the missing fundamental illusion retains sufficient low SF information to drive the model given input filters with biologically plausible SF bandwidths.