Perceptual grouping-dependent lightness processing in human early visual cortex

Abstract

Lightness, the perceived relative achromatic reflectance of a surface, depends strongly on the context within which the surface is viewed. Modest changes in the two-dimensional configuration or three-dimensional scene geometry may lead to profound variations in lightness even though the surface luminance remains constant. Despite recent progress, we are far from a complete understanding of how various aspects of spatial context affect lightness processing in the cortex. Here we use a novel stimulus to show that perceptual grouping through occluders can affect lightness. We first report behavioral results showing how lightness across occlusion depends on spatially distant image features, including luminance and contrast. Next using functional magnetic resonance imaging (fMRI) we show that human early visual cortex responds strongly to occlusion-dependent lightness variations with little or no attention. These results suggest that elements of three-dimensional scene interpretation play a role in early cortical processing of lightness.

Introduction

The flanking regions of the top image in Figure 1a are identical, yet to most observers the rightmost flank appears lighter than the leftmost flank. The stimulus is composed of four rectangles and two vertical bars. Two outer rectangles are uniform and identical in luminance, and they flank two central rectangles, one of which has higher, the other has lower luminance than the flanks. This creates three contrast borders, but the vertical bars are positioned in a way to occlude the two borders between the outer rectangles and the central ones ( Figure 1b). In this configuration, the flanks appear to differ in lightness even though their luminances are identical. However, as the bottom image in Figure 1a clearly demonstrates, this lightness effect is abolished when the contrast of the central border is too high. Intuitively, the reason is simple: the disjoint rectangular parts are grouped together by our visual system behind the occluders and this affects the lightness of the flanks. It is as though the lightnesses of the central rectangles “spread” to the flanks (“lightness assimilation”). However when the difference between the flanks and central rectangles becomes too high, as is the case for the bottom image in Figure 1a, the visual system no longer completes the rectangles behind the occluders, thus the lightness effect vanishes. This demonstration shows that perceptual grouping behind an occluder can affect the perceived lightness of a surface. This kind of perceptual grouping is known as amodal completion and it was previously shown to affect perceived transparency and induce neon light spreading (Nakayama, Shimojo, & Ramachandran, 1990). 

Figure 1

 

The “lightness effect”. (a) Perceptual grouping through amodal completion affects lightness. Physically identical flanking regions appear to differ in perceived lightness in the upper image: the left flank looks darker than the right flank. However, this lightness effect is not observed in the lower image (some observers even report a reversal of the effect, reminding us of the “simultaneous contrast” effect). This is presumably because of the breakdown of perceptual grouping due to the large luminance difference between the flanks and the central regions. (b) Perceived lightness and actual luminance along a horizontal cross-section.

In the long history of lightness research in visual sciences, similar demonstrations have been convincingly showing that contextual factors, including two-dimensional (2D) configuration and articulation (Arend & Spehar, 1993; Land & McCann, 1971; Logvinenko, 1999; Moulden & Kingdom, 1991; O'Brien, 1958), and three-dimensional (3D) scene layout and perceptual organization (Adelson, 1993; Anderson & Winawer, 2005; Bloj, Kersten, & Hurlbert, 1999; Boyaci, Doerschner, Snyder, & Maloney, 2006; Boyaci, Maloney, & Hersh, 2003; Doerschner, Boyaci, & Maloney, 2007, Gilchrist, 1977; Hochberg & Beck, 1954; Kitazaki, Kobiki, & Maloney, 2008; Knill & Kersten, 1991; Pereverzeva & Murray, 2009; Purves, Shimpi, & Lotto, 1999; Ripamonti et al., 2004) can dramatically affect the perceived lightness of a surface. Context-dependent lightness effects reported in previous studies are explained either with 3D global context-aware models, including illumination estimation models (Bloj et al., 2004; Boyaci et al., 2003; Speigle & Brainard, 1996), anchoring theory (Gilchrist et al., 1999), and probabilistic models (Kersten, Mamassian, & Yuille, 2004; Purves et al., 1999), or by models relying on 2D localized image features, including contrast models (Blakeslee & McCourt, 1999; Moulden & Kingdom, 1991), edge integration models (Land & McCann, 1971; Rudd & Zemach, 2005; Shapley & Reid, 1985), and scission models utilizing special “junctions” in the image (Khang & Zaidi, 2002). However, as yet, there is no biologically plausible model that could explain a wide range of lightness phenomena. This is partly because the neuronal underpinnings of lightness perception are still largely unknown. 

The role of early visual cortical areas in lightness processing

There is an ongoing debate regarding the role of early visual areas in lightness processing. Several recent studies, both with humans and animals, have found lightness-related activity in early visual areas (Anderson, Dakin, & Rees, 2009; Boyaci, Fang, Murray, & Kersten, 2007; Haynes, Lotto, & Rees, 2004; Pereverzeva & Murray, 2008; Rossi, Rittenhouse, & Paradiso, 1996; Sasaki & Watanabe, 2004), while some studies found no such evidence (Cornelissen, Wade, Vladusich, Dougherty, & Wandell, 2006; Perna, Tosetti, Montanaro, & Morrone, 2005), and some others offered mixed results (McCourt & Foxe, 2006; Roe, Lu, & Hung, 2005). Previous research studying context-dependent lightness processing in the early visual cortex usually used stimuli in which the lightness effect is likely to originate through 2D filling-in type mechanisms (Komatsu, 2006; Pessoa, Thompson, & Noë, 1998), e.g., lightness induction (Cornelissen et al., 2006; Pereverzeva & Murray, 2008; Rossi et al., 1996), and Craik–O'Brien effect (Anderson et al., 2009; Boyaci et al., 2007; Perna et al., 2005; Roe et al., 2005; but see McCourt & Foxe, 2006 for an EEG study with White's effect, and Sasaki & Watanabe, 2004 for cortical correlates of neon light spreading, both of which are likely to require 3D interpretation). The stimulus we used in this study, however, is fundamentally different and entails a 3D scene interpretation; specifically, the lightness effect shown in Figure 1 follows estimating depth relations between surfaces, and perceptually grouping those sharing the same depth, both of which presumably require higher level visual processing. Therefore, one could predict that there may be no neural correlates of this lightness effect in early visual areas. Alternatively, even though this lightness effect requires higher level visual processing, we may still observe a correlation between the effect and cortical activity. Here we sought to find whether there is evidence for context-dependent lightness processing in early visual areas for scenes that involve occlusion and perceptual grouping. 

Behavioral measure of the lightness effect in 3D stereo

Methods

Participants

Three observers, including the two authors FF and HB, participated in the experiment. All were experienced psychophysical observers and aware of the purpose of the experiment. All observers were right-handed, reported normal or corrected-to-normal vision, and had no known visual and neurological disorders. Informed written consent was obtained in accordance with procedures and protocols approved by the Human Subjects Review Committee of the University of Minnesota. 

Display system

Three-dimensional stimuli were presented to the participants using a computer controlled Wheatstone stereoscope in an otherwise dark room. The stereoscope was composed of a single Sony GDMC520 CRT monitor and two pairs of mirrors placed in front of the observer's eyes. The mirrors reflected two slightly different images of the stimulus generated on either half of the monitor to create a stereoscopic 3D perception. Observers were seated 63.5 cm away from the monitor and their heads were stabilized using a head and chin rest. The system was controlled by an Apple Macintosh computer (Power Mac G5). Color look-up tables were prepared after direct measurements of the luminance values of grayscale patches with a colorimeter (colorCAL, Cambridge Research Systems). The tables were then used to correct for monitor response nonlinearities. The maximum luminance achievable was 97 candelas per square meter (cd/m ²; Commission Internationale de l'Eclairage [CIE]: x = 0.312, y = 0.339; the chromaticity was held constant across all the gray levels used in the experiment. This was ensured by an initial tuning of the monitor). Experimental software was written by us in the Java programming language. 

Stimuli

The stimuli were 3D images presented stereoscopically ( Figure 2a). The occluded rectangular surface subtended 12 by 6.25 degrees of visual angle, and the bars were 0.625 by 7.5 degrees (width × height). The bars were placed 1.9 degrees laterally from the central border. The images were adjusted such that the rectangular surface had zero disparity and the occluding bars had a near disparity of 0.57 degrees. A dashed-line frame with zero disparity surrounded the image pairs to help the observers maintain fusion. The luminance pattern along a horizontal cross-section, excluding the bars, is given by  

$L=\{\begin{array}{cc}{L}_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}& \mathrm{a}\mathrm{t}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{f}\mathrm{l}\mathrm{a}\mathrm{n}\mathrm{k}\mathrm{s},\\ L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}\pm L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}\times C& \mathrm{a}\mathrm{t}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r},\end{array}$

(1)

where C is the border contrast, and L _mean was equal to half the maximum luminance achievable on the display system. Eleven border contrast levels were tested (from 0 through 1 with steps of 0.1). The luminance of the flanks remained constant across all contrast levels. To generate the bars, we first assigned a random gray level to each pixel (0–255) and then convolved the resulting image with a 6 by 6 uniform kernel, to effectively blur the image. 

Figure 2

 

Behavioral measure of the lightness effect. (a) Stereo image pairs. Left pair for crossed fusion, right one for uncrossed fusion. (b) Task. Observers adjusted the intensity of a patch superimposed on one flank to match the intensity of the opposite flank. Eleven border contrast levels were tested. (c) Individual observer results. Magnitude of the lightness effect (computed as described in the text) first increases with the border contrast but then starts to decrease after mid-levels and nearly diminishes at higher levels. This is consistent with the subjective observation in Figure 1. (Error bars indicate 95% confidence intervals.)

Experimental procedure and the task

The participant's task was to adjust the luminance of a small gray square patch (1° × 1°) on one flank of the stimulus to match the luminance of the opposite flank ( Figure 2b). Participants completed 5 trials for each border contrast level; trial order was randomized. There was no time restriction. The magnitude of the lightness effect ( LE) was then computed as the contrast between the matching patch and its background (sign-corrected for the polarity of the border contrast) as follows:  

$LE=\frac{L_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}-L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}}{2L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}}}\mathrm{sgn}\left(C\right),$

(2)

where sgn(·) is the signum function. 

Results

In this behavioral experiment, we systematically investigated the magnitude of the lightness effect as a function of the contrast of the central border ( Figure 2). Individual observer results are shown in Figure 2c. Consistent with our subjective observation in Figure 1a, the lightness effect first gets stronger with the contrast of the central border, and then starts to weaken and almost completely vanishes at very high contrast levels. As suggested previously, we conjecture that a breakdown of the perceptual grouping could be responsible for this pattern: At low contrast levels, the probability that a flank and the central region closest to it are generated by the same process is high and therefore the visual system may perceptually combine them to form a single uniform surface, which could lead to the lightness effect. However, as the difference between the flank and the central part increases, the probability that they belong to the same uniform surface decreases and the visual system may no longer combine them into a single surface, consequently the lightness effect vanishes. 

Note that, because of a possible simultaneous contrast effect between the matching patch and the background flank, the method used in this experiment might have underestimated the lightness effect. However, this would not change the general pattern of results. 

Behavioral measure of the dynamic lightness effect in 2D

In the fMRI experiment described below, we used a modified version of the lightness stimulus. First, in the fMRI experiment the stimuli were presented in 2D. Second, we used a design in which the stimulus temporally varied while the observers maintained fixation at the center of the stimulus. This was done to ensure detecting a strong fMRI signal as explained below. Because of these manipulations, to establish a better connection between the behavioral effect and the fMRI results, we performed a second behavioral experiment in 2D using dynamic presentation. 

Methods

Participants

Two observers, HB and KD, who participated in the first experiment also participated in this experiment. Informed written consent was obtained from the participants in accordance with procedures and protocols approved by the Human Subjects Review Committee of the University of Minnesota. 

Display system

The same display system as in the first experiment was used, except that the stereo mirror system was removed and the observers viewed the screen binocularly without any disparity cues to depth in the image. Only a single image was presented at the center of the screen. 

Stimuli

Two different types of stimuli were used ( Figure 3a). The “Illusory” stimulus was generated using the same equation as in the first behavioral experiment ( Equation 1). The “Real” stimulus was similar to the illusory one, except that its flanks actually differed in luminance: the luminance of each flank was equal to the luminance of the central rectangle closer to it. The occluded rectangular surface subtended 24 degrees of visual angle. The bars were 1.25 by 15 degrees and were placed 3.8 degrees laterally from the central border. A fixation mark was placed in the middle of the central contrast border. Two thin wire frames (1.6° × 1.6°) were superimposed at the center of each flank to aid participants' judgments. 

Figure 3

 

Behavioral measure of dynamic lightness effect in 2D. (a) Experimental design. In a 2IFC experiment, we compared context-dependent lightness changes to real luminance variations. Participants' task was to indicate the interval in which the flanks appeared to vary most during the dynamic display. (b) Results. Contrast of the perceptually equivalent real stimulus was estimated from observers' data and defined as the magnitude of the lightness effect. (Error bars indicate 95% confidence interval.)

Experimental procedure and the task

An adaptive-staircase procedure (1-up, 1-down) was used in a two-interval forced-choice (2IFC) experiment. In each interval, a square-wave-modulated flickering stimulus was presented. The flickering rate was 0.16 Hz, and a total of 4 frames were presented in each interval (3-s frame rate). During the flicker, only the central part reversed polarity in the illusory stimulus. In the real stimulus, both the flanks and the central part altered together in luminance. In the first interval, either an illusory or real stimulus was shown dynamically for 12 s with an interstimulus interval (ISI) of 2 s. In the second interval, the other kind of stimulus was presented. Observers were asked to maintain fixation and indicate the interval in which the flanks, within the wire frames, appeared to change most in luminance during the flicker. Border contrast of the illusory stimulus was the independent variable. Data were collected for 5 different border contrasts in two experimental sessions. Two interleaved staircases were used with a fixed number of trials (20). The contrast of the subjectively equivalent real stimulus was estimated with a maximum likelihood technique (Wichmann & Hill, 2001) and defined as the magnitude of the lightness effect. 

Results

The results are shown in Figure 3b. Consistent with the first experiment, the lightness effect first increases, then decays with border contrast. However, in the dynamic 2D version, the effect seems to start decaying at lower contrast levels compared to the static 3D version. This discrepancy could be due to the methodological differences (method of adjustment versus two-interval forced-choice) or it could be due to perceptual and neuronal differences between processing 3D and 2D stimuli, or between static and dynamic stimuli, or both. 

FMRI measure of the lightness effect

Methods

Participants

The same three observers who participated in the first behavioral experiment also participated in the fMRI experiment. Informed written consent was obtained from the participants in accordance with procedures and protocols approved by the Human Subjects Review Committee of the University of Minnesota. 

MR data acquisition

Scanning was performed on a 3 Tesla scanner (Siemens Trio) with an eight-channel phase-array head coil. BOLD signal were measured with an echo-planar imaging (EPI) sequence (echo time [TE], 30 ms; repetition time [TR], 2000 ms; field of view [FOV], 220 mm; matrix size, 64 × 64; flip angle, 75 degrees; slice thickness, 3 mm; number of slices, 28; slice orientation, axial). The first four volumes were discarded to allow for magnetization equilibrium. A high-resolution T1-weighted anatomical-volume scan (3D MPRAGE; 1 × 1 × 1 mm ³ resolution) was acquired for each participant in the same session before the functional scans. Each observer participated in two fMRI sessions on different days. One session was conducted to define retinotopic areas following the methods developed by Engel, Glover, and Wandell (1997) and Sereno et al. (1995). The experimental session included a structural scan, one region of interest (ROI) localization scan, and four scans for the measurement of cortical response to the lightness effect. 

MR display system

In the scanner, the stimuli were back-projected by a video projector (Sanyo Pro-Xtrax PLC-XP41, refresh rate 60 Hz) onto a translucent screen placed inside the scanner bore. Observers viewed the stimuli through an angled mirror located above their eyes. Color look-up tables were prepared for precise stimulus presentation after direct measurements of the luminance values of grayscale patches with a Minolta CS-100 Chromameter. The maximum luminance achievable on the translucent screen was 2917 cd/m ² (CIE: x = 0.314, y = 0.392). 

Stimuli

The stimuli used in the lightness experiment and their dimensions are shown in Figures 4a and 4b. The presentation was dynamic and in 2D. To test context-dependent lightness responses, we used two stimulus conditions that we refer to as “Illusory” and “Control” conditions ( Figure 4). The two stimuli had identical luminance profiles along a horizontal cross-section passing through the fixation point ( Equation 1). The only difference between the illusory and control stimuli was the addition of two horizontal bands with mean luminance, L _mean, above and below the rectangular surface. This small difference leads to near vanishing of the lightness effect in the control condition. In the experiment, the stimuli were presented both statically and dynamically as described below. During the dynamic display, the central border reversed its contrast polarity. This led to the perception of lightness changes at the flanks in the illusory condition but not in the control condition. By directly measuring the luminances of flanks in the illusory stimulus, we made certain that they did not actually vary during the dynamic displaying. Indeed, by covering the central region with an occluder, we could subjectively observe that the flanks did not vary in actual luminance in the illusory and control conditions. Based on the results of the second behavioral experiment, and our subjective experience under the experimental conditions (i.e., inside the scanner bore), we set the contrast of the border to 0.25 for both the illusory and control conditions, as this led to the strongest lightness effect with the dynamic presentation. L _mean was set to half of the maximum luminance of the display. 

Figure 4

 

FMRI measure of the lightness effect. (a) Stimuli used in the experiment. The stimuli in the illusory and control conditions were identical except that in the control condition we added gray bands with the mean luminance below and above the occluded rectangular surface. While a strong lightness effect is observed in the illusory condition, the effect is nearly abolished in the control condition. (b) ROI definition. The experimental data were analyzed in retinotopically identified voxels that respond to the center of the flanks as marked by the small squares subtitled “ROI”. (c) FMRI design. Both illusory and control stimuli were presented statically and dynamically in a block design. Average of last two time points of static intervals served as a common baseline in a scan. During the scan, observers performed a demanding fixation task to control for attention.

Experimental procedure and the fixation task

In the lightness experiment, observers were presented with the two stimuli shown in Figure 4a. Each stimulus was first presented statically for 18 s to ensure the asymptotic convergence of the fMRI signal to its baseline, followed by a 12-s square-wave-modulated contrast reversal at 0.25 Hz. ( Figure 4c). A dynamic presentation was necessary to obtain a strong fMRI signal. Because dynamically varying the luminance of a surface elicits cortical activity in early visual areas (Haynes et al., 2004), we reasoned that it should be possible to test whether context-dependent lightness processing takes place in these areas by measuring the cortical activity in the illusory condition. Both for the illusory and control stimuli, only the central portion reversed its polarity, leaving the flanks unchanged at all times. Each condition was repeated 4 times in a scan. A scan took 250 s, including a 10-s final blank interval. To control for attention, observers performed a demanding fixation task throughout the entire scan. The task required them to detect a target letter among distractors during rapidly changing presentation of these letters (100 to 150 ms for each letter, target letter: “X” distractors: “Z”, “L”, “P”, and “J”). Observers' overall successful detection rate was 61.46% (SEM 1.51) and average reaction time was 606.52 ms (SEM 8.78). 

ROI localization

We identified regions of interest (ROIs) in a separate scan using square-wave-modulated contrast-reversing (8 Hz) black-and-white checks covering an area of 2 by 2 degrees in observers' periphery on a gray background with the same mean luminance as the stimuli used in the lightness experiment. The ROIs were located at 6 degrees to the left and to the right of the fixation mark. The locations were chosen to maximize the distance from any edges in the image (see Figure 4b). ROI localizers were presented for 12 s followed by a 12-s blank, repeated for 7 times. One localizer scan was performed on each observer. 

Data processing and analysis

Using BrainVoyager QX (Brain Innovation, Maastricht, The Netherlands), we first preprocessed the functional images to correct for 3D head motion and to remove linear trend and to filter out low temporal variations (below 0.015 Hz; Smith et al., 1999). The structural images acquired in the retinotopic mapping scans were inflated for visualization with BrainVoyager QX. For each observer, the functional images from all subsequent scans were spatially transformed and aligned with the structural images obtained in the retinotopic mapping scan. Boundaries between retinotopic areas were drawn manually with BrainVoyager QX after visual inspection of the cross-correlation maps of the BOLD response and the rotating wedges and of the BOLD response and the expanding annuli (Engel et al., 1997; Sereno et al., 1995). A general linear model (GLM) procedure was utilized for ROI analysis (p(corr.) < 10⁻⁴) with BrainVoyager QX and visualized on inflated cortices for each individual observer. Time courses of fMRI signal in the experimental scans from each ROI were extracted and further analyzed by our own numerical routines in Java platform. This analysis included an event-related averaging of each stimulus condition (with the average of the last two measurements of all static conditions—both illusory and real—in a scan serving as a single common baseline for all stimulus conditions in the same scan). We further computed the average BOLD response from the third through sixth time points (between 6 and 12 s) after the onset of the counter-phase flickering for each dynamic stimulus condition and applied paired t-tests to determine the statistical significance of differences between conditions averaged across observers. 

Results

Possible outcomes

The dynamic presentation of illusory and control conditions are sketched in Figure 5a. In both conditions, there are no physical changes at the flanks within the ROIs. Therefore, in the null hypothesis we expect to find no increased cortical activity in either condition ( Figure 5b). This outcome would provide no evidence for context-dependent lightness processing. However, as shown in Figure 5c, there are dynamically varying distant image features that could lead to an increased fMRI activity (Cornelissen et al., 2006). Those dynamically varying image features include polarity reversal of the central border, luminance changes in the central rectangles, and the change in contrast between the central rectangles and the occluding bars (Figure 5c). Because these dynamically varying features were identical across the control and illusory conditions, we expect to find nonzero and equal cortical responses in the two conditions. This outcome would not provide any evidence for context-dependent lightness processing, as well. Note that there was one time-varying feature that was not identical across conditions. That was the contrast variation at the border between the central rectangles and the upper and lower regions adjacent to them. In the illusory condition, the central rectangles neighbored the black background whereas in the control condition they neighbored a mean luminance gray band. This could lead to differences in the fMRI signal between the two conditions. However, the effect of these features is likely to be relatively small because of their small size and their large distances to the ROIs. As a final alternative hypothesis, we expect to find a larger fMRI signal in the illusory condition, because the lightnesses of the flanks change dynamically only in that condition. If the signal is indeed larger in the illusory condition, this would constitute a strong evidence for context-dependent lightness processing in the cortical areas under investigation. 

Figure 5

 

Possible outcomes of the fMRI experiment. (a) Dynamical presentation of the stimulus. (b) Because there is no physical change at the flanks, in the null hypothesis we expect no increase in fMRI signal to dynamically presented stimuli. (c) Alternatively, because of distant physical changes we would expect an increased fMRI activity. If these temporal physical variations were the sole factors driving the activity in the cortex, we would expect the fMRI signal to be identical across the illusory and control conditions. (d) Lightness changes at the flanks only in the illusory condition, not in the control condition. If the cortical activity correlates with perceived lightness rather than physical luminance, we would expect to find a larger fMRI activity in the illusory condition.

The results of the fMRI experiment are shown in Figure 6. Time courses of fMRI signal averaged across scans and observers from V1, V2, and V3 are plotted in Figure 6a. Figure 6b shows a summary of results, where we averaged the signal for each condition from 3rd to 6th time points (6th to 12th s). The averaged signal was significantly larger for the illusory condition in all areas we investigated ( p < 0.001). Therefore, we cannot reject the hypothesis in which the cortical responses correlate with context-dependent lightness changes, not with luminance. 

Figure 6

 

FMRI results. (a) Time course of activity in V1, V2, and V3 averaged across observers. (b) Averaged response from 3rd to 6th time point. There is a significant difference between the conditions in all three cortical areas (*** p < 0.001; error bars are one standard error of the mean).

Note that there is a small but nonzero signal in the control condition. This may be because of the residual lightness effect in the control condition (we have not repeated the second behavioral experiment with the control condition to quantitatively determine the magnitude of the residual lightness effect, but observers subjectively reported a small lightness effect in this condition), or it may stem from neural responses to distant dynamic image features (Cornelissen et al., 2006). 

Discussions

Using a simple stimulus, we show that perceptual grouping through amodal completion can strongly affect lightness. Consistent with our subjective experience in Figure 1, results of two behavioral experiments show that this lightness effect depends on the context in a complex way: the effect first gets stronger as the contrast of the central border increases (a distant image feature), but then starts getting weaker and finally vanishes ( Figures 2c and 3b). It is not too difficult to offer an intuitive explanation for this pattern of results. When the luminance of a flank and the nearest central rectangle have sufficiently similar luminances, the probability that they are generated by the same process, i.e., same light–surface interaction, is high. Therefore, the visual system perceptually combines these distant regions into a single uniform surface. Once combined into uniform surfaces, the lightnesses of the central rectangles influence the lightnesses of the flanks that are nearer to them. It is as though the lightnesses of the central rectangles spread to the flanks (a “lightness assimilation”), leading to the lightness effect we observe in Figure 1a. (The visual system should have no difficulty to estimate that the two central rectangles have different lightnesses because of the clearly visible contrast border between them.) However, as the luminance differences between the center rectangles and the flanks increase, the probability that the same process generated them decreases. In that case, the visual system no longer combines those distant parts into uniform surfaces, and the lightness effect diminishes. 

Next, in an fMRI experiment, using a 2D and dynamic version of the stimulus, and a minimally modified version in which the lightness effect nearly vanishes, we investigated cortical correlates of context-dependent lightness perception. Consistent with recent literature (Anderson et al., 2009; Boyaci et al., 2007; McCourt & Foxe, 2006; Pereverzeva & Murray, 2008; Roe et al., 2005; Rossi et al., 1996), our results suggest that the cortical responses in V1, V2, and V3 correlate with context-dependent lightness changes (but see Cornelissen et al., 2006; Perna et al., 2005 for contrary findings). 

One of the earliest results showing context-dependent lightness responses in striate cortex was by Rossi et al. (1996), where neuronal activity was recorded from anesthetized cats using the lightness induction stimulus.¹ Recently, Pereverzeva and Murray (2008) convincingly demonstrated that the fMRI activity in human early visual areas correlated with context-dependent lightness perception. In their study, Pereverzeva and Murray (2008) manipulated the perceptual strength of the lightness induction effect by using different levels of luminance for the target region and analyzing the data in multiple ROIs. Results of Pereverzeva and Murray's (2008) study showed a strong cortical response to dynamic changes in distant features that could be best explained by a model in which neurons respond to context-dependent lightness changes within their receptive fields. Meanwhile, Roe et al. (2005) examined context-dependent lightness responses in V1 and V2 of anesthetized monkeys. Roe et al. (2005) used a rectangular version of the well-known Craik–O'Brien stimulus (Cornsweet, 1970; Land & McCann, 1971; O'Brien, 1958) and showed that V2 neurons respond to context-dependent lightness variations, as well as luminance variations. However, in V1 their results were mixed. Later using the Craik–O'Brien stimulus with a different configuration Boyaci et al. (2007) found evidence of cortical processing of lightness in human early visual areas, V1, V2, and V3. More recently using Craik–O'Brien type stimuli Anderson et al. (2009) found further evidence supporting that the visual areas as early as LGN contribute to lightness processing. 

Even though the approach was similar in spirit to ours, the stimuli used in previous studies almost always relied on 2D filling-in mechanisms to generate context-dependent lightness effects, such as lightness induction and Craik–O'Brien effect (except McCourt & Foxe, 2006, where White's effect was used in an EEG experiment). The stimulus we used here, however, entails 3D scene interpretation and perceptual grouping, therefore likely to require higher level visual processing. Here we were able to show that cortical activity of early visual areas correlated with context-dependent lightness without attention, even in situations that required 3D scene perception and perceptual grouping. Therefore, the results found here carry further significance for understanding the mechanisms of lightness perception in the human visual system. 

However, many questions still remain open. First, we cannot be sure whether the cortical activity reflects a feed-forward, feedback, or lateral interaction mechanism. Although we controlled for attention, it is not possible to rule out feedback mechanisms under conditions without attention or conscious perception (Fang & He, 2005; Moore & Egeth, 1997). Second, we still do not know the exact role of early areas in lightness perception. Does this activity constitute a required step toward lightness estimation of surfaces, or does it ensue the lightness estimate computed by higher level areas? If the latter is true, why does the visual system care to override the activity to proximal stimulus of lower level areas by feedback from higher levels? Some of these questions can be answered by converging methods of cognitive neuroscience, for example, TMS and fMRI, or by detailed analysis of the timing of activity in the cortex (Boyaci, Fang, Murray, Albanese, & Kersten, 2008; McCourt & Foxe, 2006). 

Acknowledgments

This research was funded by Grant R01 EY015261 from the National Institute of Health. The 3T scanner at the University of Minnesota, Center for Magnetic Resonance Research, was supported by National Center for Research Resources P41 008079 and P30 NS057091 and by the MIND Institute. Huseyin Boyaci was also supported by the Grant Marie Curie IRG 239467 from the European Commission, by the Grant 108K398 from the Scientific and Technological Research Council of Turkey (TUBITAK), and by the Grant GEBIP from the Turkish Academy of Sciences (TUBA). Fang Fang was also supported by the National Natural Science Foundation of China (Project 30925014). We are grateful to David Brainard for his helpful comments and suggestions. 

Commercial relationships: none. 

Corresponding author: Huseyin Boyaci. 

Email: [e-mail]. 

Address: Department of Psychology and National Research Center for Magnetic Resonance (UMRAM), Bilkent University, Ankara 06800, Turkey. 

Footnote

Footnotes

1  Rossi et al. (1996) emphasizes “brightness induction.” Nevertheless, the stimulus used was “lightness induction.” As a matter of fact, the term “brightness” is often used interchangeably with “lightness” in literature. See Gilchrist (2007) regarding this confusion of terminology.