Brightness and transparency in the early visual cortex

Abstract

Abstract

Abstract:

Abstract  Several psychophysical studies have shown that transparency can have drastic effects on brightness and lightness. However, the neural processes generating these effects have remained unresolved. Several lines of evidence suggest that the early visual cortex is important for brightness perception. While single cell recordings suggest that surface brightness is represented in the primary visual cortex, the results of functional magnetic resonance imaging (fMRI) studies have been discrepant. In addition, the location of the neural representation of transparency is not yet known. We investigated whether the fMRI responses in areas V1, V2, and V3 correlate with brightness and transparency. To dissociate the blood oxygen level–dependent (BOLD) response to brightness from the response to local border contrast and mean luminance, we used variants of White's brightness illusion, both opaque and transparent, in which luminance increments and decrements cancel each other out. The stimuli consisted of a target surface and a surround. The surround luminance was always sinusoidally modulated at 0.5 Hz to induce brightness modulation to the target. The target luminance was constant or modulated in counterphase to null brightness modulation. The mean signal changes were calculated from the voxels in V1, V2, and V3 corresponding to the retinotopic location of the target surface. The BOLD responses were significantly stronger for modulating brightness than for stimuli with constant brightness. In addition, the responses were stronger for transparent than for opaque stimuli, but there was more individual variation. No interaction between brightness and transparency was found. The results show that the early visual areas V1–V3 are sensitive to surface brightness and transparency and suggest that brightness and transparency are represented separately.

Introduction

The physical appearance of a surface is determined by the reflectance of the surface and the amount of light landing on the surface. Correspondingly, the perceived properties of the surface are lightness (perceived reflectance) and brightness (perceived luminance) (reviewed in Gilchrist, 2007). With simple visual stimuli, e.g., a gray square surrounded by another shade of gray, both lightness and brightness refer to the same percept. Brightness and lightness depend on the luminance ratios of the retinal image but are also affected by higher-level interpretations, such as figure-ground segregation (Benary, 1924; White, 1979). If the stimulus contains cues of depth or transparency, lightness and brightness may dissociate because additional cues are available to differentiate between luminance changes due to material or illumination (Adelson, 1993). In order for humans to operate efficiently in the natural environment, our visual system should be able to segregate changes due to illumination (a change in environment) and reflectance (a change in object). However, it is still unresolved how the visual system interprets luminance changes and computes brightness and lightness and at what cortical level these properties are represented. In the current study, the cortical correlates for perceived brightness and transparency were investigated with functional magnetic resonance imaging (fMRI). 

Brightness is mainly determined by the local contrast signal at the border of an achromatic surface (Wallach, 1948). The dependency of brightness on local contrast suggests that brightness might readily be computed from small receptive fields, which points to the importance of early visual areas in brightness perception. Accordingly, single-cell recordings of cats and monkeys, both anaesthetized and awake, have shown that the activity of some cells in V1, V2, and V3 correlate with the brightness (Kinoshita & Komatsu, 2001; MacEvoy, Kim, & Paradiso, 1998; Rossi & Paradiso, 1999; Rossi, Rittenhouse, & Paradiso, 1996) and luminance (Kinoshita & Komatsu, 2001; Peng & Van Essen, 2005) of a surface. In addition, some studies suggest that, while cells in V1 are able to represent real changes in brightness, only cells in V2 and higher areas are able to represent illusory (the Craik-O'Brien-Cornsweet, COC) brightness changes (Hung, Ramsden, Chen, & Roe, 2001), suggesting qualitative differences between V1 and V2 data processing. In agreement with single-cell studies, some fMRI studies have found correlations between activity in the early visual cortices and brightness or lightness (Boyaci, Fang, Murray, & Kersten, 2007, 2010; Haynes, Lotto, & Rees, 2004; Pereverzeva & Murray, 2008; van de Ven, Jans, Goebel, & De Weerd, 2012). However, other fMRI studies have not found evidence of brightness representations in V1 or V2 (Boucard, van Es, Maguire, & Cornelissen, 2005; Cornelissen, Wade, Vladusich, Dougherty, & Wandell, 2006; Perna, Tosetti, Montanaro, & Morrone, 2005). Hence, the fMRI results are discrepant, and the role of early visual areas in processing surface brightness remains largely unknown. 

One major challenge in measuring brightness responses with fMRI is dissociating responses caused by perceptual change in brightness from responses caused by (mean) luminance and local contrast. In previous studies, the blood oxygen level–dependent (BOLD) response to brightness in early visual areas has been, at least partially, coupled with a response to border contrast or mean luminance. The response to luminance may, however, be confounded by the scattering of the light in the eye (Stenbacka & Vanni, 2007). The main motivation for this study was to overcome the coupling between brightness and other parameters. We designed a setup in which the mean luminance of the display as well as the time-averaged local luminance contrast of the target were kept constant and identical in the conditions we compare. Hence, the only difference between the stimuli in the compared conditions is in the brightness. If the responses to these stimuli differ, this would provide straightforward evidence for brightness encoding in early visual areas. We used “second order” brightness stimuli (Figure 1), in which the target surface is always surrounded by both luminance increments and decrements that cancel each other out, and thus the brightness is varied independently of average border contrast and mean luminance. 

Figure 1

 

Stimuli. A) White, B) Control, C) transparent White, D) Cross. In A, B, and D, the local border contrasts of the left and right side targets are all identical: a gray target surrounded by an equal amount of white and black borders. In C, to induce transparent appearance, rectangular shapes were placed above and below the target square, and the contrast of the upper and lower border of the square targets is reduced compared to the targets in A, B, and D.

We used both static and temporally modulating (luminance) target stimuli. In half of the conditions, brightness was modulated (either illusory or real change), and in the other half, it was perceived to be constant (either nulled illusion or constant). In addition, we wanted to test the effect of transparency on brightness responses and their interaction because high-level factors have been shown to contribute to surface perception (Adelson, 1993; Anderson & Winawer, 2005; Benary, 1924; White, 1979). One possible neural mechanism for segregating transparent surfaces is border ownership assignment (Zhou, Friedman, & von der Heydt, 2000). Both in single-cell recording (Qiu & von der Heydt, 2007) and in fMRI studies (Fang, Boyaci, & Kersten, 2009), the responses from V2 have been shown to modulate according to border ownership. Thus, we expected responses to opaque and transparent surfaces to diverge more in V2 than in V1. 

The mean signal changes in V1, V2, and V3 were compared across conditions. Stronger BOLD responses were found for brightness modulation than for constant brightness in all areas. Similarly, transparent stimuli evoked larger BOLD responses than opaque stimuli. However, no interaction between brightness and transparency or systematic difference between the responses in V1–V3 was found. The results suggest that brightness and transparency are represented separately in the early visual cortex. 

Methods

Subjects

Eight subjects with normal or corrected-to-normal vision participated in the study. The experiments were approved by the ethics committee of the Hospital District of Helsinki and Uusimaa and were conducted according to the declaration of Helsinki. Subjects gave written informed consent before the measurements. Each subject participated in three separate measurement sessions (retinotopic mapping, fMRI measurement, and behavioral experiment) on three different days. 

Stimuli

The stimuli were created and presented using Presentation^TM software (Neurobehavioral Systems, www.neurobs.com). In the fMRI experiment, the stimuli were displayed with a calibrated Christie X3 (Christie Digital Systems, www.christiedigital.com) data projector on a semitransparent screen. The subjects viewed the screen through a mirror from a distance of 34 cm. In the behavioral tests, a calibrated Samsung LCD display was used. Both displays had similar luminance ranges, and the mean luminance of the displays was 60 cd/m². 

Every stimulus contained two gray target squares (3° × 3°; dotted lines in Figure 1B) presented at 4.8° (from fixation to square center) eccentricity, one in the left upper and the other in the right lower quadrant (Figure 1). The two squares were embedded in four different surrounds (9° × 9°; dashed lines in Figure 1B) that changed the perceived brightness and/or transparency of the target squares. We call the stimuli (a) White (or opaque White), (b) Control, (c) transparent White, and (d) Cross. White stimulus is a variant of White's illusion (White, 1979), and the two target squares appear different in brightness (Figure 1A). In Control stimulus, the squares have equal local contrasts in comparison to White stimulus, but there is no brightness illusion (Figure 1B). Transparent White is another variant of White's illusion, and in addition to the brightness illusion, the targets appear transparent (Figure 1C). In Cross stimulus, the targets appear transparent, but there is no brightness difference between the squares (Figure 1D). 

In every condition, the contrast of the surround was sinusoidally modulated at 0.5 Hz with peak-to-peak variation reaching 50% Michelson contrast. The luminance of the target squares was either static or dynamic, and thus there were eight stimulus conditions in total (Figure 2). Due to the surround modulation, the apparent brightness of the static target squares changed (illusory brightness) in two conditions (opaque White and transparent White; Figure 1A and 1C; Figure 2, conditions 1 and 5) but remained constant in two conditions (Control and Cross; Figure 1B and 1D; Figure 2, conditions 3 and 7). In the dynamic conditions, the apparent brightness change was nulled by modulating the luminance of the target (10% Michelson contrast) in counterphase to the brightness modulation (Figure 2, conditions 2 and 6). Hence, the brightness change in the opaque White and transparent White conditions were “cancelled out,” and the squares appeared constant in brightness, but in the Control and Cross conditions, the brightness changed (real luminance change, Figure 2, conditions 4 and 8). The eight different conditions are demonstrated in Movies 1 through 8. 

Figure 2

 

Experimental conditions. Surround configuration (Stimulus: White, Control, transparent White, Cross) and target luminance (static/dynamic) were varied to create an opaque or transparent appearance and modulating or constant brightness. In the analysis, the brightness change conditions were compared to no change conditions, static target luminance was compared to dynamic target, and conditions with transparent appearance were compared to stimuli with opaque appearance.

The luminance and contrast variations of the targets and surrounds are shown in Figure 3B and 3C, respectively. The local contrasts (inside the dashed circles in Figure 3A) of the squares were identical in the White, Control, and Cross stimuli (Figure 3C). In the transparent White stimuli, the luminance/contrast of the two small rectangles above and below the squares was reduced in order to create the transparent interpretation, and thus the local contrast in transparent White, 0%–19%, was reduced in comparison to the other three, 0%–24%, stimuli (Figure 3C). In addition, the stimulus contained an additional border between the transparent rectangle and the surround. The 50% Michelson contrast variation of the surround corresponded to rms contrasts (standard deviation of luminance values divided by the mean luminance) of 0.26, 0.26, 0.24, and 0.18 for White, Control, transparent White, and Cross stimuli, respectively. The Michelson contrasts of the luminance edges in the surround were identical in the White, Control, and Transparent White stimuli (Figure 3C). The surround of the Cross stimuli contained luminance edges only between the modulated areas and the background, and so the Michelson contrast of the luminance edges was reduced in comparison to other stimuli (Figure 3C). The luminance of the static squares and the mean luminance of the display background was 60 cd/m². A black fixation cross in the center of the screen on a white background (1.3° × 1.3°) was always present. 

Figure 3

 

Luminance and local contrast of the target and surrounds. A) The targets (dashed circles on the top row) were identical in the White, Control, and Cross stimuli but slightly modified in the transparent White stimulus. B) In all conditions, the luminance of the surround was sinusoidally modulated. The luminance of the target was constant or modulated. C) The Michelson contrast of the surround varied from 0% to 50%. The target contrast (average of increment and decrement borders) varied from 0% to 24% in the White, Control, and Cross stimuli, and from 0% to 19% in the transparent White stimulus. Static and dynamic targets had almost equal average contrasts.

Two functional localizer stimuli assisted in isolating the regions of interest in the cortex. The center localizer was a 2.25° × 2.25° checkerboard pattern, centered at target and contrast reversing at 7 Hz, and the border localizer was a corresponding 3.75° × 3.75° checkerboard with the 2.25° × 2.25° center checks removed (Figure 4A). 

Figure 4

 

Functional localizers. A) Two checkerboard localizers were used, one for the target and one for the target border. The localizers were flickered at 7 Hz. The dotted red line shows the size of the target. B) fMRI BOLD responses to the target localizer clearly reveal separable V1, V2, and V3 voxels. Examples of the activated voxels from two subjects. Data from the significantly activated voxels in areas V1–V3 were selected for further analysis.

Behavioral tests

Two behavioral experiments were conducted in order to confirm that the subjects saw the changes in brightness, that the strength of the brightness illusion was similar in transparent and opaque stimuli, and that the counterphase nulling was effective. The viewing distance was approximately 50 cm (no chin rest was used), and the subjects were allowed to move their eyes. 

In the first behavioral experiment, the stimuli (Figure 1) were presented with constant 50% Michelson contrast, i.e., without modulation. The subjects' task was to adjust the luminance of the targets by pressing two keys on a keyboard so that the two targets appeared as similar as possible. In other words, the task was to null or cancel out the effect of the surround. The keyboard presses modified the relative contrast between the targets, i.e., the luminance of one target was increased, and the luminance of the other target was decreased simultaneously. At the beginning of each trial, the contrast of the target was randomly set in the range from −0.2 to 0.2. The subject then adjusted the contrast of the target until she/he was content with the adjustment. The duration of adjustment was not limited. The next trial began by pressing a third key on the keyboard. For each stimulus, the final adjusted contrast was calculated as an average of four repetitions. In total, each subject made 16 adjustments (4 × 4 stimuli) in random order. If the surround does not have an effect on the brightness of the targets, the subjects would be expected to adjust the contrast to zero. However, if brightness is modulated by the surround of the target, the subjects should adjust the contrast in counterphase to brightness modulation to null the effect. Prior to the experiment, subjects practiced the adjustment task with the target embedded in a uniform surround. 

In the second behavioral experiment, the four stimuli (Figure 1) were presented with modulating surround contrast. The subjects' task was to rate the amount of brightness modulation of the targets on a three-step scale: 1 = no modulation, 2 = weak modulation, 3 = strong modulation. The contrast of the target was either constant (0% modulation) or modulated in counterphase to brightness induction (0.1%) or in phase to brightness induction (−0.1%). The first two conditions were identical to the conditions in the fMRI experiment. The last condition provided an example of clear brightness modulation. Each target modulation was repeated four times, so in total, the subjects made 48 ratings (3 × 4 × 4 stimulus) in random order. 

fMRI data acquisition

fMRI data was measured with a General Electric 3 T scanner (Signa HDxt^TM) equipped with a 16-channel, receive-only head coil. Each measurement session started with a low-resolution structural MR image with a 3D T1-weighted sequence (128 × 128 acquisition matrix, FOV = 23.5, 1.5 mm slice thickness). Then one functional localizer and six main experiment runs were conducted using a gradient-echo echo planar imaging sequence (TR = 2000 ms, TE = 30 ms, flip angle = 60°, 64 × 64 acquisition matrix, FOV = 20 cm, 3.0 mm slice thickness, resulting in 3.1 × 3.1 × 3.0 mm resolution). 

The fMRI data were analyzed with the SPM8 Matlab^TM toolbox (Penny, Friston, Ashburner, Kiebel, & Nichols, 2006) and Freesurfer (Dale, Fischl, & Sereno, 1999) software packages. In preprocessing, the acquisition order of the functional images and head motion were corrected. No spatial smoothing was applied. 

The mapping of visual areas was done in a separate measurement session on a different day with a multifocal mapping procedure (Vanni, Henriksson, & James, 2005). From the 24 regions in the visual field (three rings, eight polar segments), the representations of the vertical and horizontal meridians were used to identify the retinotopic areas V1, V2, and V3 for each subject. The details of the mapping method are described elsewhere (Henriksson, Karvonen, Salminen-Vaparanta, Railo, & Vanni, 2012). 

Procedure

BOLD signal changes from the visual cortex were measured in eight different stimulus conditions with a 2 × 2 × 2 block design. In these eight conditions, (a) the target luminance was static or dynamic, (b) the surface appeared opaque or transparent, and (c) brightness was modulated or was constant (Figure 2). 

Subjects were instructed to fixate on the fixation cross, pay attention to the targets on the upper left and lower right quadrants, and to track possible brightness changes within and between the targets. No responses were collected during the scan, and the eye movements were not monitored. Six of the eight subjects were very experienced in holding a steady fixation; healthy subjects can typically hold a fixation very still with less than 10 arcmin accuracy (Putnam et al., 2005). 

There were seven experimental runs: one localizer, three runs with the transparent stimuli, and three runs with the opaque stimuli. Each run consisted of four repetitions of the four stimulus blocks and a fifth resting block. The order of stimulus blocks was pseudorandom and balanced, but every fifth block was a resting block. The duration of each block was 16 s, and thus the total duration of a run was 4 × 5 × 16 s, i.e., 5 min 20 s. At the beginning of each run, eight volumes were discarded to reach stable magnetization. The order of runs was randomized between the subjects in a balanced manner. 

Data analysis

The average BOLD signal change for the eight different conditions in the cortical areas V1, V2, and V3 was calculated for those voxels that were significantly activated by the checkerboard localizer (region of interest, Figure 4). We analyzed the data using a general linear model, including a high pass filter with a cutoff frequency at 1/128 Hz, and removed temporal autocorrelation with the AR(1) model. The percentage signal change was then calculated by dividing the coefficient of the effect of interest by the coefficient of the mean signal of the voxel. Next, we calculated the average signal change over the voxels within a region of interest and normalized the signal change within each subject and within each visual area by dividing the signal change with the average across conditions. The within-subjects effects (hemisphere, brightness, transparency, target modulation, and visual area) were then tested using repeated measures ANOVA. After the ANOVA, paired sample t tests were conducted within each area to confirm the differences. For this analysis, the average across hemispheres was calculated for each subject (separate stimuli were presented simultaneously to the left and right visual fields in every condition; Figure 1). The eight conditions were grouped three times to two different sets, and the average signal changes were compared to reveal sensitivity to (a) change in brightness, (b) static versus dynamic stimuli, and (c) change in transparency. In the first (Figure 2, brightness change yes vs. no) and second comparisons (Figure 2, target luminance static vs. dynamic), both sets contained stimuli of the same type (White, Control, transparent White, Cross), and in the third comparison, the stimulus sets were of different types (Figure 2, transparency no vs. yes; White, Control vs. transparent White, Cross). The statistical tests were conducted using PASW Statistics 18 (SPSS Inc.). 

Results

Behavioral tests

In the first behavioral experiment, the subjects adjusted the contrast of the targets so that they matched each other in brightness. As expected, the subjects adjusted the target contrast in Control and Cross conditions to near zero as the surrounds do not induce brightness illusions (Figure 5A). With White stimulus, the subjects adjusted the contrast of the targets, on average, to 0.11 and with transparent White to 0.14 (Figure 5A). The brightness illusion in transparent White was slightly stronger than in White, but the difference was not significant, F(1, 7) = 0.59, p = 0.805. The difference of the adjustments between White and Control, F(1, 7) = 14.27, p = 0.007, and between Transparent and Cross, F(1, 7) = 27.68, p = 0.001, were statistically significant. 

Figure 5

 

Results of the behavioral tests. Error bars represent 95% confidence intervals. Eight subjects averaged. A) Adjustment experiment and the strength of brightness illusion. The amount of contrast needed to null the brightness change with different stimuli. The difference between White and Control as well as between transparent White and Cross were significant (p < 0.01). B) Rating experiment with dynamic stimulus identical to the fMRI experiment. The scaled rating of modulation strength (0 = no modulation, 1 = strong modulation) as a function of modulation contrast (−0.1 = modulation in phase to brightness illusion, 0.0 = no modulation, 0.1 = modulation counterphase to brightness illusion). Control and Cross stimuli were perceived as veridical. Brightness illusion (0.0 contrast) with White and transparent stimuli was reduced due to counterphase modulation (0.1 contrast) of the target (*p < 0.05, **p < 0.01).

In the second behavioral experiment, the subjects rated (1 = no modulation, 2 = weak modulation, 3 = strong modulation) the amount of modulation with similar dynamic stimuli that were used in the fMRI experiment. When the target was modulated at −0.1 contrast (i.e., in phase with perceived brightness change), all the subjects' ratings indicated that, as was expected, the brightness was clearly modulated (Figure 5B). The modulation was stronger with stimuli producing brightness illusion because the modulation was in phase with the illusory brightness change. When the target was not modulated (Figure 5B, 0.0 contrast), the subjects' ratings indicated that White and transparent White stimuli were more clearly modulated than Control and Cross stimuli. With a target contrast modulation of 0.1, the pattern of results was reversed, and the modulation of Control and Cross stimuli was rated to be stronger than the modulation with White and transparent White stimuli (Figure 5B). With Control, F(1, 7) = 26.881, p = 0.001, and Cross, F(1, 7) = 5.717, p = 0.048, brightness modulation was clearer with 0.1 than 0.0 contrast, and with White, F(1, 7) = 6.445, p = 0.039, and transparent White, F(1, 7) = 15.909, p = 0.005, the modulation was clearer with 0.0 than 0.1 contrast. In summary, the stimuli evoked percepts that were expected. 

fMRI localizer

Our stimulus activates visually sensitive cortical areas representing both the target (center) and the surrounding patterns. However, we are interested in how the visual cortex representing the perceived target area responds. To isolate the target response, we acquired a functional localizer run with a stimulus confined inside the target location (Figure 4A). The retinotopic areas V1, V2, and V3 were mapped in a separate mapping measurement. From these visual areas, the voxels that were significantly activated in the functional localizer run were selected for further analysis (threshold T = 4.8; minimum cluster size three voxels; Figure 4B). Next, the mean signal changes across the regions of interest were calculated for the different conditions. 

Brightness and transparency

Repeated measures ANOVA was conducted with the following within-subjects effects: hemisphere (left vs. right), target modulation (dynamic vs. static), transparency (transparent vs. opaque), brightness (modulating vs. constant), and visual area (V1 vs. V2 vs. V3). The effect of the hemisphere, F(1, 7) = 2.208, p = 0.181, was not significant. The effect of modulating the target luminance was not significant, F(1, 7) = 0.005, p = 0.946. However, both the effect of modulating brightness, F(1,7) = 7.262, p = 0.031, and perceived transparency, F(1, 7) = 6.170, p = 0.042, were statistically significant, but the interaction between transparency and brightness was not, F(1, 7) = 1.119, p = 0.325. 

Due to normalization of the data within each area, the effect of the area could not be significant. Instead, the possible effect of an area can be tested by examining the interactions between the area and brightness and transparency, i.e., by investigating whether brightness and transparency differently affect the responses in areas V1–V3. All the tested interactions were nonsignificant: area × brightness, F(2, 14) = 1.649, p = 0.228, area × transparency, F(2, 14) = 2.432, p = 0.124, area × target modulation, F(2, 14) = 2.247, p = 0.142, and area × brightness × transparency, F(2, 14) = 0.405, p = 0.674. 

Visual areas V1–V3

Only the brightness and transparency had significant effect on the measured BOLD responses. Further, because the interactions with area were not significant, brightness and transparency seem to have similar effects in all the studied visual areas. To further confirm this, the average between the hemispheres was calculated and the main effects of interest (brightness, transparency, and target modulation) were tested with paired sample t tests in areas V1–V3. 

Conditions with modulating brightness (Figure 2, conditions 1, 4, 5, and 8) were compared to conditions with constant brightness (conditions 2, 3, 6, and 7). Physically, the stimuli in the two sets were similar and contained all the different surround configurations, but the sets differed in target brightness. Stronger responses were found for modulating brightness than for constant brightness (Figure 6A) in the early visual areas V1, t(7) = 2.428, p = 0.046; V2, t(7) = 2.587, p = 0.036; and V3, t(7) = 2.454, p = 0.044. 

Figure 6

 

fMRI results. Mean signal changes from voxels in the areas V1–V3. Individual data from eight subjects and the average across subjects. Error bars represent 95% confidence intervals of the average. A) Brightness. Conditions with modulating brightness compared to conditions with constant brightness. B) Conditions with dynamic target luminance compared to conditions with a static target. C) Transparency. Conditions with opaque stimuli compared to conditions with transparent stimuli. Note that the scale of the y-axis is larger in C than in A and B (*p < 0.05).

To confirm that modulating the target luminance in the dynamic setup did not have a significant effect, we compared static (Figure 2, conditions 1, 3, 5, and 7) and dynamic (conditions 2, 4, 6, and 8) conditions. The difference in the mean signal change between dynamic and static targets was not significant (Figure 6B) in the early visual areas: V1, t(7) = 0.577, p = 0.582; V2, t(7) = −1.034, p = 0.335; V3, t(7) = −0.0257, p = 0.805). The static and dynamic conditions differ in local contrasts due to dynamic modulation of the target only in the latter conditions. The difference in average local contrast between the dynamic and static conditions is relatively small (Figure 3C). This gives a likely explanation for the similar responses found in both conditions. 

The comparison between transparent and opaque surfaces (Figure 2, conditions 1–4 vs. 5–8) revealed stronger responses for transparent than for opaque surfaces (Figure 6C) in V1, t(7) = 2.482, p = 0.042; V2, t(7) = 2.591, p = 0.036; and V3, t(7) = 2.790, p = 0.027. In this comparison, the stimuli were slightly different (Figure 1A and 1B vs. 1C and 1D). The target border contrast was reduced in transparent White stimuli compared to other stimuli (Figure 3C). Further, the surround contrast in Cross stimuli was reduced, and the rms contrast was slightly lower (0.18) than in the other stimuli (0.24–0.26). We replicated the analysis without Cross (and Control) stimuli, and hence similar rms contrasts in both sets, and the difference between the transparent and opaque stimuli was still significant: V1, t(7) = 0.2618, p = 0.034; V2, t(7) = 0.2572, p = 0.037; V3, t(7) = 0.2411, p = 0.047. 

Illusory versus real brightness

In the modulating versus constant brightness change analyses above (Figure 6A), both illusory brightness change (conditions 1 and 5) and real luminance change (conditions 4 and 8) were combined. Comparing real and illusory conditions separately from a constant brightness condition did not reach statistical significance. However, we compared the illusory and real brightness change conditions directly. In this analysis, illusory brightness change (Figure 2, conditions 1 and 5) evoked stronger responses than real brightness change (Figure 2, conditions 4 and 8) in V1, t(7) = 2.536, p = 0.039, and V3, t(7) = 2.563, p = 0.037, but not in V2, t(7) = 1.117, p = 0.301. Larger responses to illusory than to real brightness could, however, result from the strong response to transparent White stimuli and not from the illusory brightness as such. When opaque and transparent stimuli were analyzed separately, there was a trend indicating a difference between the illusory and real brightness in V1 for both stimuli, opaque, t(7) = 1.508, p = 0.175, and transparent, t(7) = 1.595, p = 0.155, and in V2 and V3 for opaque stimuli, V2: opaque, t(7) = 1.902, p = 0.099, and transparent, t(7) = 0.181, p = 0.861, and V3: opaque, t(7) = 1.844, p = 0.108, and transparent, t(7) = 1.238, p = 0.256. This suggests that the difference between real and illusory brightness is not due to transparent White stimulus. 

Additional analyses

All the analyses above were conducted using the voxels that were activated by the target localizer. To further control for the possible confounding effect of the surround, we tested the effect of brightness and transparency with a stricter voxel selection using an additional functional localizer that corresponded to the retinotopic area of the target border (Figure 4A). First, we calculated the BOLD signal changes for both localizers in the voxels activated by the target localizer. Then we included only those voxels that were more strongly activated by the target than the border localizer. Due to the small number of voxels found in each visual area, we averaged the data across the areas V1–V3. The responses were stronger for modulating than for constant brightness, t(7) = 2.507, p = 0.041. The average responses were also stronger for transparent than for opaque targets, but clear individual variation emerged, and hence the overall group difference was not any more statistically significant, t(7) = 0.806, p = 0.447). A closer look at the individual data revealed that for four subjects the transparent target evoked a stronger response than the opaque target: S2, t(12) = 2.421, p = 0.0323; S5, t(28) = 5.214, p = 0.0001; S6, t(8) = 3.350, p = 0.0101; S8, t(28) = 7.504, p = 0.0001. For one subject, the response was stronger for the opaque target: S4, t(17) = −2.486, p = 0.0236 And the for the remaining three subjects, no difference was found: S1, t(13) = 0.0705, p = 0.493; S3, t(6) = −0.355, p = 0.735; S7, t(12) = −1.525, p = 0.153. 

The data were also analyzed individually by calculating voxel-wise SPM t contrast images for the eight different conditions (against the rest block) and the main effects (brightness modulation vs. no modulation, transparent vs. opaque appearance). For every subject, clear and significant activity was found in the visual areas in every stimulus condition. The activations for brightness and transparency, however, varied between individual subjects. The amount of statistically significant clusters was modest although some activity was found in early visual areas in all subjects. 

Discussion

We investigated whether surface brightness and transparency are represented in the early visual areas V1–V3. We found that, when equated in local contrast, mean luminance, and rms contrast, stimuli with apparent brightness modulation evoked stronger responses in V1–V3 than stimuli without apparent brightness modulation. The responses were also stronger for transparent than for opaque surfaces. Neither systematic differences between the visual areas nor interaction between brightness and transparency were found. The results show that early visual areas are sensitive to surface brightness and opacity and suggest that early visual areas contribute equally to decomposing visual images into separate representations of brightness and transparency. 

Decoupling brightness and contrast

A major challenge in brightness studies is dissociating BOLD responses to perceptual changes in brightness from BOLD responses to luminance and contrast. Different studies have approached the problem using various solutions. Perna et al. (2005) compared responses to the COC illusion to responses to a stimulus comprising identical local energy but no brightness illusion. They did not find brightness-related responses in early visual areas. Boucard et al. (2005) and Cornelissen et al. (2006) compared responses to real luminance modulation with responses to induced brightness due to modulation of surround luminance. They found responses to luminance and edge contrast but not for brightness as such. Also, Haynes et al. (2004) found clear responses to surface luminance from the early visual areas. Boyaci et al. (2007, 2010) found larger BOLD responses to the COC illusion and real luminance change stimuli than to control stimuli without perceived change in surface brightness. However, their control stimuli had different mean luminance and contrast than their lightness/brightness stimuli. van de Ven et al. (2012) compared the effect of surround luminance modulation on the brightness of gray and black disks. Brightness was modulated in the gray disk but not in the black disk. Correspondingly, the BOLD responses were larger to the former than to the latter. However, the stimuli in these two conditions also had different mean luminance and local contrast. Pereverzeva and Murray (2008) decoupled brightness from local contrast by varying surface luminance: With the lowest luminance level (black surface), the contrast was maximal but brightness induction minimal, and with the highest luminance (mid-gray), the contrast was minimal but the brightness induction strongest. They found that the BOLD responses correlated with brightness and not with border contrast. In fact, they found the strongest response with a 0% average contrast border (i.e., when surround modulates around the target luminance). However, in their setup, the mean luminance of the display also varied as a function of brightness, and in a control experiment, they also reported minor luminance response. 

In the present study, we used second-order stimuli that contained both increment and decrement of luminance that cancelled each other out, thus keeping the local contrast and mean luminance constant. In agreement with previous fMRI studies on surface perception (Boyaci et al., 2007, 2010; Haynes et al., 2004; Pereverzeva & Murray, 2008; van de Ven et al., 2012), we found a brightness response from the early visual areas. Some single-unit studies suggest that V1 responds mainly to real brightness change whereas V2 could also represent illusory brightness (Hung et al., 2001). Our fMRI measurements did not replicate this result because we found a response to illusory brightness already in V1. Further, we found the opposite results: stronger responses for illusory than real brightness in V1 and V3 but not in V2. These results suggest that some qualitative differences might exist between brightness processing in V1, V2, and V3. 

Brightness

The perceptual differences in target brightness can be explained by selective filtering/normalization (Blakeslee, Pasieka, & McCourt, 2005) or selective integration of the contrast according to figure-ground segregation (Ross & Pessoa, 2000) or layered image representations (Anderson, 2003). The BOLD magnitude differences between different conditions likely emerge from the corresponding neural mechanisms. Our results suggest that the selective integration of contrast is accomplished already in V1. It is also possible that adaptation contributes to the measured responses. The neural mechanism encoding surface brightness might be more strongly adapted in the constant than in the modulating brightness condition. The signal for surface properties, such as brightness and transparency, might also originate from higher-level visual areas, as suggested by Perna et al. (2005) and Lin and He (2012), and feed back to early visual areas (Lin & He, 2012). Nevertheless, according to our results, V1 activity correlates with brightness and transparency, suggesting that the neurons in V1 explicitly represent or multiplex information (Rossi et al., 1996) concerning surface properties. This interpretation is not dependent on whether or not any modulating signals originate from higher-level areas or the neurons surrounding the target. 

In the single-neuron studies (Kinoshita & Komatsu, 2001; Rossi et al., 1996), the measured responses have been quite straightforwardly interpreted as representations of surface brightness. We have also used the term representation to describe our results on brightness-related BOLD responses. We acknowledge that the interpretation of the BOLD responses is more complicated than the spike rate and that the responses are correlative. However, the increase of the BOLD signal strength may well be associated with actual representation of brightness. The fMRI signal strength can comprise information about the stimuli as shown by the correlation between signal changes and decoding accuracy in multivariate pattern analysis (Tong, Harrison, Dewey, & Kamitani, 2012). The ROI analysis (averaging) unavoidably loses information, but still the signal strength likely reflects the neural representation of the parameter of interest. An alternative interpretation would be that such variations reflect some epiphenomenon without a link to neural information. 

Previous psychophysical results using filtering and noise masking suggest that only a narrow band of spatial frequencies contributes to brightness perception (Perna & Morrone, 2007; Salmela & Laurinen, 2005, 2009). Further, in White's illusion mask, containing both a narrow orientation and a narrow spatial frequency band is sufficient for diminishing the illusion (Salmela & Laurinen, 2009). These results support the role of V1 in brightness perception because the properties of the effective filters and noise masks resemble the properties of the simple cells in V1 (De Valois, Albrecht, & Thorell, 1982; De Valois, Yund, & Hepler, 1982). The role of V1 in brightness perception is also in accordance with some of the models of brightness perception that are based on mechanisms similar to those found in the early visual cortex, e.g., low-level filtering and normalization (Blakeslee et al., 2005). 

The transparent stimulus configuration allows perceptual separation of brightness and lightness. Because lightness corresponds to the reflectance of the surface, which is typically constant, our dynamic changes of the target are apparently interpreted as changes in brightness (e.g., due to change in illumination). However, with transparent stimuli, lightness can be dissociated from brightness. In the transparent White stimulus (Figure 1C), the target surface appeared to have constant lightness, which changed only when the polarity of the surround changed (Figure 2, lightness change is systematic). In the transparent Cross stimulus, the lightness was more ambiguous, and the interpretation (transparent light bar in front of the opaque dark bar vs. transparent dark bar in front of the opaque light bar) fluctuated (Figure 2, lightness change fluctuating). Because lightness co-varies with transparency, we cannot exclude the possibility that, in transparent conditions, the lightness mechanism, if different from the brightness mechanism, could also affect the measured BOLD response. 

Transparency

There is some evidence for the processing of transparency in the early visual areas. Border ownership of the responses to luminance borders, represented in V2 (Fang et al., 2009; Qiu & von der Heydt, 2007), may support the representation of transparency. In addition, chromatic stimuli inducing illusory transparent surfaces have evoked V1 responses (Sasaki & Watanabe, 2004). However, Dojat et al. (2006) reported inconsistent results in V1. In agreement with former findings, we found stronger responses to transparent than opaque stimuli in areas V1–V3. 

There were some differences between the transparent and opaque stimuli. Cross stimuli had reduced surround contrast, but this seems not to explain the results because the effect of transparency was significant when this stimulus was excluded from the analysis. Another explanation could be the size of the target, which was larger in the transparent than opaque White stimulus. Cross stimuli can be interpreted to contain only two transparent bars, and thus the target in Cross stimulus is even larger than in transparent White stimulus. However, when responses to different stimuli were compared, the largest response was found for transparent White stimulus. Thus, we think that the perceived size of the target does not explain the different responses. In transparent White stimuli, the target contrast was reduced compared to other stimuli (Figure 3C), but it contained an additional luminance edge near the target. The reduced local contrast is an unlikely explanation for the increased response, but the increased number of edges could be a confounding factor between the transparent and opaque targets. However, there is no obvious method to control this effect. Further, in half of the subjects, the effect of transparency diminished when stricter voxel selection was used, suggesting a confounding effect of the surround for these subjects. 

Limitations

The subjects' attention was not controlled during the fMRI measurements, but instead, the subjects were instructed to pay attention to whether or not the target brightness modulated. We assume that the orienting of attention is not affected by the presence of brightness modulation. Attention is oriented to the target, and there is no reason to assume that perceiving brightness modulation would require more attention than perceiving a constant surface. Further, it would have required some effort for the subjects to selectively attend to certain stimuli more than others. The difference between the conditions (modulating vs. constant brightness) was also relatively small, and hence there was no strong “pop-out” type of stimulus that could have drawn their attention automatically. Further, the real and illusory brightness modulations were perceptually indistinguishable, but still we found differences in the evoked responses. Modulating brightness and transparent surface percepts may evoke stronger responses due to higher saliency of the target and thus higher evoked attention. However, the saliency must emerge first from the processing of the target, especially of illusory brightness, before attention can be drawn into it. Hence, if attention is confounding our results, it would be secondary to initial processing of the brightness and transparency of the target. 

The experimental setup of the behavioral experiments and the fMRI measurements were not identical. In the behavioral tests, we measured an estimate of brightness, and the subjects could view the stimuli freely for as long as they wanted to before making a decision. However, the main aim of the behavioral experiment was to confirm that the subjects saw the difference between the conditions we compared. 

The response to the surrounding stimulus also unavoidably modulates the BOLD response in the area primarily representing the target (Nurminen, Kilpelainen, Laurinen, & Vanni, 2009). This cannot be avoided in any center-surround paradigm, which means that the target signal is dominated by the target response but contributed by the response to the surround. For broadband images, rms contrast is a typically used metric for contrast (Bex & Makous, 2002; Olman, Ugurbil, Schrater, & Kersten, 2004). Because the surrounds that we compared have similar rms contrasts, the different spatial configurations as such should not produce differences in the magnitude of the measured BOLD signal. Further, the stricter voxel selection did not have any effect on the difference between the responses to modulating and constant brightness. 

Summary

Several single-cell recordings and fMRI studies suggest that the brightness of achromatic surfaces is represented in the responses of the neurons in the primary visual cortex. Our study provides further support for the role of V1 in brightness perception and extends the findings to more complex images composed of both luminance increments and decrements. Further, our results suggest that surface transparency is also represented in early visual cortices. We did not find an interaction between brightness and transparency or differences between the early visual areas. The results suggest that many perceptual properties of achromatic surfaces are separately represented in the neural activity of early visual areas. 

Supplementary Materials

mov S1 

mov S2 

mov S3 

mov S4 

mov S5 

mov S6 

mov S7 

mov S8 

Acknowledgments

We thank Linda Henriksson, Lauri Nurminen, and Fariba Sharifian for help in data analysis and fMRI measurements and two anonymous reviewers for helpful comments that improved the manuscript. This study has been supported by the Finnish Cultural Foundation, the Finnish Graduate School of Neuroscience, and the Academy of Finland (National Program for Centers of Excellence 2006–2011 and grants 124698, 140726, and 218054). 

Commercial relationships: none. 

Corresponding author: Viljami R. Salmela. 

Email: [e-mail]. 

Address: Institute of Behavioural Sciences, Division of Cognitive and Neuropsychology, University of Helsinki, Finland; Advanced Magnetic Imaging Centre, Aalto Neuroimaging, Aalto University, Aalto, Finland; Brain Research Unit, O.V. Lounasmaa Laboratory, Aalto University, Aalto, Finland.