Early correlates of visual awareness in the human brain: Time and place from event-related brain potentials

Abstract

When something appears, how soon is the first neural correlate of awareness of it, and where is that activity in the brain? To answer these questions, we measured the electroencephalogram under conditions in which visual stimuli changed identically but in which awareness differed. We manipulated awareness by using binocular rivalry between orthogonal gratings viewed one to each eye. Then we changed the orientation of the grating to one eye to be the same as that to the other eye. Because of the rivalry, sometimes this happened to the visible grating, producing a clear change in perceived orientation, and other times it happened to the invisible grating, producing no change in perceived orientation. This procedure allowed us to analyze time-locked topographic scalp and tomographic primary current densities of the event-related potentials to physically identical events differing in their perceptual consequences. When the change in orientation reached awareness, neural responses began at about 100 ms, spreading mainly from dorsal occipital areas. When the change in orientation did not reach awareness, neural responses also began at about 100 ms, but they were attenuated, particularly in the right fusiform gyrus. We place the earliest correlate of visual awareness following binocular rivalry in the ventrolateral occipitotemporal cortex.

Introduction

When something new appears in front of our eyes, we may or may not become aware of it. The question about the processes underlying awareness is old (e.g., Donders, 1868/1969; Hirsch, 1865), but modern techniques such as fMRI and EEG have recently been deployed to answer it (e.g., Engel & Singer, 2001; Koivisto, Revonsuo, & Lehtonen, 2006; Lumer, Friston, & Rees, 1998; Pins & ffytche, 2003; Rees, 2001). These new techniques help to reveal when and where in the brain the processes mediating awareness take place. 

To answer questions about the earliest time and place of correlates of awareness in the human brain, we used an approach pioneered by Kaernbach, Schröger, Jacobsen, and Roeber (1999). They measured event-related potentials (ERPs) in two conditions that differed only in awareness, exploiting the phenomenon of binocular rivalry (e.g., Blake, 2001; Dutour, 1760, translated by O'Shea, 1999; Wheatstone, 1838). In both conditions, observers viewed a left-oblique grating with one eye and an identical right-oblique grating with the other. We call this rivalry stimulation. Because of binocular rivalry, sometimes only the left-oblique grating was visible to an observer while the right-oblique grating was completely invisible, and sometimes it was the opposite. Kaernbach et al. had each observer press keys to signal which of the two gratings he or she was perceiving. Every now and then, one grating was changed, say the left-oblique grating, to give it the same orientation as the other grating. We call this fusion stimulation. When the change in orientation happened while the observer was perceiving the left-oblique grating (say), he or she became aware of the change in orientation. Kaernbach et al. defined awareness as the observer's pressing a key to denote that the orientation changed. We call this sort of change a percept-incompatible transition. When the change happened while the observer was seeing the right-oblique grating, he or she did not become aware of the change in orientation. Kaernbach et al. defined lack of awareness as the observer's not pressing a key. In fact, under these circumstances, some observers noticed a very subtle change in the appearance of the visible stimulus, such as a slight enhancement in its contrast. For our purposes, we will treat this as lack of awareness. We call this sort of change a percept-compatible transition. Because percept-incompatible and percept-compatible transitions are physically identical, any difference in their neural response characteristics is a correlate of visual awareness. By comparing ERPs from these two transitions, Kaernbach et al. found activity about 200 ms after the transition that was correlated with awareness. Similar research excluding differences in the effect of percept-compatibility between flickering (16.7 Hz), and steady stimulus presentations found an even earlier time for a neural correlate of awareness at about 100 ms (Roeber & Schröger, 2004). This time agrees with that estimated using different approaches to manipulate awareness, such as visual masking (Koivisto et al., 2006) and multistable images (Kornmeier & Bach, 2005, 2006; Pitts, Nerger, & Davis, 2007). 

Those ERP studies relied on the exquisite temporal resolution of ERP data but had limited spatial resolution to locate underlying neural activity. Imaging studies, such as via fMRI on the other hand, have focused on the brain regions underlying visual awareness For example, Tse, Martinez-Conde, Schlegel, and Macknik (2005) found activity in occipital regions when simple stimuli were presented. Polonsky, Blake, Braun, and Heeger (2000) and Tong, Nakayama, Vaughan, and Kanwisher (1998) found activity in ventral occipitotemporal regions correlated with awareness during binocular rivalry. And Lumer et al. (1998) found activity in extrastriate ventral, frontal, and parietal regions that reflected perceptual transitions during binocular rivalry. As in all fMRI studies, the temporal resolution was limited by that of the BOLD response in the order of several seconds. 

To study the earliest brain correlates of visual awareness, we use a recent method of analyzing EEG data that retains its exquisite temporal resolution while offering spatial resolution in the order of (although still less than) that from fMRI: variable resolution electromagnetic tomography (VARETA; Bosch-Bayard et al., 2001). To locate the places in the brain underlying the early awareness-related responses, we used 61 scalp electrodes. To avoid the junction of the foveal representations of areas V1, V2, and V3 at the occipital pole (foveal confluence, e.g., Tootell, Dale, Sereno, & Malach, 1996), we used annular stimuli with contours at least 0.2° from central vision. Using ERPs and VARETA, we were able to locate the earliest correlate of visual awareness to around 100 ms in higher-order visual areas, especially in the ventrolateral occipitotemporal cortex. 

Methods

Participants

Fourteen participants performed the experiment either for course credit or for payment (5 per hour). All had normal or corrected-to-normal acuities in each eye, and all gave written informed consent prior to the experiment. Participants were selected after they showed normal binocular rivalry in a 10-minute test session. Data of three participants were excluded from analyses because too few EEG epochs remained after valid trial selection and artifact rejection (see ERP analysis section). Mean age of the remaining eleven participants (all right-handed; two male) was 24.2 years (range, 19–37 years). 

Apparatus

Stimuli were displayed on a Belinea 106080 monitor (1,152 × 768 pixels; 60 Hz) controlled by a PC using the ERTS software package (BeriSoft). Participants viewed the stimuli through a mirror stereoscope (SA-200-Monitor-Type) resulting in a viewing distance of 41.5 cm. The experiment was conducted in a shielded booth. During the experiment, the participant's head was stabilized by a head and a chin rest. Participants responded using the ERTS keypad. 

EEG recording

The electroencephalogram (EEG) was recorded with NeuroScan amplifiers using 61 electrode sites according to the international 10–10 system (for the positions used, refer to Figure 2) and referenced to an electrode placed on the right earlobe. To control for eye movements, vertical and horizontal electro-oculograms (EOG) were recorded with bipolar electrodes placed above and below the right eye (vertical EOG) and the outer canthi of both eyes (horizontal EOG). Impedances were kept below 5 kΩ. Data were digitized at 500 Hz and online band-pass filtered from 0.1 to 100 Hz. 

Stimuli

In order to avoid foveal stimulation, we used annulus-shaped patches of sine wave gratings. The patches had a diameter of 5.5° and a spatial frequency of 0.8 cycles/degree. There was an inner medium grey (58.5 cd/m ²) disc of 0.4° diameter, which included a black (0.1 cd/m ²) fixation cross (of 0.3° diameter). At the inner and outer edges of the annulus, the contrast of the grating was smoothed to zero with a cumulative gaussian profile of three standard deviations (1 SD = 0.03°). The patches had a mean luminance of 121.7 cd/m ² and a contrast of 0.998. The gratings were slanted orthogonally to the upper left and upper right by 45° from vertical. Stimuli were dichoptically presented on a medium grey (58.5 cd/m ²) background. Positions of the stimuli were adjusted for each participant to align the stimuli for relaxed binocular viewing. 

Procedure

The participant's task was to report the exclusive visibility of one or the other orientation by holding down one or another key. Neither key was to be pressed if any combination of the two orientations was perceived. We used key presses only to classify the perceptions of the participants; we did not analyze them further except to confirm that the distributions of rivalry-dominance durations has the typical gamma shape (Fox & Herrmann, 1967; Levelt, 1967). 

The experiment was subdivided into 21 blocks of five minutes each. Within a block, periods of rivalry stimulation and periods of fusion stimulation were randomly exchanged such that no more than two periods of rivalry stimulation and no more than two periods of fusion stimulation directly succeeded each other (for a typical trial sequence, see Figure 1A). Periods of rivalry were shown for at least 6,000 ms until the participant's next key press. The rivalry display then continued for a random interval of between 800 and 1,000 ms before a change to fusion stimulation. Periods of fusion stimulation lasted 2,000 to 3,000 ms. Changes to the stimuli occurred during the vertical retrace interval of the monitor. 

Figure 1

 

Design and setup of the relevant events. (A) Periods of rivalry stimulation (gratings on left and right eye differ in orientation) and periods of fusion stimulation (gratings on left and right eye do not differ in orientation) were randomly exchanged. Periods of rivalry stimulation induced binocular rivalry, alternations in the perceived orientation. Periods of rivalry lasted for at least 6 s until the participant's next key press. After that key press, the rivalry display continued for a random interval of between 0.8 and 1 s before a change to another rivalry or fusion stimulation. Periods of fusion stimulation lasted 2 to 3 s. Transitions from rivalry to fusion stimulation could (light blue marks on the time scale in panel A) (B) be compatible with the reported percept, a percept-compatible transition; or (C) be incompatible with the reported percept, a percept-incompatible transition. Transitions from fusion to fusion stimulation (dark-blue mark on the time scale in panel A) served as control condition which did not involve rivalry, a non-rival transition.

ERP analysis

For data analyses, EEGs were off-line low-pass filtered below 35 Hz (Kaiser window sinc FIR filter; Kaiser beta = 5.653; filter order = 1,812). Events were classified depending on the physical input and the currently reported percept. Transitions from rivalry to fusion could be either compatible with the reported percept, leading to no perceptual change ( percept-compatibletransition; see Figure 1B) or incompatible with the reported percept, leading to a perceptual change ( percept-incompatibletransition; see Figure 1C). Event classification resulted in an average of 121 (±6) percept-compatible and 133 (±8) percept-incompatible transitions per participant. We also had transitions when two periods of fusion stimulation directly succeeded each other, allowing us to measure a condition involving no binocular rivalry. For example, in a first period, both gratings might have been left-oblique; in the second period they both changed to right-oblique. There were 108 (±2) of these transitions per participant, which we call non-rivaltransitions (dark-blue mark in Figure 1A). 

For all transitions, we extracted epochs in a 1,200-ms time window time locked to the onset of the fusion (or second fusion) stimulation and including a pre-stimulus transition baseline of 200 ms. Epochs were considered only (1) if there was no key press or key release from 200 ms preceding to 200 ms following the stimulus transition; (2a) if the key press was changed to the correct key after a percept-incompatible or non-rival transition; or (2b) if the correct key remained pressed after a percept-compatible transition. ERPs were averaged across the individual data. Artifact rejection was performed prior to averaging in order to eliminate trials contaminated with extensive EOG activity (maximum signal change of 100 μV within a time window from 100 before to 600 ms after stimulus transition). ERPs were computed from an average of 112 (±6) percept-compatible, 125 (±7) percept-incompatible, and 96 (±4) non-rival epochs per participant. 

We focused especially on the earliest differences among ERPs to percept-incompatible, to percept-compatible, and to non-rival stimulus transitions. Based on visual inspection, we found this earliest difference at the first positive peak (P1) on posterior electrode sites in the ERP. All analyses described below are based on the mean signal amplitude derived from the individual ERPs within a 30-ms interval around the latency of this peak in the grand-average ERP (96 to 126 ms after stimulus transition). 

Using a spherical spline interpolation, we generated scalp potential maps to analyze the spatiotemporal structure with a higher spatial resolution. We estimated the scalp current density (SCD) distribution from the surface Laplacian (second spatial derivative of the potential distribution; Perrin, 1990; Perrin, Pernier, Bertrand, & Echallier, 1989) with a conductivity of 0.45 S/m, choosing the maximum degree of the Legendre polynomials to be 50 and the order of splines to be 4. For statistical analyses, two triplets of parietal and parieto-occipital electrodes in each hemisphere were selected that cover the components of the P1 sources. In order to account for interindividual variability in topography, we averaged the individual potential and SCD values across electrode triplets (refer to Figure 2 for the electrodes involved in averaging). Differences in scalp potential and SCD between percept-incompatible and percept-compatible transition conditions were assessed by repeated measures analyses of variance (ANOVAs), which included the factors percept-compatibility (compatible vs. incompatible), hemisphere (left vs. right), and electrode line (parietal vs. parieto-occipital). In addition, to compare differences in topographical shape independent of amplitude differences, we scaled potential data according to the vector length method proposed by McCarthy and Wood (1985; as recommended by Picton et al., 2000) and reassessed statistical significance by equivalent repeated measures ANOVAs. 

Figure 2

 

Electrophysiological data. The graphs show voltage on the Y-axis and time on the X-axis. As is conventional with ERPs, positive voltages are shown below zero on the Y-axis and negative voltages above. Time 0 on the X-axis is when the stimulus change took place. Key presses of observers (indicating that the stimulus change was seen in appropriate conditions) took place after more than 600 ms. The ERPs are the averages of the ERPs of the participants. There are three ERPs per graph. The red line is when the rivalry-to-fusion stimulus change led to a perceptual change (percept-incompatible transitions). The green line is when the identical stimulus change did not lead to a perceptual change (percept-compatible transitions). The blue line is when the stimulus transition was non-rival, which always led to a perceptual change (fusion to fusion transitions). EEG activity was recorded from electrode positions as depicted in the schematic head. ERP data shown are averaged across four electrode triplets (as indicated by black contours) to give frontal, central, parietal, and parieto-occipital sites for the left and right hemisphere. Individual ERPs were first averaged across electrode triplets, and the resulting curves were then averaged across participants. Significant early differences between percept-incompatible and percept-compatible transitions are shown as shaded bars. The ERP-components P1, N1, and P3b are highlighted (light-grey boxes) where they are most pronounced (P1 and N1, parieto-occipital right; P3b, parietal right). The magnified curves from right parieto-occipital sites highlight the earliest ERP difference between percept-incompatible and percept-compatible transitions: P1 (dark grey box). Blue-shaded boxes indicate the parietal and the parieto-occipital sites included in the repeated measures ANOVAs of the P1 effect.

To reveal the generators of P1, we applied brain electrical tomography analyses by means of the VARETA approach (Bosch-Bayard et al., 2001; Picton et al., 1999; Valdés-Sosa, Marti, Garcia, & Casanova, 1998/2000). With this technique, sources are reconstructed by finding a discrete spline-interpolated solution to the EEG inverse problem: estimating the spatially smoothest intracranial primary current density (PCD) distribution compatible with the observed scalp voltages. This allows for point-to-point variation in the amount of spatial smoothness and restricts the allowable solutions to the grey matter (based on the probabilistic brain tissue maps available from the Montreal Neurological Institute; Evans et al., 1993). This procedure minimizes the possibility of “ghost sources,” which are often present in linear inverse solutions (Trujillo-Barreto, Aubert-Vázquez, & Valdés-Sosa, 2004). A 3D grid of 3,244 points (voxels, 7 mm grid spacing), representing possible sources of the scalp potential, and the recording array of 61 electrodes were registered to the average probabilistic brain atlas. Subsequently, we transformed the scalp potentials for all transition conditions separately into source space (at the predefined 3D grid locations) using VARETA. Statistical parametric maps (SPMs) of the PCD estimates were constructed based on a voxel-by-voxel Hotelling T² test against zero in order to localize the sources of the P1 for each transition condition. We compared percept-incompatible and percept-compatible transitions by a repeated measures ANOVA to localize differences in activation. Corresponding SPMs were constructed based on the ANOVA's output. For all SPMs, we used the random field theory (Worsley, Marrett, Neelin, & Evans, 1996) to correct activation threshold for spatial dependencies between voxels. We show results as 3D activation images constructed on the average brain. 

Results

We show ERPs to transitions from rivalry to fusion stimulation—time-locked to the onset of the fusion stimulation—in Figure 2. Note that to be consistent with the majority of ERP literature, we show positive deflections (P) going below the Y-axis and the negative deflections (N) going above (e.g., Luck, 2005, p. 10). Also to be consistent with ERP literature, we give these deflections numbers depending on their order (e.g., P1, N1) from time 0 on the X-axis (e.g., Luck, 2005, pp. 10–11). Percept-incompatible and percept-compatible transitions elicited P1 and N1, which were most prominent at parieto-occipital and parietal sites. There were also positive deflections starting at about 300 ms after the transition, which were most pronounced at parietal and central sites (marked as P3b at the parietal right electrode cluster in Figure 2); these were associated with detection of a task-relevant event (P3b) as reflected by key releases and presses. Only percept-incompatible and non-rival transitions were followed by a release of the key that indicated the perceived pre-transition orientation and a subsequent key press indicating the perceived post-transition orientation. Mean reaction time for those key presses were 758 (±38) ms for percept-incompatible transitions and 634 (±35) ms for non-rival transitions. Participants did not release or press any key after percept-compatible transitions. 

To show the dynamics of brain activation following the stimulus transitions to fusion stimulation, we provide a movie of the VARETA analyses as moving window averages (20 ms windows) of the PCD distributions, with one frame every 10 ms from 0 to 600 ms after stimulus transition ( Movie 1). Starting with the onset of the fusion stimulation for the three different transitions ( Movies 1A– 1C), the movie shows activity in occipitotemporal regions being followed by surges of activity, especially after percept-incompatible and non-rival transitions ( Movies 1A and 1C), in occipital regions reaching a maximum at 100 ms after the transition dorsally in the middle occipital gyrus (P1). Activity then spreads to temporal regions while retaining its parieto-occipital center reaching another maximum there at about 170 ms (N1), especially after percept-compatible transitions ( Movie 1B). This is followed by further spreading of activity through occipital, parietal, and temporal cortex, reaching motor areas at about 600 ms after percept-incompatible and non-rival transitions ( Movies 1A and 1C), possibly indicating the preparation of the key press (P3b; reaction times were at about 630 to 760 ms on average). The movie for the contrast between percept-incompatible and percept-compatible transitions ( Movie 1D) shows first significant differences in activation at a latency of about 50 ms in the left fusiform gyrus, which are likely to be artifacts because we do not see them in the ERP traces. However, at about 80 ms, differences in activation start to emerge in the right superior temporal areas, surging into lateral–ventral occipto-temporal areas as well as dorsal parieto-occipital and occipital areas (P1) before they decline. Activation differences then build up anew at around 200 ms (N1) and surge back and forth especially between the middle occipital and the superior temporal gyri but also spreading into adjacent occipital, parietal, and temporal regions, including motor areas at about 530 ms (P3b). 

 

Movie 1

 

Primary current densities (PCDs) of the electrophysiological scalp activity from 0 to 600 ms after stimulus transition estimated by the VARETA approach are shown as intensity projection movies, as moving window averages (20 ms windows), with one frame every 10 ms from 0 to 600 ms after stimulus transition (thresholded to p < 0.0001; the hotter colors correspond to higher probability values), for (a) percept-incompatible, (b) percept-compatible, (c) and non-rival transitions as well as for (d) the contrast between activations due to percept-incompatible and percept-compatible transitions.

Because we are interested mainly in the earliest difference (P1) in the stream of visual processing between stimulus changes with and without awareness, we do not elaborate further on N1 and P3b effects. Figure 3 depicts the topographic (scalp potential and SCD) and tomographic (PCD) distributions within the P1 time window (96 to 126 ms). We also show results for equivalent analyses of the ERP and its P1 component to non-rival transitions that compel a perceptual change. This allows us to compare electrophysiological activity in a condition in which no perceptual ambiguity is involved. We only ran statistical comparisons between percept-incompatible and percept-compatible transitions from rivalry to fusion stimulation because they are physically identical, and therefore any difference between their neural response characteristics must be ascribed to the perceptual awareness of the stimulus transition. This is not the case for non-rival conditions, in which the physical changes in the stimulus could be said to be confounded with the perceptual changes. 

Figure 3

 

Topographic and tomographic distributions of the P1 component. Scalp potential maps were generated by means of spherical spline interpolation (top row). SCDs were estimated by computing the second derivative (surface Laplacian) of the interpolated potential distributions (middle row). PCDs were computed by VARETA analysis (third row). These analyses are shown for percept-incompatible transitions (first column), for percept-compatible transitions (second column), for the difference between them (third column), and for non-rival transitions (fourth column). In the first and second rows, p value maps (smaller heads) indicate electrode sites with amplitude values that differed significantly from zero (after Bonferroni correction). In the bottom row, PCDs are shown as SPMs on orthogonal planes of the average brain. Slices were taken on the respective centers of gravity (we give the MNI coordinates for each slice in the figure). Note, the third column depicts the projections of the SPMs of the inverse solution for the contrast between activations due to percept-incompatible and percept-compatible transitions. The hotter colors correspond to higher probability values (thresholded to T ² > 25.45, which corresponds to p < 0.0001).

The potential maps (absolute voltage) of P1 for all transitions show bilateral maxima at posterior electrode sites, one on each hemisphere ( Figure 3, top row). The ANOVA results indicate a significant percept compatibility by electrode-line interaction on both the absolute and the vector-scaled potentials: percept-incompatible transitions elicited larger amplitudes than percept-compatible transitions, but this difference was larger on parieto-occipital than on parietal electrodes: F(1,10) = 8.31 for the absolute and F(1,10) = 8.33 for the vector-scaled data, both p = 0.016. 

The SCD maps ( Figure 3, middle row) show a corresponding posterior pattern of pronounced bilateral parieto-occipital/parietal maxima (current sources), which are shifted slightly lateral for percept-compatible as compared to percept-incompatible transitions. The ANOVA results indicate a significant interaction between percept-compatibility and electrode line: percept-incompatible transitions yielded larger amplitudes than percept-compatible transitions at parieto-occipital sites only, F(1,10) = 20.75, p = 0.001. We also found a significant main effect of hemisphere: SCD amplitudes were larger on the right than on the left hemisphere, F(1,10) = 5.21, p = 0.046. 

The critical part of Figure 3 is the bottom row, which shows the brain electrical tomography analysis. For both percept-incompatible and percept-compatible transitions, most PCD activation (center of gravity) occurs dorsally, in the middle occipital gyrus, close to the junction of the occipital, the parietal, and the temporal lobes. Activity was distributed over occipital, temporal, and parietal areas (refer to the maximum intensity projections between 80 and 130 ms in the movies). Activation was more widespread for percept-incompatible than for percept-compatible transitions (78.2 vs. 52.1 cm ³ significantly activated tissue). In the third column of the bottom row in Figure 3, we show the contrast in PCD between percept-incompatible and percept-compatible transitions. It shows significant differences in source activation between both transitions most prominently in the posterior fusiform gyrus but yields additional local maxima in the inferior temporal gyrus and in the middle occipital gyrus. Activation differences occurred especially in the right hemisphere (28.8 cm ³ significantly activated tissue in the right hemisphere vs. 2.7 cm ³ significantly activated tissue in the left hemisphere). 

Discussion

Our results show that the earliest modulation of the ERP happens about 100 ms after a change in stimulus orientation. This modulation of ERP at 100 ms is significantly enhanced when the orientation change is associated with awareness (percept-incompatible transitions) compared with an identical stimulus change that is not associated with awareness (percept-compatible transitions). Within this time window, processing of both types of transitions involves a similar network of occipital, parietal, and temporal brain areas, but critical differences between perceived and non-perceived changes emerge in the ventrolateral occipitotemporal cortex. Note that we are not saying that awareness begins 100 ms after a stimulus transition; we are saying only that the earliest neural correlate of awareness occurs 100 ms after a stimulus transition. 

The behavioral data we measured during periods of rivalry confirm that perception was fluctuating: The distribution of dominance phase durations nicely follows a gamma function typical for phenomena of perceptual ambiguity (Fox & Herrmann, 1967; Levelt, 1967). The behavioral data we measured following the various transitions show that awareness differed in the percept-incompatible and the percept-compatible transitions: Observers pressed keys in the former but not in the latter. This is consistent with psychophysical data for similar transitions collected by Julesz and Tyler (1976) and Tyler and Julesz (1976). They found that times required to detect transitions from rivalry to fusion in dynamic random-dot stereograms were about eight times longer than those to detect transitions from fusion to rivalry. 

The electrophysiological data we measured show that percept-compatible transitions elicit similar exogenous ERP components (P1, N1) as percept-incompatible transitions do—indicating that stimuli are initially processed even though participants were not aware of them (for corresponding results, see de Labra & Valle-Inclán, 2001; Valle-Inclán, Hackley, de Labra, & Alvarez, 1999). More importantly, the data show that the P1 component is enhanced when the participants are aware of the change, confirming earlier findings (Roeber & Schröger, 2004). Our new findings extend earlier work by giving coherent estimations about the brain structures involved in the modulation of awareness. The topographic (SCD) and the tomographic (VARETA) analyses appear highly consistent and provide converging evidence for the involvement of a similar network of occipital, parietal, and temporal brain areas in the processing of both percept-incompatible and percept-compatible stimulus transitions (see Figure 3). 

Focusing on the earliest occurrence of a percept-dependent modulation, we would like to argue that the effect in the P1 range is due to enhanced neural responses to percept-incompatible over percept-compatible transitions rather than due to additional generators. Our VARETA results support this argument: The main focus of activation was located in the dorsal part of the middle occipital gyri, relatively close to the junction of occipital, parietal, and temporal lobes for both percept-incompatible and percept-compatible transitions. This region is close to the brain structures retinotopically mapped and functionally described as visual areas V3/V3a (e.g., Tootell et al., 1997). From the bottom row of Figure 3 it can be seen that activation is not restricted to that dorsal region but is more widespread including ventrolateral occipitotemporal areas. 

Thus, our findings yield two implications so far: First, the brain structures involved in generating the P1 component to pattern onset stimuli (Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2001) are also involved in generating the P1 to a stimulus change within situations of permanent visual input. Also, the topographic and the tomographic maps of the P1 activation pattern elicited by non-rival transitions (the right column in Figure 3) show a similar distribution that apparently differs in activation strength only. Second, although activation spreads dorsally and ventrally through extrastriate occipitotemporal cortex, there is no indication of an additional brain region responsible for mediating perceptual awareness by suppressing or boosting conflicting stimulus information. 

Importantly, for both dorsal and ventral extrastriate P1 sources, we observed percept-dependent modulations of activity for transitions from rivalry to fusion stimulation. More specifically, although the dorsal occipital and the parieto-temporal activations differ a little between percept-incompatible and percept-compatible transitions, it is the activity in the ventrolateral occipitotemporal region, more specifically in the posterior fusiform gyrus, that appears to be most strikingly affected by the percept incompatibility of the rivalry to fusion transition (refer to the SPM map in the third column of the bottom row in Figure 3). We take this region-specific effect of percept dependency as the crucial difference in processing the same physical stimulus transition with or without awareness. This finding is complemented by results from a combined ERP and fMRI-study on near-threshold stimuli (Pins & ffytche, 2003), showing a differential response for seen versus unseen stimuli within the (right) lateral occipital region at about 100 ms (P1). Supporting evidence also comes from a monkey single cell study showing that activity correlates with the perceptual fluctuations during binocular rivalry more strongly in V4 neurons than in V1 and V2 neurons (Leopold & Logothetis, 1996). Moreover, such a pattern fits to notions that the ventral circuit is a necessary prerequisite for conscious visual perception as opposed to the dorsal circuit, which is necessary for the visual control of actions (Goodale & Milner, 1992). 

The special role of ventral areas for visual awareness receives further support from studies on visual spatial attention, which find enhanced activation in the occipitotemporal stream for attended as compared to unattended stimuli (Heinze et al., 1994; Mangun, Hopfinger, Kussmaul, Fletcher, & Heinze, 1997). It seems that spatial attention exerts a selective gain control or amplification of attended inputs in extrastriate cortex before visual processing proceeds into temporal brain regions (Hillyard, Vogel, & Luck, 1998). Although there is an ongoing debate about the relationship between attention and awareness (Koivisto et al., 2006; Lamme, 2003; Milner, 1995), it seems plausible from our results to infer a similar mechanism of amplification for visual input in order to reach perceptual awareness, which—at a latency of around 100 ms—is especially mediated by ventral areas. Our findings also show a right-hemispheric dominance, which strengthens ideas about the functional asymmetry of the human brain. There is evidence from fMRI studies that visual ventral stream processing shows more activation in the right than in the left hemisphere (Fink, Marshall, Weiss, & Zilles, 2001; Macaluso & Frith, 2000; McCarthy, Puce, Gore, & Allison, 1997). The reasons for this asymmetry are not well understood yet. Goodale and Milner (2004) have proposed that the corresponding areas in the left hemisphere are functionally assigned to speech and language processing leaving fewer resources for visual processing. 

An alternative perspective is not that awareness enhances neural activity, but that binocular rivalry suppression—the tool we used to manipulate awareness—attenuates neural activity. In some senses, the difference between whether awareness enhances activity or rivalry attenuates it is only semantic. In order to have any point of comparison, awareness always needs to be compared with some reduced form of awareness that we achieved with binocular rivalry. In any case, the conditions for binocular rivalry ended when we changed the stimulus (giving conditions for binocular fusion). This is time zero for the ERPs. Nevertheless, it is likely that the effects of binocular suppression endured for some time after the offset of the rival stimuli. Suppression continuing to attenuate neural activity is consistent with monkey single-cell and human fMRI studies showing modulations in activation correlated with the perceptual fluctuations of binocular rivalry in the lateral geniculate nucleus (Haynes, Deichmann, & Rees, 2005; Wunderlich, Schneider, & Kastner, 2005) and in striate and early extrastriate areas (Leopold & Logothetis, 1996; Tong & Engel, 2001; Tong et al., 1998). For human observers, we can relate the modulation in these areas to its occurrence in time, which to our knowledge has not been done before due to the limitations in temporal resolution of fMRI data or of the steady-state stimulation technique used in MEG or EEG studies (Brown & Norcia, 1997; Srinivasan & Petrovic, 2006; Srinivasan, Russell, Edelman, & Tononi, 1999). The activation differences in these areas occur as early as 100 ms after stimulus transition, which is very fast (Koivisto et al., 2006), but too slow to account for modulations in V1 during initial stimulus processing (Hochstein & Ahissar, 2002; Lamme & Roelfsema, 2000). Our results parallel findings from a combined ERP and fMRI study on spatial attention by Martínez et al. (1999). They could not find an attentional modulation during initial processing of visual input in their ERPs and thus related the modulation in V1 fMRI activity to delayed or re-entrant feedback of enhanced visual signals from higher extrastriate areas. Our movie of the PCD time courses provides some evidence for a potential role of feedback (Movie 1). These show activity in occipitotemporal regions being followed by surges of activity (during percept-incompatible transitions) in occipital regions. Along these lines, monkey experiments have also revealed late responses to be relevant for perceptual awareness (Supèr, Spekreijse, & Lamme, 2001). 

Conclusion

Taken together, our results suggest that neural events as early as in the P1 range are correlates of perceptual awareness following binocular rivalry and provide support for the ventral pathway's special role in mediating perceptual awareness. 

Acknowledgments

This research was supported by the German Research Foundation (DFG grant no. RO 3061/1-1). 

Commercial relationships: none. 

Corresponding author: Urte Roeber. 

Email: [e-mail]. 

Address: Seeburgstr. 14-20, D-04103, Leipzig, Germany.