Stimulus rivalry and binocular rivalry share a common neural substrate

Abstract

When two incompatible images are shown separately to each eye, a perceptual process known as binocular rivalry occurs by which the two images compete for awareness. The site of competition for binocular rivalry has been a topic of debate, and recent theories are that it may occur either at low levels of the visual system where the inputs from the two eyes are combined or at high levels of the visual system where the two images are processed. One of the major pieces of evidence for a high-level image account of rivalry is a phenomenon known as stimulus rivalry, in which two competing stimuli are swapped between the eyes at 3 Hz. However, there is little available neurophysiological evidence for a neural substrate for this high-level competition. Here, we used frequency tagging of two competing stimuli in binocular rivalry and stimulus rivalry in humans to evaluate whether the steady-state visually evoked potentials (SSVEPs) show similar signatures of neural competition for both conditions. We found that flickering the stimuli generates spectral power at the tagged frequencies in both types of rivalry in the early visual cortex. We then quantified dynamic signatures of competition by tracking amplitude changes in the frequency tags, which showed that both types of rivalry colocalized in occipital regions of the cortex. Thus, contrary to our hypothesis that stimulus rivalry was being mediated by high-level competition between the images, we find that neural competition measured by the SSVEP instead suggests that the sites of competition for stimulus rivalry and binocular rivalry may similarly include the occipital pole and middle temporal gyrus (hMT+/V5) of the visual system, consistent with a low-level, binocular interpretation.

Introduction

In natural viewing, the human visual system fuses the two images from each eye into one representation of the outside world. However, when the images become sufficiently different, a perceptual process known as binocular rivalry may occur, in which one eye's visual information is alternately suppressed while the other eye dominates perception. Reciprocal inhibition of monocular neurons in V1 is considered an important mechanism in the resolution of rivalry (Blake & Logothetis, 2002; Tong, Meng, & Blake, 2006). Although an important neural mechanism of binocular rivalry (Blake, 1989), the reciprocal inhibition of eye-specific channels alone does not explain the occurrence of rivalry generated by high-level differences in stimuli that bypass monocular competition (Leopold & Logothetis, 1999; Wolf & Hochstein, 2011). To account for these findings, binocular rivalry is hypothesized to be a hybrid of both binocular pattern-level and monocular eye-level competition with feedforward and feedback influences across the visual hierarchy (Blake & Logothetis, 2002). Although this theory is sufficiently broad to encompass most, if not all, findings, it lacks detail on to what extent high, stimulus-specific, and low, eye-specific visual areas are involved in rivalry and what may be the influences between areas. 

Whether the neural competition yielding rivalry takes place between eye-level or stimulus-level neurons has still yet to be clearly demonstrated (Blake & Logothetis, 2002). One primary piece of evidence in favor of stimulus-level competition is a phenomenon discovered by Logothetis, Leopold, and Sheinberg (1996) they called “stimulus rivalry.” It occurs when incompatible dichoptic stimuli are swapped between the eyes at a rate of about 3 Hz, such that each eye, over time, views both of the competing stimuli. Logothetis et al. found that the perceptual transitions remain as they do in binocular rivalry, leading them to propose that competition during binocular rivalry is actually between the binocular stimulus representations situated at later visual stages, such as extrastriate regions or beyond. Thus, it was hypothesized that inhibition could potentially occur between neurons representing incompatible stimulus features in extrastriate regions or beyond (Leopold & Logothetis, 1999). 

Opposite to Logothetis et al.'s (1996) theory, electrophysiological studies suggest that changes occur throughout the brain during binocular rivalry, including in very early visual areas of the visual system. For example, Gail, Brinksmeyer, and Eckhorn (2004) found local field potentials from early visual areas in the monkey brain whose amplitudes are correlated with perception during binocular rivalry. Furthermore, when stimuli are flickered to generate a steady-state visual evoked potential (SSVEP), the signatures of competition can be found in the occipital and middle temporal (hMT/V5) visual areas (Zhang, Jamison, Engel, He, & He, 2011). Investigations of concurrent SSVEP and fMRI studies suggest involvement of other brain areas in parietal and cingulate regions alongside early visual areas (Roy, Jamison, He, Engel, & He, 2017). Thus, early visual areas show competitive interactions with coordinated activity occurring throughout the visual system as a consequence of resolving rivalry. 

The presumption of high-level competition during stimulus rivalry has also recently become less clear. Psychophysical evidence shows a monocular contribution to stimulus rivalry (Bhardwaj & O'Shea, 2012; Brascamp, Sohn, Lee, & Blake, 2013). Neuroimaging, using fMRI, suggests that brain networks of stimulus rivalry and binocular rivalry largely overlap with stimulus rivalry showing the same but generally weaker brain network activation (Buckthought, Fesi, Kirsch, & Mendola, 2015). Thus, although seemingly mediated by high-level processes, a difference in the locus of neural activation between stimulus rivalry and binocular rivalry has yet to be shown. 

Do the signatures of competing neural representations differ when the competition is at the level of the stimulus representations compared to when it is between the monocular representations? We tested the hypothesis that stimulus rivalry would show a similar pattern of SSVEP responses as binocular rivalry but among pattern-level representations in higher level brain regions (Leopold & Logothetis, 1999). Specifically, we used SSVEP frequency tagging to track the competing neural representations in binocular rivalry and stimulus rivalry. We used two different frequencies that stayed with the stimuli while subjects reported perceptual transitions and quantified neural competition from the amplitude changes in each frequency tag. We found that competitive neural signatures are localized to occipital brain areas in both binocular rivalry and stimulus rivalry. Thus, contrary to our hypothesis, these results indicate that stimulus rivalry and binocular rivalry competition may be supported by overlapping binocular neural mechanisms located in the occipital brain regions. 

Materials and methods

Participants

We studied normal human subjects with ages ranging from 18 to 65 years. A total of 40 subjects completed the behavioral experiment, and of those, a total of 26 subjects, nine of whom were male, participated in the EEG experiment. Subjects reported having normal or corrected vision and had no known history of neurological disorders. All experiments began after subjects signed and gave written consent of being informed of the experimental procedures in compliance with the University of Minnesota Institutional Review Board regulations on human subjects. 

Stimuli

We presented stimuli on a 24-in. HD LED ASUS monitor at a 144 Hz refresh rate and standard default display settings. Stimuli were orthogonal (+45° and +135°) red/green colored gratings of mean luminance 36.0 cd/m². We accounted for gamma correction by making photometer luminance measurements to ensure isoluminance between gratings. Stimuli had a contrast of 25%, the same as that used by Logothetis et al. (1996). Flicker was on/off going from 25% contrast to 0% contrast while maintaining constant mean luminance of 36 cd/m², in line with contrast values reported by previous studies of stimulus rivalry (Logothetis et al., 1996). Stimuli were in the shape of a square spanning 3° of visual angle on each side. The background included lines bisecting the screen to help with convergence along with surround contours around each stimulus and the fixation cross at the center. Background luminance was 5 cd/m². Stimulus flicker frequency was selected based on a pilot series of experiments on five subjects in which proportion of behavioral dominance was analyzed at different flicker frequencies ranging from 0 to 20 Hz. The green stimulus was frequency tagged at 12.0 Hz, and the red stimulus was tagged at 14.4 Hz because the proportion of dominance in binocular rivalry and stimulus rivalry were robust to flickering frequency combination. 

Experimental paradigm

Subjects began each experiment with a training session composed of each of the three experimental conditions in order to familiarize themselves with the stimulus and reporting procedures. Subjects viewed the stimulus through a mirror stereoscope, and care was taken to ensure that subjects fused both stimuli before training and experimental sessions began. Once subjects went through, on average, two sessions of practice in reporting the red, green, and mixed percepts (for stimulus rivalry, binocular rivalry, and replay, respectively), they were asked if they were confident in reporting each perceptual state. If they were confident in their reports, the experiment began; if not, they went through one or two more practice sessions. 

Subjects performed the experiment in a soundproof chamber with lights off, and they initiated continuation of each session with the press of any key, before which subjects could take breaks of variable durations according to their needs. There was a mandatory break halfway between the sessions. Each condition began with one run of stimulus rivalry, binocular rivalry, and replay, each for 60 s. For replay, reported perceptual transitions during binocular rivalry were replayed back to the subjects unless the binocular rivalry run had less than five responses total, in which case a standard template of durations was used. The transitions were as long as the indicated transitions measured during binocular rivalry. Otherwise, the template included transitions that were from a subject with 67 responses, of which 30 were transitions of different durations. A fading between gratings or a wedge sweeping radially across the grating were randomly alternated as simulated transitions for the replay condition. Please refer to the Supplementary Movie S1 for a demonstration of the replay condition and simulated transitions. 

Each condition had two types of stimulus dynamics, an SSVEP-on case, in which the stimuli were flickering at a specified frequency (F1 = 14.4 Hz, red grating; F2 = 12.0 Hz, green grating), and a nonflickering control condition, in which the gratings were not flickered. In binocular rivalry, the red and green gratings were shown separately to each eye for the duration of the session, and for stimulus rivalry, the stimuli were swapped at 3.15 Hz. In replay, the same stimulus was always shown to both eyes. Before and after each condition, we collected a 1-min baseline, in which subjects fixated the same display as in the experimental trials except that there was no grating. Instead, subjects viewed the background convergence lines, the surround contours, and the fixation with the grating replaced by a black outline square, the same size as the grating, with lines of 3° width surrounding an uncontoured area of the same luminance as the background. In these baseline conditions, in which a box was presented, the display was simply the background convergence lines, the surround contours, and the fixation without any stimulus (see Stimuli section). Each 60-s block of flicker-on and flicker-off was repeated three times for each condition in the order of stimulus rivalry, binocular rivalry, and replay. 

Subjects sat upright facing a computer screen 55 cm from a chin rest on which they rested their head during the experiment. After we outfitted the proper cap size for the subject, each electrode was filled with a conductive gel and to ensure impedance <10 kOhms. Subjects responded on a computer keyboard and were instructed to report their perceptual state in binocular rivalry by pressing with their right index finger the “j” key if they saw the red grating and the “f” key with their left index finger if they saw the green grating. Transitions between the red and green gratings were reported when less than 75% of the dominant stimulus became suppressed by pressing both the “f” and “j” keys together. Subjects were told to hold down the keys for the whole duration of the three potential perceptual states. 

Data acquisition

Data was collected on a Neuroscan SynAmps 2 setup and amplifiers with a parallel port triggering system running between the acquisition computer and the amplifier to synchronize button-press timing to the EEG data collection. Data were online filtered at 0.1–200 Hz, and the sampling rate was 1,000 Hz. Zero-phase filtering was done twice: once for a high pass at 4 Hz and once again for a low pass at 30 Hz. Both filtering operations used a windowed linear-phase finite impulse response filter design. We used a 64-channel Neuroscan Quick-Cap EEG, of recommended sizes based on manufacturer specifications after measurement of horizontal head circumference. This cap conforms to the UI10/10 system of channel names and locations (Jurcak, Tsuzuki, & Dan, 2007). Ground and reference were on the anterior and posterior central regions of electrodes. Reference was placed between Cz and CPz, and ground was placed between Fz and Fpz. The EEG signal at each electrode was collected relative to the reference electrode during the experiment. During off-line analysis, each electrode's time series was rereferenced to a common average reference. Not all 68 channels were used; scalp electrodes (62 electrodes) were used in the analysis after removing EOG, EKG, vertical EOG, horizontal EOG, mastoid 1, and mastoid 2. Finally, electrode locations were digitized with a Polhemus Fastrak digitizer relative to the nasion, right preauricular, and left preauricular anatomical locations for each subject. 

Data preprocessing

Data were preprocessed with automated custom MATAB (MathWorks, Natick, MA) scripts. Each stage of the processing pipeline is described, in order, below: 

•  
  Early preprocessing: Subject-specific bad channels, if any existed, were interpolated from surrounding four electrode time series. All data were then rereferenced to a common average reference and then underwent removal of the DC offset for each electrode.
•  
  Temporal filtering: Resulting time series underwent general high-/low-pass filtering between 4 and 30 Hz surrounding the frequency tags F1 = 14.4 Hz and F2 = 12.0 Hz.
•  
  SSVEP filtering: Each SSVEP time course was extracted with a recursive least-square (RLS) filter (Zhang et al., 2011) at the SSVEP frequencies F1 and F2. The RLS filter was set to an 834-ms moving window, and frequencies for F1 and F2 were matched to the frequency tags. The resulting coefficients were used to calculate the amplitude and phase of the SSVEPs (Tang & Norcia, 1995).
•  
  Event-related metrics: Trials were created by taking the estimated SSVEP amplitude time course and selecting a window of 2 s prior to and 2 s after the button press, each separated based on the type of percept (i.e., red or green grating). Trials were then averaged after removing the mean of each trial for each electrode.

Response distributions

We concatenated all dominance times by taking the response durations from all three completed sessions of a condition for a particular subject. We then thresholded the responses at a minimum of 20 responses; below that number, there was no reliable estimate of the distributions. If the subjects exceeded the threshold, then we fit each subjects' distribution to a gamma function. For each subject the a and b parameters were calculated and then stored for statistical testing. 

Signal-to-noise ratio (SNR)

Because harmonics of the swap frequency were close to the SSVEP frequency in stimulus rivalry, we decided to use the power spectrum from sessions with the flicker on divided by the power spectrum from sessions with the flicker off. This gave us a ratio power spectrum that isolated the neural responses generated by the flickering frequencies and, thus, avoided any swapping harmonics in stimulus rivalry. 

Statistics

All statistical t tests were independent-samples t tests unless otherwise reported. 

Rivalry index permutation

To get an idea of whether the computed rivalry indices were significant, we used a permutation procedure that utilized the nonflickering control conditions. For example, for each condition, we showed subjects either binocular rivalry, stimulus rivalry, or replay with flickering gratings and then, subsequently, the same gratings and condition but without the flicker. We then analyzed the data in exactly the same way as in the flickering condition and computed rivalry indices for each electrode, subject, and condition. We combined both of the rivalry index distributions of the flicker and nonflicker conditions across subjects for each electrode and condition and randomly sampled to generate a distribution of empirical t values. We then applied a threshold at the 95% t value after 1,000 iterations and assessed whether the observed t value was greater or less than this value. Significant values were greater than the 95% threshold and were colored in the t value topographies. 

Source localization

Source localization was conducted in a manner similar to Zhang et al. (2011) with some slight differences. Phase alignment was used on the button-press timing in order to ensure the retention of the SSVEP activity, which oscillates at a fast rate and which would otherwise be smeared by differences in button-press timing. So, for each trial and for the occipital electrode, we searched in a window of 100 ms before and after the button press to find the peak (pi/2) of the oscillation and then shifted the button-press time such that the peak corresponded to t = 0 ms. We then took the mean of all of the aligned button presses for each of the two frequencies and then took the Hilbert transform of the final average time course for each electrode. We then took the real and imaginary components of the Hilbert transformed time course and input them into a minimum norm source localization algorithm to estimate the cortical sources of activity (Hämäläinen & Ilmoniemi, 1994). We specifically localized the time point of the peak of the amplitude of the dominantly perceived SSVEP in the window during perception of the corresponding grating (e.g., the green grating frequency tag at 12 Hz during perception of the green grating). A total of 15,001 sources were used and extracted from the segmented surface of the standard Colin brain. Sources were then constrained to be perpendicular to the cortical surface. For all subjects, a standard template of electrode locations was used. Finally, to get a metric of neural competition, this same time point of the peak amplitude in the perceived SSVEP was also localized for the suppressed stimulus' frequency tag and then subtracted from the original dominant frequency tag source map. The difference source maps were then taken for each subject and normalized via z score, and then the mean values across subjects were plotted on the source map. Identified peaks of the sources corresponded to standard atlas labels of occipital pole (left and right) and middle temporal (hMT+/V5) gyri (left and right), which is in line with previous work on dipole source localization during binocular rivalry and cortical generators of SSVEPs (Di Russo et al., 2007; Zhang et al., 2011). 

Results

Rivalry stimuli and differences in stimulus rivalry and binocular rivalry response distributions

Our primary aim was to identify whether stimulus rivalry would transpire at higher levels in visual processing, and we first investigated perceptual dominance time distributions generated by the subjects' responses to the stimuli. We projected the same flickering red/green isoluminant, orthogonal gratings to each eye, only changing the presentation: separately to each eye (binocular rivalry), swapping between each eye (stimulus rivalry), or congruently one grating to both eyes (replay) (Figure 1A). We used red/green colors to enhance the SSVEP power by presumably engaging more neurons sensitive to color in the parvocellular pathway activated by a flickering stimulus (Vialatte, Maurice, Dauwels, & Cichocki, 2010). As a consequence of the added colors, the SNR of the frequency tag should be enhanced in the power spectrum. In addition, previous work indicated that stimuli differing along more than one visual dimension (e.g., frequency or contrast) can potentially enhance the occurrence of stimulus rivalry as opposed to the fast swapping percept of the two stimuli (Denison & Silver, 2012; Silver & Logothetis, 2007). Furthermore, it is believed that flickering might mask the transients associated with the swap and help promote stimulus rivalry in general (Lee & Blake, 1999). Thus, we tagged, in all conditions, a particular grating (red grating = F1 = 14.4 Hz; green grating = F2 = 12.0 Hz) with a particular frequency that stayed with that grating for the duration of the session (Figure 1B). 

Figure 1

 

Rivalry stimuli, conditions, and apparatus. (A) Time course of stimulus presentation for each condition. In replay, the same stimulus is shown to both eyes, and a transition is simulated as a wedge sweeping across the box or a fading of one color (not shown). In stimulus rivalry, the two stimuli are swapped between the eyes at an interval of 3.1 Hz. In binocular rivalry, the stimuli stay constant in each eye. Each session lasted 60 s with frequency tag on and frequency tag off with three repetitions of each condition with frequency tag on and off. (B) Rivalry stimuli used in the experiment and the apparatus used to generate binocular rivalry. The subjects viewed stimuli through a stereoscope and responded by saying they saw the green grating, red grating, or a piecemeal mixture of the two. Frequency tagging was done with a luminance modulation between high/low (on) and mean luminance (off) states.

Previous studies using orthogonal black-and-white gratings flickering at 18 Hz showed that binocular rivalry and stimulus rivalry had very similar normalized dominance time distributions that could be estimated by a gamma density function (Logothetis et al., 1996). We fit gamma distributions to each subjects' reported dominance time histograms thresholding at a minimum of 20 responses per fit (i.e., for a given subject, three sessions of 60 s of rivalry needed a minimum of 20 total responses; see Methods). The gamma distribution from the average a and b parameters across all subjects is shown in Figure 2A and B for the red and green percepts, respectively. The a parameter of the gamma function did not differ significantly between stimulus and binocular rivalry (red: p = 0.062, t = −1.898, df = 72, n = 37; green: p = 0.558, t = −0.589, df = 80, n = 41), but the effect size was 0.45 for stimulus rivalry's shape parameter being less than binocular rivalry's, indicating the peak of the distribution was slightly shifted left for the red percept (Figure 2A). The b parameter, however, showed a significant difference (red: p = 3.54e-5, t = 4.4, df = 72, n = 37; green: p = 0.0058, t = 2.84, df = 80, n = 41) with the effect size larger for both red, 1.04, and green, 0.64, indicating the width of the distributions were broader for binocular rivalry compared to stimulus rivalry (Figure 2A and B). Thus, our data suggest that periods of dominance/suppression are longer for binocular rivalry than for stimulus rivalry as supported by other studies (Bhardwaj & O'Shea, 2012; Patel, Stuit, & Blake, 2015) and highlights potential differences in neural processing between the two conditions. 

Figure 2

 

Differences in stimulus rivalry and binocular rivalry dominance distributions and frequency heat map. (A) Dominance duration histograms for the reported perception of the red grating (red: stimulus rivalry, blue: binocular rivalry). (B) Dominance duration histograms for the reported perception of the green grating (line color same as panel B). (C) Proportion of the trial with either a red or green grating dominant in perception as opposed to a mixed percept. Each box represents a particular combination of the flickering frequency for stimulus 1 (frequency 1) and stimulus 2 (frequency 2). Left panel shows the proportion of the trial for stimulus rivalry, and right panel shows the proportion for binocular rivalry.

Finally, to ensure our frequency tag selection does not change the proportion of dominance distributions, we conducted a pilot test on five subjects in which we chose a series of frequency tag combinations between the two stimuli and measured the proportion of dominance time in either binocular or stimulus rivalry. This is a behavioral measure of the overall clarity of rivalry with longer proportions of mixed percepts indicative of less clear rivalry and longer red/green proportions indicative of stable rivalry. As can be seen in Figure 2C, the proportion of stable percepts in binocular rivalry varied minimally as a function of frequency tag combination, and the same was observed for stimulus rivalry (n = 5 subjects). Thus, we chose frequencies F1 = 14.4 Hz and F2 = 12.0 Hz to tag each of the stimuli in the subsequent EEG experiments because they minimized subjective flicker and gave reasonably high proportions of stable percepts in both types of rivalry. 

Power of SSVEP differs in strength between the tag frequencies

We began the investigation of the electrophysiological differences between the types of competition by looking at the power spectra calculated over a given session. This would tell us any prolonged changes in neural response to the flicker that were present during a session of binocular rivalry, stimulus rivalry, or replay. We calculated the power spectra when the stimuli were flickering and divided each corresponding frequency's power by the power spectra calculated during sessions without the flicker, giving a ratio power spectrum. This spectrum isolated electrophysiological changes elicited specifically by the flickering stimuli for each condition and removed any intrinsic oscillations or swap harmonics (particularly for stimulus rivalry). 

For all completed binocular rivalry sessions grand averaged across subjects (n = 26), the power spectrum for an occipital electrode (Oz) was analyzed because previous studies have suggested reliable SSVEP power at this or nearby electrodes (Brown & Norcia, 1997; Zhang et al., 2011). The power spectrum revealed the presence of both SSVEP frequencies F1 = 14.4 Hz and F2 = 12.0 Hz and the first harmonic; however, the 12.0-Hz SSVEP frequency was higher on average than the 14.4-Hz SSVEP (Figure 3A). For replay, SSVEP power at both frequencies was also present in the spectrum (Figure 3B) although the 12.0-Hz frequency was now smaller in magnitude compared to binocular rivalry, and the 14.4-Hz frequency stayed unchanged. To evaluate whether this difference was significant, we searched each subject's time series for the electrode with the maximum peak and SNR values for the frequency tag frequencies in the power spectrum. This accounted for any variability across subjects' cap positioning or underlying anatomical differences that could change the topography of the SSVEP power spectrum. Comparing the maximum SNR values for replay and binocular rivalry showed that the red stimulus frequency tag at 14.4 Hz was not significantly different in replay and binocular rivalry (p = 0.1264 t = 1.5581, df = 44); however, the green stimulus frequency tag at 12.0 Hz showed a larger power during binocular rivalry (p = 0.0041, t = 3.0254, df = 44). This suggests that the 12.0-Hz frequency tag was selectively enhanced in the presence of interocular competition as opposed to binocular integration (see Discussion). 

Figure 3

 

Power spectra of SSVEP differs in strength between the tag frequencies for binocular rivalry and stimulus rivalry. (A) The average SSVEP power spectra calculated across n = 26 subjects. Spectra with flicker on were divided by the power observed in a control nonflickering condition. Average of three sessions of stimulus rivalry each lasting 60 s. (B) Same as panel A except for binocular rivalry. (C) Same as panel B except for replay.

Finally, we compared the power spectra of binocular rivalry and stimulus rivalry at the same electrode across subjects to assess whether the neural response would change depending on the type of visual competition. We hypothesized a reduction in SNR in stimulus rivalry if the pattern-level competition is more engaged. As can be seen in Figure 3C, the magnitude of the power spectrum was larger for binocular rivalry for the 12.0-Hz frequency tag but not the 14.4-Hz frequency tag (red: p = 0.3183, t = 1.0095, df = 44; green: p = 0.0034, t = 3.099, df = 44). Additionally, we noticed the stimulus power was better matched for both frequencies in stimulus rivalry, indicating that swapping between the eyes accounts for any eye-specific preferences of each frequency–stimulus pair. Thus, binocular rivalry showed an enhancement of the 12.0-Hz frequency tag when compared to stimulus rivalry in addition to the same enhancement seen when compared with the replay condition. Overall, this may be due to eye dominance because the stimuli stay presented to one eye for the duration of the binocular rivalry session. Nevertheless, the power spectra confirm the presence of SSVEP signals generated by the flicker in all conditions. 

Spatial topography of the power spectra generated by flicker localize to occipital regions in sensor space

To assess how the spatial distribution of the SSVEP power changes across conditions, we then looked at scalp maps of the power at the flicker frequencies. This could tell us if there were regional differences in sensitivity at the level of the scalp to the frequency tag and also if there were regional changes of activation across conditions. As can be seen in Figure 4, the topographical distribution of SSVEP SNR at the frequency tag frequencies (power spectra peaks divided by the surrounding noise frequencies ±0.5–1 Hz) calculated on the ratio spectrum (as in Figure 3, taking the ratio of flicker and nonflicker conditions), showed an occipital source for all conditions. Furthermore, this source overlapped for both frequency tags in all conditions in the occipital pole, suggesting both frequency tags were processed in the same area. Thus, flickering stimuli generated a reliable signal at the tagged frequencies in all conditions, which localized to the same occipital patch across the scalp, highlighting a similar mechanism of neural processing between stimulus and binocular rivalry. 

Figure 4

 

Spatial topography of the power spectra generated by flicker localized to occipital regions in sensor space. (A) SNR topography of each condition (rows) and each stimulus' frequency tag (columns) for n = 26 subjects. SNR was computed on the ratio power spectrum (see text) as the peak divided by the surrounding 1-Hz frequency bins. SNR topography for stimulus rivalry. (B) Same as panel A except for binocular rivalry sessions. (C) Same as panel A except for replay sessions.

Rivalry index topographies show significant modulation in occipital cortex

The power spectrum indicated that, across the duration of a session, there is little difference in the topographies of binocular rivalry and stimulus rivalry. We then hypothesized that if the competition was between the patterns in stimulus rivalry, then signatures of high-level neural competition might instead be identified at local time points during a session. We, therefore, extracted the time course of modulations of the SSVEP amplitude for the two frequency tags during both stimulus rivalry and binocular rivalry. Previous studies have shown that during binocular rivalry the frequency tag amplitudes show a counterphase relationship with each other, meaning that during the perception of the red grating the SSVEP amplitude is high for the red frequency tag and low for the green grating's frequency tag, and the trend is reversed when the green grating is perceived (Brown & Norcia, 1997; Zhang et al., 2011). This counterphase behavior is primarily localized around occipital areas for binocular rivalry and replay and is assumed to be a signature of the competitive neural interactions (Brown & Norcia, 1997; Zhang et al., 2011). 

We tested the hypothesis that stimulus rivalry would show a similar counterphase relationship between the SSVEP signals but among pattern-level representations in higher level brain regions (Leopold & Logothetis, 1999). We quantified the counterphase behavior by computing a rivalry index, which takes the sum of the absolute differences between the amplitudes of both SSVEP frequencies over the time window around a button press (Figure 5A). We assessed statistical significance by computing a rivalry index independently for each electrode and then used permutation statistics to evaluate empirical distributions from the null hypothesis observed during the nonflickering conditions. 

Figure 5

 

Rivalry index topographies show significant modulation in occipital cortex. (A) Visual of the rivalry index computation in which we sum the absolute difference between the amplitude of two SSVEP signals, effectively measuring the area of counterphase modulation. Windows of 2 s before and after were used around each button press for a stabilized percept. (B) For stimulus rivalry, the rivalry index for each electrode in SSVEP and nonflickering control conditions were combined, and a 99% confidence interval was computed based on random sampling of a subset of rivalry indices from the new distribution. The red dots are significant rivalry indices computed for a particular electrode. Bottom panel shows the t values in one dimension with electrodes spaced equally on the x-axis. Blue lines correspond to the 95% confidence interval of the bootstrap. (C) Same as in panel B except for binocular rivalry. (D) Same as in panels B and C except for replay.

We plotted the observed t values across the topography of the scalp for each electrode and highlighted the significant (p < 0.05) p values with a red dot (Figure 5B and C). We found that the rivalry indices were significant only in an occipital region of the scalp topography (p < 0.05) in all conditions. Thus, contrary to our hypothesis, these results suggest that both binocular and stimulus rivalry share dynamic SSVEP-based competitive neural interactions in the early occipital areas at local time points based on the perceptual state. 

Time course of significantly modulated regions show equivalent depth of modulation

We then checked whether the competitive neural interactions are modulated to the same extent for the two different types of rivalry, which would give us a measure of the strength of suppression in the occipital area. The depth of counterphase modulation was measured by the peak-to-trough distance of the counterphase SSVEP signals averaged across all occipital electrodes. We measured the peak-to-trough distance for the two signals' SSVEP amplitudes for either the red or the green percept within the 2-s peri-time window around the button press. We found that in the occipital electrode the depth of modulation in stimulus rivalry was not significantly different from that of binocular rivalry or replay as seen with the corresponding 95% confidence intervals (Figure 6). Overall, this result suggests that the early visual areas have the same level of neural suppression in all conditions, indicating the mechanism of competition engages occipital areas to the same degree. 

Figure 6

 

Time course of significantly modulated regions show similar depth of modulation for binocular and stimulus rivalry. (A) Peak-to-trough distance was calculated by finding the maximum and minimum values of the average amplitude of the SSVEP over a subset of occipital electrodes. Electrodes were chosen as the occipital electrodes significantly modulated as found in the previous section in terms of rivalry indices. Bars indicate 95% confidence intervals calculated across subjects. (B) Same as in panel A except for the green grating percept.

Source localization on aligned peaks of rivalry time course

To get a more precise spatial measure of the sources accounting for the modulations in SSVEP amplitude and account for any effects of volume conduction on the scalp potentials, the competing SSVEP signals in each condition were localized in the source space. We identified five subjects with clear counterphase modulation of the SSVEP amplitudes during binocular and stimulus rivalry and localized the SSVEPs at the time point of the peak of the counterphase modulation (see Methods). This corresponds to the time point of maximum ocular suppression and could identify any underlying differences in suppressive mechanism between stimulus and binocular rivalry. To take into account the neural competition, we also took the difference in the two SSVEP frequency tag source topographies. Taking the difference in the SSVEP maps generated by Freq(percieved) − Freq(unperceived), it can be seen that the average topography was not different for stimulus rivalry and binocular rivalry (Figure 7). The anatomical label for the peak activations of both conditions corresponded to the right and left occipital poles and an additional source in hMT+/V5 (middle temporal cortex). Thus, source analysis corresponded with the power spectra and rivalry index topographies and suggests further that neural competition during stimulus and binocular rivalry share a similar substrate in the early occipital cortex. 

Figure 7

 

Source localization on aligned peaks of rivalry amplitude time course. (A) Schematic of the SSVEP amplitudes used to find the time point of peak in competition measured as the maximum of difference between SSVEPs after the button press (t = 0). This time point was used in source localization. (B) The difference in the two competing SSVEPs was localized on a standard brain for the green (top) and red grating percept (bottom) during binocular rivalry. Subjects (n = 5) with clear counterphase amplitude modulation during either stimulus rivalry or binocular rivalry were selected for source localization. SSVEP signals were phase-aligned and then localized to a standard brain with standard electrode positions common for all subjects. Right panel shows the same source localization procedure but for stimulus rivalry.

Discussion

We used EEG frequency tagging to give us SSVEP signals that tracked the time course of neural competition between stimuli in binocular and stimulus rivalry. We evaluated a model of rivalry that posits competition for stimulus rivalry occurs at the level of stimulus representations in extrastriate regions (e.g., V4 or IT) and found that the SSVEPs instead colocalized in the occipital cortex. This was observed for the spectral power, which is sensitive to prolonged changes in neural response; the rivalry index, which quantifies competitive neural processes relative to button-press time windows; and source analysis, which was performed at the peak in neural competition. Overall, these results suggest an early mechanism for stimulus rivalry that is centered in occipital cortex, similar to conclusions from previous research (Bhardwaj & O'Shea, 2012; Brascamp et al., 2013; Buckthought et al., 2015). 

Differences in dominance distributions of binocular and stimulus rivalry

In this study, we found a difference in behavioral dominance distributions between binocular rivalry and stimulus rivalry (Figure 1B and C). Previous reports have suggested that binocular and stimulus rivalry have similar distributions that both conform to a gamma function; however, it was unclear whether absolute durations were different given that distributions were normalized (Logothetis et al., 1996). Other studies looking at binocular and stimulus rivalry found similar gamma distributions between the two conditions, but stimulus rivalry dominance durations were typically shorter than binocular rivalry (Patel et al., 2015). In our study, we also found that across many subjects (n = 40) a significant decrease in the shape parameter of the gamma function for stimulus rivalry without normalizing the distribution. Overall, our results are consistent with previous results (Bhardwaj & O'Shea, 2012; Patel et al., 2015) and emphasize that in stimulus rivalry the dominance times seem to be shorter than for binocular rivalry. 

Increase of 12.0-Hz frequency tag in binocular rivalry power spectrum

We observed a significant increase in the 12.0-Hz frequency tag in the power spectrum for binocular rivalry compared to stimulus rivalry and replay. One potential reason is that the 12.0-Hz frequency tag was presented for a shorter duration in the replay condition because one frequency is present at a given moment in that condition. If it was an effect of overall presentation duration, we would expect to see the 14.4-Hz frequency tag reduced as well, which was not the case. Another reason could be that the red stimulus proved to be more salient in replay than when it was engaged with a competing stimulus in binocular rivalry. However, the behavioral dominance distributions for perception of red and green are similar for binocular rivalry (Figure 2A and B), and the physical characteristics of the stimuli are matched between conditions (see Methods), so it is unlikely to be an inherent stimulus saliency difference that could explain the findings. Another explanation may be that eye dominance enhanced the 12.0-Hz frequency tag because the green grating (flickering at 12.0 Hz) was always presented to the left eye during binocular rivalry (Ehrenstein, Arnold-Schulz-Gahmen, & Jaschinski, 2005). This is in line with the power spectrum data from stimulus rivalry, which showed that when the two stimuli are swapped between the eyes the 12-Hz power enhancement goes away and the power at both tagged frequencies is equal. Our speculation is that in the replay condition, because it always followed the binocular rivalry condition, the left eye neurons adapted to the 12.0-Hz flicker and, therefore, gave a weaker response than the 14.4-Hz neurons, which did not adapt in the left eye. Thus, when the 14.4-Hz frequency tag was presented in the left eye in replay, it could generate a larger response than the 12.0-Hz frequency tag because the neurons had not adapted as much in the dominant eye. 

Overlap of the power spectrum of binocular and stimulus rivalry

We used a ratio power spectrum to reliably estimate the power of the SSVEP SNR by dividing the power calculated from sessions with flickering stimuli by the power calculated from sessions with the stimuli not flickering. This reduced any intrinsic oscillatory activity from rivalry or other task-related oscillations and enhanced the SSVEP power at the tagged frequencies. From the power topographies, it was apparent that most of the SSVEP activity generated by the flicker was centralized around occipital areas, and it was localized to the same areas for binocular and stimulus rivalry. Previous studies also showed that SSVEPs are localized in occipital areas for rivalry stimuli (Zhang et al., 2011), but our results suggest that swapping the stimuli between the eyes at a rapid rate (3 Hz) activates the SSVEPs in the occipital cortex in a similar manner as when they are not swapped. Our interpretation is that monocular channels are unable to switch their inhibition at 333 ms, and thus, binocular neurons integrating information across both eyes are the site of competition for rivalry. However, both monocular and binocular neurons overlap in the occipital pole of the early visual cortex, and additional studies may be needed to address this exact mechanism. 

We should also point out that there is a difference between the locus of competition and the neurons perhaps modulating or driving that competition. One possibility is that the conflict between the images is registered in high-level regions, and these regions then send feedback projections to modulate early visual areas to resolve the conflict. We cannot rule out the possibility of the feedback models or potentially a combination of feedforward and feedback models from our SSVEP experiments. It is worth noting that multiple regions are involved during the rivalry process, and being a multistage process, there are interactions between the visual regions; however, we emphasize from our observations that the early visual areas are strongly modulated with perceptual transitions in rivalry. 

We did not observe intermodulation components in the power spectra across conditions (i.e., F1 = 14.4, F2 = 12.0, and Fswap = 3.15 and their intermodulation terms of F1 + F2, F1 − F2, F1 + Fswap, F2 − Fswap, etc.). However, a purely monocular signal should generate intermodulation terms that are a product of Fswap and either SSVEP frequency tag F1 or F2 (Supplementary Figure S1). Therefore, we interpret the lack of intermodulation components as suggesting a binocular site for rivalry given that a purely monocular signal should give intermodulation components at Fswap and F1 or F2 in stimulus rivalry (as called to our attention by an anonymous reviewer). Thus, binocular neurons in early visual areas may be a candidate source for neural competition in both binocular and stimulus rivalry (O'Shea, 1998). 

Rivalry index colocalizes to the occipital cortex for both stimulus and binocular rivalry

Finally, our metric of neural competition, the rivalry index, was based on the counterphase activity of the SSVEP amplitude of the two competing frequency tags during rivalry. We found an occipital colocalization of the rivalry index calculated across the scalp in all conditions. Source localization on the SSVEPs at the peak of the counterphase modulation showed that sources generating the SSVEPs originated in the occipital pole and in hMT+/V5. The finding of SSVEP modulation in the occipital pole and hMT+/V5 is similar to previous studies of binocular rivalry in EEG (Zhang et al., 2011) and magnetoencephalography (MEG; Srinivasan, Russell, Edelman, & Tononi, 1999) and also consistent with previously identified cortical generators of SSVEPs (Di Russo et al., 2007; Norcia, Appelbaum, Ales, Cottereau, & Rossion, 2015). 

Given that the eye swapping in stimulus rivalry is thought to bypass eye-specific channels, it was expected that the rivalry index for stimulus rivalry would not be high in early visual areas (Logothetis et al., 1996; Tong et al., 2006). Contrary to this hypothesis, however, we found that the rivalry index and source localization results showed competitive signatures in the occipital pole, a cortical region where early binocular channels reside. This suggests that part of the mechanism of stimulus rivalry may incorporate early binocular channels. Previous fMRI studies suggest largely overlapping cortical networks when comparing binocular rivalry and stimulus rivalry (Buckthought et al., 2015), and behavioral studies of stimulus rivalry suggest that it may still incorporate early visual interactions (Bhardwaj & O'Shea, 2012; Brascamp et al., 2013). Our results are consistent with those findings and further illustrate an electrophysiological substrate for neural competition in stimulus rivalry not completely inconsistent with an early binocular-based mechanism. 

Although previous results at lower frequencies around 6 and 7 Hz showed similar localization in occipital areas (Brown & Norcia, 1997; Zhang et al., 2011), the choice of frequency tag can potentially influence the localization given that temporal tuning of visual neurons changes along the visual hierarchy (Gauthier, Eger, Hesselmann, Giraud, & Kleinschmidt, 2012). In particular, it may be that higher areas show a modulation with stimulus rivalry but fail to be captured by the 12.0- and 14.4-Hz frequency tags. However, previous studies using higher frequency tags (11 and 12 Hz) showed robust modulations in posterior parietal, lateral temporal, and prefrontal regions during rivalry generated by half-image binding across visual hemifields (Sutoyo & Srinivasan, 2009). An MEG SSVEP study of rivalry also showed power increases across sensors that covered the temporal and frontal electrodes (Srinivasan et al., 1999). We should point out that we did see some level of SSVEP activity in sensors that were around higher level areas, but this activity was much less than that of the early visual areas. 

Future studies could more systematically address whether frequency tagging at lower frequencies (e.g., below 6 Hz) could target high-level visual areas more robustly and whether these signals would show competitive signatures in stimulus rivalry. This is potentially interesting within the domain of bistable images, which are not generated through interocular conflict but nevertheless utilize early visual cortex. Monocular rivalry, for example, has been shown to activate largely similar visual networks to binocular rivalry with an emphasis on early visual areas (Buckthought, Jessula, & Mendola, 2011). Necker cube reversals are also associated with right inferior parietal activity (Britz, Landis, & Michel, 2009), activity that, although not captured with frequency tagging, may nevertheless be important in modulating perceptual transitions in rivalry. 

We should note that we cannot rule out the effects of attention on influencing part of the rivalry index modulations seen in either stimulus or binocular rivalry. Previous reports indicate that in the absence of attention the rivalry index drops dramatically for binocular rivalry (Zhang et al., 2011), so it is possible that attention may also impact the counterphase modulation for stimulus rivalry. It may do so in a biasing fashion similar to the drive in finding a perceptually consistent interpretation, which involves attentional feedback from parietal areas, as hypothesized to be the case in motion-induced blindness (Davies, 2017). Further studies could examine the timing of attentional effects in generating counterphase modulations of the SSVEP signals during rivalry. 

Overall, these results suggest that binocular and stimulus rivalry show similar neural signatures in the topography of the frequency tag power spectrum, time course of SSVEP amplitude modulations, and source localization topographies. This suggests that part of the mechanism of stimulus rivalry may incorporate early visual cortical mechanisms to resolve the visual conflict, and may be more similar to binocular rivalry than initially hypothesized. 

Acknowledgments

This work was supported in part by NIH EY023101 and NSF DGE-1069104. 

Commercial relationships: none. 

Corresponding author: Sheng He. 

Email: . 

Address: Department of Psychology University of Minnesota Minneapolis, MN, USA.