In Experiment 1, subjects viewed rivaling, frequency-tagged patterns, while we measured EEG signals at both fundamental and intermodulation frequencies. The two eyes' inputs flickered at different frequencies (4.72 and 8.5 Hz), which we label
f1 and
f2. Figure 2A (upper) shows a spectral plot of the EEG response for a single representative subject at the occipital electrodes O
Z, O
1, and O
2.
Figure 2A (lower) shows the average spectrum for all subjects. The response was maximal at the fundamental frequencies
f1 and
f2, with smaller responses at their second harmonics 2*
f1 and 2*
f2. Some previous SSVEP studies have used contrast-reversing stimuli, which produce the largest responses at the second harmonic (Brown & Norcia,
1997). Our stimuli alternated with a mean field (“on-off” flicker), and the dominance of the “on” response produced substantial energy at
f1 and
f2.
The spectra also show peaks at the
f2–
f1 and
f2+
f1 intermodulation frequencies, with higher energy at
f2–
f1 than
f2+
f1. Higher-order intermodulation frequencies (e.g., 3*
f1–
f2, 2*
f2–
f1) could be seen in some but not all subjects; they are not reported in subsequent results but when analyzed showed similar effects to the ones reported below.
Figure 2C shows the averaged topographies of the fundamental (upper) and intermodulation (lower) frequencies. Both sets of frequencies were observed predominantly at the occipital and parieto-occipital electrodes with peaks near O
Z, indicating that they emerged from early visual cortex.
The fundamental frequencies reflected each eye's perceptual dominance and suppression.
Figure 3A,
B shows
f1 and
f2 signals averaged around button responses that indicated “normal” rivalry transitions, in which first one eye then the other is relatively completely suppressed. Because subjects reported the state of the nonsuppressed eye, we term such events
dominance-to-dominance transitions (reported transition at 0 s; error bars, ±1 standard error of the mean;
N = 8). Because the time courses had their means and very low frequency trends removed (see the
Methods section), they are centered around 0 on the
y-axis. As in previous results (Brown & Norcia,
1997; Zhang et al.,
2011), when subjects reported perceptual dominance transitioning from the left to right eye (
Figure 3A), energy at the frequency of the left eye's stimulus (
f2, red) decreased, and energy at the right eye's frequency (
f1, blue) increased. The opposite pattern of response was observed for transitions from right to left eye (
Figure 3B).
The curves in
Figures 3A and
B crossed each other about 690 ms before subjects pressed the button to indicate that their percept had changed. This difference likely reflects the time required to make a perceptual decision and generate and execute the button press. Previous SSVEP studies have not reported this number (Brown & Norcia,
1997), and the latency is slightly longer than estimates based on other techniques (∼500 ms; Alais, Cass, O'Shea, & Blake,
2010; Einhäuser, Stout, Koch, & Carter,
2008). This difference may be due to greater perceptual and decision uncertainty in our work, owing to the relatively large number of ambiguous, “mixed” percepts.
Indeed, our stimulus possessed various features, such as a large aperture (∼16° diameter), nonchromatic checks, and rapid flicker that particularly prolonged mixed percepts, in which neither eye reaches complete dominance (Blake et al.,
1992; Kovács, Papathomas, Yang, & Fehér,
1996).
Figure 3C,
D show fundamental frequency envelopes as subjects reported their perception transitioning from such mixed percepts to a rivalry percept in which one eye is dominant. As in dominance-to-dominance transitions, the frequency envelope corresponding to the right eye (
f1, blue) increased as subjects went from a mixed percept to right eye–dominant percept (
Figure 3C). The opposite pattern was observed as subjects transitioned from mixed to the left eye dominance, but with a lower magnitude (
Figure 3D). The diverging point between the two frequencies as the subjects transitioned from a mixed to dominance percept occurred 1.25 s before the button press (
t = 0).
Figures 3E,
F plot the envelopes for dominance to mixed transitions. The stimulus frequencies that corresponded to the dominant eyes had a larger response as compared with the suppressed eyes about 3 s prior to button press. At about 1.25 s prior to button press, the two frequency responses converged, reflecting the subjects' transition into mixed percepts.
Figure 4 plots the intermodulation frequencies around perceptual transitions, averaged for the two largest intermodulation terms,
f2–
f1 and
f2+
f1. For the transitions from a mixed perceptual state to normal rivalry in which the image from one eye is dominant (
Figure 4A), the intermodulation signals peaked at about 2.2 s before the button press and then dropped below baseline as rivalry dominance restarted. The pattern of intermodulation energy was almost identical for transitions to each of the two eyes' dominance (dotted blue and red lines). The decrease in signal amplitude was completed by 1.1 s before the button press.
The decrease in intermodulation frequencies during transitions from mixed to dominance is consistent with the idea that they arise from neurons that are active during resolution of interocular conflict. To evaluate the statistical significance of the decrease, we performed
t tests between two 1-s epochs starting at 3 s and 1 s before the reported transition. There was a marked decrease in the intermodulation signal between these epochs (
p < 0.002;
Figure 4A).
The intermodulation signal at the dominance-to-dominance transitions (
Figure 4B) did not follow a consistent trend. It is likely that these transitions were too rapid to produce measurable signals in the EEG (see
Discussion). At dominance-to-mixed transitions (
Figure 4C), only inconsistent trends were seen in the intermodulation terms. This suggests that conflict is not signaled strongly until some time into the mixed percept, occurring at variable durations after the transition. It is possible that the intermodulation signals arise from neurons active only immediately before the perceptual conflict is resolved. The trend for a late increase (4-s posttransition) may reflect a rise preceding the subsequent mixed-to-dominance transition.
Although generally supporting the presence of neurons that signal conflict, our results are also consistent with an alternative hypothesis. During mixed percepts, neurons receiving input from each eye are active, and so the intermodulation signals may have been produced by other neurons that sum or integrate information from the two eyes. These may have been more active during mixed percepts than during rivalry, in which one eye was suppressed.
In Experiment 2, we attempted to rule out this alternative, by parametrically increasing interocular conflict while simultaneously decreasing total stimulus contrast. Neurons that integrate information from the two eyes should reduce their activity as stimulus contrast decreases (and conflict increases). Neurons that signal interocular conflict should show the opposite pattern.
We presented plaid patterns in which the contrast of a different one of the two component gratings was reduced in each eye. The contrast reduction induced interocular conflict, and the parameter controlling it was termed the
IOCD (see
Methods). Flickering plaid patterns with each IOCD were presented for 50 s at a time, with 3–10 presentations per IOCD for each subject.
For all four IOCD conditions, subjects reported which eye was dominant or whether they saw a mixed or a fused percept. The percentage of time spent in dominance increased with IOCD (
Figure 5), with some rivalry present at each level. During fusion (IOCD of 0), the residual dominance periods were due to monocular pattern rivalry (Wade,
1975). For the highest IOCD, clear rivalry alternations were perceptually reported and were also visible in the counterphase modulation of the EEG fundamental frequency envelopes (as in
Figure 3; data not shown).
Figure 6A plots the averaged power of the two fundamental frequencies,
f1 and
f2, and the intermodulation frequencies,
f2–
f1 and
f2+
f1, as a function of IOCD. The power of the fundamental frequencies fell monotonically across conditions. This pattern suggests that those frequencies reflected activity in neurons whose response was determined by stimulus contrast.
The intermodulation signals, however, showed an inverted-U shape as a function of increasing interocular conflict, parameterized by IOCD. This result is consistent with the hypothesis that a subset of neurons responsive to conflict increased their activity as IOCD increased, causing the intermodulation signal to rise initially. At higher IOCDs, however, this rise was apparently outweighed by the overall reduction of population activity with decreasing stimulus contrast, which caused the curve to decline (see below).
To test whether the intermodulation frequencies showed a different pattern than the fundamental frequencies, we performed a 2 × 2 repeated-measures analysis of variance across subjects with factors frequency type (two levels: fundamental and intermodulation frequencies) and IOCD (four levels). There was a significant main effect of IOCD (p < 10−10), which is clearly evident in the decrease in FFT power with overall stimulus contrast. There was also a significant interaction between frequency type and IOCD (p < 0.019), showing that the fundamental and intermodulation frequencies had different patterns as a function of IOCD (i.e., monotonic decrease vs; inverted-U, respectively). To test specifically for effects at the first two IOCD levels, we conducted a t test between the ratio of FFT power of the averaged intermodulation frequencies to the averaged fundamental frequencies at IOCDs 0 and 0.2. This difference was significant (p < 0.04).
In addition to the parametric tests, we also formally tested the inverted-U shape of the intermodulation curve by fitting quadratic functions to the FFT power versus IOCD curve across 10,000 bootstrapped averages (resampling with replacement across subjects). These fits were used to estimate the IOCD at which the power was maximal in the interval 0–0.8. The location of the peak response for the intermodulation frequencies had a median between IOCD of 0.1 and 0.2. The fundamental frequencies' median peak fell at 0, and the difference in peak location between the two sets of frequencies was highly reliable (p < 0.005).
The shape of the curves suggests that the intermodulation signals contain contributions from both binocular integration neurons, whose response depends on total stimulus contrast, and interocular conflict neurons, whose response increases with IOCD, with each type possessing a saturating output function. If this were the case, then the small reduction in contrast as IOCD increases from 0 to 0.2 would be expected to change the response of integrating neurons only very little, as the high contrasts at those IOCDs place them at a flat part of their response curve. The fact that the fundamental signals are relatively flat across low values of IOCD is consistent with this account. However, the increase in IOCD from 0 to 0.2 may have had a relatively large effect on neurons that signal conflict, as this change would be in the steep portion of their output function. Combining signals from the two types of neurons would thus yield a prediction of increasing intermodulation signals from IOCD 0 to 0.2. At higher values, the effects of IOCD should reverse, with the response of integrating neurons falling rapidly and responses of conflict detecting neurons increasing only a little. Combining signals from the two types of neurons would then yield a function that decreases from IOCD 0.2 to 1.0.
We simulated such a model using two neural populations, one that increased its response with the overall stimulus contrast (binocular summation/integration neurons),
Ri, and another that increased its response with increasing interocular conflict,
Rc. Response saturation was implemented using a normalization nonlinearity (e.g., Albrecht & Hamilton,
1982; Carandini, Heeger, & Movshon,
1997; Heeger,
1992) according to the equation
, where
k indexes over the two types of neurons (
σi,
σc) determined the saturation point of each curve, and
Ai,
Ac, determined the amplitude of each curve.
Ci was the total contrast of the two gratings summed across both plaids (2, 1.8, 1.6, 1.2), and
Cc was the contrast difference between the two gratings summed across both plaids (0, 0.4, 0.8, 1.6). Using a grid search, we found many values of saturation constants and amplitudes for which the model showed an inverted-U shape behavior, similar to what was observed in our data. One example is plotted in
Figure 7.