We make predictions about eye of origin of visual input: Visual mismatch negativity from binocular rivalry

Abstract

The visual mismatch negativity (vMMN) is a negative component of event-related potentials (ERPs). It occurs when an infrequent visual stimulus, a deviant, is randomly and unpredictably presented in a sequence of frequent visual stimuli, the standards, and is thought to reflect prediction and prediction error of visual input. We investigated the sensitivity of vMMN to eye of origin (utrocular) information as well as to orientation information. We presented 80% of binocular rivalry standards (one grating to one eye and an identical, orthogonally oriented grating to the other eye), and 20% of deviants, either by swapping the gratings between the eyes to change the eye of origin of the gratings (an eye-swap deviant) or by rotating the gratings by 45° to change the orientation of the gratings (an orientation deviant). We found an orientation vMMN that was maximal at about 250 ms and an eye-swap vMMN that was maximal at about 380 ms. We also found deviance-related activity to both sorts of stimuli earlier than is traditionally defined as a vMMN. We used standardized low-resolution brain electromagnetic tomography (sLORETA) to localize each vMMN component and found similar sources for both vMMNs in occipital and frontal areas of the brain but differences in parietal and temporal areas. We conclude that eye of origin information can be used to elicit vMMN, that eye-swap vMMN is different to orientation vMMN, and that vMMN can be generated from information of which observers are unaware.

Introduction

When an unexpected visual event occurs in our field of view, there is a sudden change in the electrical activity of our brains as measured from electrodes on the scalp (electroencephalography, EEG), called the visual mismatch negativity (vMMN; e.g., Kimura, Ohira, & Schröger, 2010). A negative component of event-related potentials (ERPs), vMMN has a posterior scalp distribution and occurs between 150 and 400 ms after stimulus onset. Also, vMMN is typically observed in the oddball paradigm: an infrequent visual stimulus—a deviant—is randomly and unpredictably presented in a sequence of more frequent visual stimuli—the standards. vMMN is seen most clearly in the difference wave: the ERP for the deviant minus the ERP for the standard (for reviews, see Czigler, 2007; Kimura, 2012; Kimura, Schröger, & Czigler, 2011; Pazo-Alvarez, Cadaveira, & Amenedo, 2003; Stefanics, Kremláček, & Czigler, 2014). 

The current consensus explanation of vMMN is predictive coding (Kimura et al., 2011; Stefanics et al., 2014; Winkler & Czigler, 2012; for auditory MMN, see Garrido, Kilner, Stephan, & Friston, 2009; Wacongne, Changeux, & Dehaene, 2012; Winkler, 2007). According to predictive coding, our brains construct a model of the regularities in our visual environment (i.e., the standards) and when this model is violated by an unexpected visual event (i.e., a deviant), extra processing is required to update the model (Friston, 2005; Rao & Ballard, 1999). 

Any unexpected change in visual input can elicit vMMN. So far, it has been observed in response to deviants for low-level features of vision such as color, direction of motion, orientation, spatial frequency, location, shape, size, and contrast/luminance. vMMN has also been observed in response to deviants for high-level features of vision, such as human facial expressions, gender of human faces, lateralization of human hands, and bilateral symmetry (for a thorough, up-to-date list of stimulus features that elicit vMMN, see Czigler, 2014; Kimura, 2012; Stefanics et al., 2014). We set out to confirm a new, low-level, visual stimulus for eliciting vMMN: eye of origin (utrocular) information. 

To vary eye of origin, we presented standards comprising a grating to one eye and an identical, orthogonally oriented grating to the other eye, and deviants in which the gratings were swapped between the eyes (we call this an eye-swap deviant). That is, the orientation information presented to the two eyes is identical for standards and eye-swap deviants; the only difference is the eye of origin of the orientations. We also had a control condition for which we expected to find a conventional vMMN: we showed deviants comprising gratings rotated in both eyes by 45° (we call this an orientation deviant). We expected to find a vMMN for orientation deviants, because unexpected changes in orientation information are known to elicit vMMN (Astikainen, Lillstrang, & Ruusuvirta, 2008; Astikainen, Ruusuvirta, Wikgren, & Korhonen, 2004; Czigler & Pató, 2009; Kimura, Katayama, Ohira, & Schröger, 2009; Sulykos, Kecskés-Kovács, & Czigler, 2013; Takács, Sulykos, Czigler, Barkaszi, & Balázs, 2013). 

Why is it interesting and important that vMMN could occur to unexpected changes in eye of origin information? Eye of origin is an important aspect of most stimuli we look at every day, and it is vitally important to us for telling how close things are to us, known as depth perception. For example, a hungry lion behind a thick sheet of glass projects images into each eye that if swapped between the eyes would show the lion as dangerously in front of the glass (Howard & Rogers, 2012; Wheatstone, 1838). From a functional perspective, it makes sense to process such information without requiring visual consciousness, because this would take time away from our escaping the lion in the latter case. This prompts us to pose two questions about the vMMN to eye of origin: 

1.  
   Is conscious experience of the standards and deviants necessary for vMMN to be generated? There is a clear difference in conscious experience from typical oddball sequences, for example, with standards in which both eyes view a grating of one orientation and deviants in which those gratings are rotated by 90°, and with the ones we used. Showing a grating to one eye and an identical, orthogonally oriented grating to the other eye yields the experience of binocular rivalry (Breese, 1899; Wheatstone, 1838; for recent reviews, see Alais & Blake, 2004; Blake & O'Shea, 2009; Miller, 2013). During binocular rivalry, perception alternates back and forth such that at any given moment an observer sees only the grating presented to one eye, only the grating presented to the other eye, or some dynamic, patchy combination of both gratings. These percepts each last for a second or so, although the durations are highly variable both within and between individuals, and the progression from exclusive visibility of one grating to the other is random (Levelt, 1967; Walker, 1975; Zhou, Gao, White, Merk, & Yao, 2004). Furthermore, when the eyes view dissimilar gratings, an observer cannot tell which eye is viewing which grating, a failure of utrocular discrimination (Blake & Cormack, 1979), and rapidly swapping them between the eyes yield dominance durations that last several eye-swaps (Logothetis, Leopold, & Sheinberg, 1996). Therefore, if vMMN can be generated to eye-swap deviants, then conscious experience of the standards and deviants is not necessary to yield vMMN.
2.  
   What regions of the brain are involved in the generation of vMMN? This question can be answered by neuroimaging, which we have done in this study. Recently, Kimura (2012) summarized the results of four studies and concluded that the neural generators of vMMN can be found in visual striate and extrastriate areas as well as in right prefrontal cortex. But we argue that one can also find converging evidence for neuroimaging by taking a psychoanatomical approach: by showing whether the binocular rivalry mechanism is serially earlier than the vMMN mechanism or after it (Blake, 1995; Julesz, 1971). Simply put, if the binocular rivalry mechanism has resolved the differences between the gratings viewed by the two eyes prior to neural input to the vMMN mechanism, then there should be no vMMN. But if the differences between the two eyes are conveyed to the vMMN mechanism, then there should be a vMMN. The brain areas responsible for binocular rivalry have been extensively studied via neuroimaging (Tong, Meng, & Blake, 2006). Moreover, binocular rivalry's dependence on and independence of other visual mechanisms has been extensively studied (Blake, 1995, 1997).

van Rhijn, Roeber, and O'Shea (2013) have taken a first step to showing that eye of origin information can be used to elicit vMMN. They presented 80% of standards comprising a horizontal grating to one eye and a vertical grating to the other eye, 10% of eye-swap deviants, and 10% of orientation deviants. In half of their experimental blocks, participants tracked their ever-changing percept of binocular rivalry using two keys on a response pad (they called this their attend-to-rivalry condition); in the remaining blocks, participants performed a demanding 2-back task at fixation (they called this their reduced-attention condition). van Rhijn and colleagues found a vMMN from about 140 ms to about 290 ms after stimulus onset in the attend-to-rivalry condition and from about 140 ms to about 220 ms in the reduced-attention condition. In both conditions, the pattern of activity was essentially the same for eye-swap and orientation deviants. van Rhijn and colleagues concluded that eye of origin information can be used to elicit vMMN. 

However, van Rhijn et al. (2013) conceded that their results are only a first step in establishing eye of origin information as a stimulus feature that can be used to yield vMMN. In the present study, we take two additional steps. 

1.  
   Although van Rhijn et al. (2013) used physically identical stimuli for their eye-swap vMMN, they did not for their orientation vMMN. That is, their standards and eye-swap deviants were aligned to cardinal (horizontal and vertical) orientations, whereas their orientation deviants were aligned to oblique (diagonal at ±45°) orientations. It is possible that this physical difference contributed a source of variability in their results that obscured possible differences between their eye-swap and orientation vMMNs (for discussion, see Kujala, Tervaniemi, & Schröger, 2007). In the experiment we report here, we ensured that we computed vMMNs from physically identical standards, eye-swap deviants, and orientation deviants; the stimuli differed only in their frequency and in their predictability.
2.  
   van Rhijn et al (2013) measured their ERPs with 26 electrodes, preventing them from conducting a detailed analysis of the sources of the vMMNs they found. In the experiment we report here, we used 58 electrodes, allowing us to use standardized low-resolution brain electromagnetic tomography (sLORETA; Pascual-Marqui, 2002). sLORETA computes the distribution of current source density of EEG and provides a solution to the inverse problem of localizing cerebral sources, and has been shown to achieve zero localization error in two independent investigations under ideal conditions (Greenblatt, Ossadtchi, & Pflieger, 2005; Sekihara, Sahani, & Nagarajan, 2005), making it a useful technique for our purposes: localizing vMMN activity in the brain.

Similar to the results of van Rhijn et al. (2013), we found an eye-swap vMMN and an orientation vMMN, but unlike their results, our eye-swap vMMN was maximal about 130 ms later than our orientation vMMN. We also found that both vMMNs shared similar sources in occipital and frontal areas of the brain but different sources in parietal and temporal areas. 

Method

Participants

Fifteen volunteers participated in our study. There was no reward or financial incentive offered to participants. All participants gave written informed consent prior to the experiment, had normal or corrected-to-normal visual acuity in both eyes, and showed normal binocular rivalry in a 12-minute pre-test session. Data of three participants were excluded from further analyses, because too few epochs remained after data pre-processing. Mean (SD) age of the remaining 12 participants, of whom seven were male, was 30 (12) years. The study was approved by Southern Cross University's Human Research Ethics Committee (ECN-11-149) and was conducted in accordance with the ethical standards laid down in the Declaration of Helsinki (World Medical Association, 2000). 

Apparatus

The experiment was conducted in the EEG Research Laboratory at Southern Cross University, Coffs Harbour, Australia, in a sound-attenuated (42 dB) room with the display of the stimuli providing the only light. During the experiment, each participant sat in a chair at a desk with his or her head stabilized by a chin-and-forehead rest. Stimuli were presented on a Samsung 2233RZ monitor (1024 × 768; 60 Hz) at a viewing distance of 57 cm when viewed through a mirror stereoscope (Screenscope-SA-200-Monitor-Type; Stereo Aids, Albany, WA, Australia). The experiment was controlled by a Macintosh Mini running specially written Matlab scripts using the Psychophysics Toolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997). Participants responded using two keys on a response pad. 

EEG was recorded from 58 Ag/AgCl active electrodes placed according to the extended 10-20 system (AF7, AF3, AF4, AF8, F7, F5, F3, F1, Fz, F2, F4, F6, F8, FT7, FC5, FC3, FC1, FC2, FC4, FC6, FT8, T7, C5, C3, C1, Cz, C2, C4, C6, T8, TP7, CP5, CP3, CP1, CPz, CP2, CP4, CP6, TP8, P7, P5, P3, P1, Pz, P2, P4, P6, P8, PO9, PO7, PO3, POz, PO4, PO8, PO10, O1, Oz, O2) and referenced to FCz, with the ground at AFz. A vertical electrooculogram (EOG) was recorded by placing an electrode above (we used FP2) and below the right eye; a horizontal EOG was recorded by placing an electrode on the outer canthi of each eye. We also placed an electrode on each earlobe. The sampling rate of the EEG was 500 Hz and the online filtering was 1000 Hz. 

Stimuli

We used annulus-shaped patches of black (0.40 cd/m²) and white (86.67 cd/m²; Michelson contrast of .99) sine-wave gratings on a mean-luminance (43.54 cd/m²) gray background as binocular-rivalry stimuli. The grating presented to one eye could be vertical (0°), in which case the grating presented to the other eye was horizontal (90°), or left-oblique (−45°), in which case the grating presented to the other eye was right-oblique (45°). The gratings had a spatial frequency of 1.6 c/°. The outer diameter of the gratings was 3.2° of visual angle; the inner diameter was 0.67°. The central area contained a central red fixation cross of 0.3° with a line width of 0.1°. The gratings were surrounded by three white fusion rings; these served to lock vergence. The outer diameter of the largest ring was 6.4°, and each ring had a line width of 0.05° and was 0.3° from its neighbor. 

Design and procedure

The experiment consisted of 16 blocks. In half of them, participants were presented with cardinal standards—a horizontal grating to one eye and a vertical grating to the other eye. In the other half, participants were presented with oblique standards, a left-oblique grating to one eye and a right-oblique grating to the other eye. Furthermore, in half of the cardinal blocks and in half of the oblique blocks, participants were presented with eye-swap deviants, with the gratings swapped between the eyes. In the other half, participants were presented with orientation deviants, with the gratings rotated by 45°. Therefore, there were four different types of blocks: cardinal standards with cardinal eye-swap deviants (Figure 1a), cardinal standards with oblique orientation deviants, oblique standards with oblique eye-swap deviants, and oblique standards with cardinal orientation deviants (Figure 1b). Each block type was presented four times for each participant. The order of these blocks was counterbalanced within participants and the starting block type was counterbalanced over participants. 

Figure 1

 

Illustration of two possible sequences of standards and deviants. We presented 80% of binocular rivalry standards, one grating to one eye and an identical, orthogonally orientated grating to the other eye, and 20% of deviants, shown in each sequence as the fifth display of the stimuli. (a) A sequence of cardinal standards, in this case, a horizontal grating to the left eye and a vertical grating to the right eye, and an eye-swap deviant, in this case, a vertical grating to the left eye and a horizontal grating to the right eye. (b) A sequence of oblique standards, in this case, a left-oblique grating to the left eye and a right-oblique grating to the right eye, and an orientation deviant, in this case, a vertical grating to the left eye and a horizontal grating to the right eye. We also tested all other possible eye-stimulus arrangements of cardinal and oblique standards, eye-swap deviants, and orientation deviants.

Each block contained 480 trials. An individual trial comprised a display of grating stimuli for 500 ms. Following this, the gratings shifted 180° in phase. This change, which happened between one frame of the display and the next, marked the end of one trial and the beginning of the next, allowing us to time lock the EEG signal. Standards comprised 80% of all trials within a block; the remaining 20% of trials were either eye-swap deviants or orientation deviants. The order of trials within a block was random for each participant with the constraints that the first three trials and the last two trials had to display standard stimuli, and any two deviants had to be separated by at least three standards. 

The participant's task was to look at the fixation cross in the center of the gratings, to report binocular rivalry dominance of one or the other grating by pressing down one or another key on the response pad, and to refrain from pressing either key if any combination of the two gratings was perceived. The assignment of keys to orientations was counterbalanced over participants. There were two response events: key presses, when a key was pressed, and key releases, when a key stopped being pressed. We used key presses and releases to determine participants' mean binocular rivalry dominance durations and to assess the intrusiveness of deviants on participants' visual consciousness. 

ERP analysis

For data analyses, we re-referenced the EEG data offline to the average of all scalp electrodes and we filtered the data using a 0.1 to 35 Hz phase-shift free Butterworth filter (48 dB/Octave slope). We extracted the epochs from −100 to 500 ms, and we baseline corrected all epochs to their mean voltage from −100 to 0 ms. We excluded all epochs with signals exceeding peak-to-peak amplitudes of 200 μV at any EEG channel, or of 60 μV at any EOG channel, as well as any standards that immediately followed a deviant. We computed ERPs separately for each axis (cardinal, oblique) and for each stimulus (standard, eye-swap deviant, orientation deviant) for each participant, and excluded any data sets containing fewer than 250 epochs for any ERP. 

To assess vMMN, we computed difference waves by subtracting the grand-averaged ERPs for standards from the grand-averaged ERPs for either eye-swap deviants or orientation deviants. This yielded four difference waves: cardinal eye-swap difference waves (cardinal eye-swap deviants minus cardinal standards), cardinal orientation difference waves (cardinal orientation deviants minus cardinal standards), oblique eye-swap difference waves (oblique eye-swap deviants minus oblique standards), and oblique orientation difference waves (oblique orientation deviants minus oblique standards). 

Using visual inspection of the ERPs and difference waves, we determined the time windows for the orientation vMMN (240 to 290 ms) and for the eye-swap vMMN (350 to 400 ms) components. We also determined the time window for the P1 (84 to 134 ms), which is an ERP component associated with the sensory and perceptual processing of visual stimuli (Luck, 2012); time windows for a deviance-related positivity (84 to 94 ms) and a deviance-related negativity (110 to 130 ms), which is a sustained difference between standards and deviants during the time range of the P1; and a time window for a sustained negativity (420 to 470 ms), which are sustained differences between standards and deviants after the vMMN components. We analyzed the mean amplitudes of the ERPs for the time windows of interest using repeated-measures ANOVA with factors region (anterior, central, posterior), hemisphere (left, right), axis (cardinal, oblique), and stimuli (standards, eye-swap deviants or orientation deviants). We also calculated voltage maps for the orientation and eye-swap vMMN time windows and used sLORETA (Pascual-Marqui, 2002) to locate the current sources in the brain with a voxel-by-voxel t-test. A randomization test based on statistical non-parametric mapping (SnPM; number of randomizations: 5,000) was used to correct for multiple comparisons (Nichols & Holmes, 2002). 

Results

Behavioral data

The time between a key press and its subsequent release yielded the time of one episode of binocular rivalry dominance. As expected, the distribution of these times had the typical gamma-like shape (Levelt, 1967; Walker, 1975; Zhou et al., 2004). Mean (SD) binocular rivalry dominance duration was 1.43 (1.04) s in cardinal blocks with cardinal eye-swap deviants, 1.59 (1.17) s in cardinal blocks with oblique orientation deviants, 1.47 (0.94) s in oblique blocks with oblique eye-swap deviants, and 1.61 (1.08) s in oblique blocks with cardinal orientation deviants. These times were not significantly different: repeated-measures ANOVA on log-transformed data with factors axis (cardinal, oblique) and block (eye-swap block, orientation block) found no significant main effects or interactions (see Table S1), suggesting that participants' experiences of binocular rivalry did not differ between experimental blocks. 

To assess the intrusiveness of deviants on participants' conscious experience of binocular rivalry, we compared dominance durations containing only standards with dominance durations containing either an eye-swap deviant or an orientation deviant. However, because episodes of binocular rivalry dominance begin about 450 ms ahead of a key press and end about 450 ms ahead of a key release (Alais, Cass, O'Shea, & Blake, 2010), we classified dominance durations as containing a standard, an eye-swap deviant, or an orientation deviant only if a key was pressed 1 ms or more before the onset of the deviant (we chose this time to ensure that an episode of dominance started before the onset of the deviant) and only if the key was released 1 ms or more after the offset of the deviant (we chose this time to ensure that an alternation occurred some time after the onset of the deviant). We also analyzed other key press and release times, such as excluding key presses and releases within 300 ms of the onset of the deviant; the results (see next paragraph) did not change with other criteria. 

Mean binocular rivalry dominance duration was 1.73 (0.69) s for cardinal standards, 2.21 (1.36) s for cardinal eye-swap deviants, 2.17 (1.63) s for cardinal orientation deviants, 1.79 (0.40) s for oblique standards, 2.08 (1.26) s for oblique eye-swap deviants, and 1.65 (0.58) s for oblique orientation deviants. These times were not significantly different: repeated-measures ANOVA on log-transformed data using the factors axis (cardinal, oblique) and stimuli (standards, eye-swap deviants, orientation deviants) found no significant main effects or interactions (see Table S2). This suggests that eye-swap and orientation deviants did not influence participants' experiences of binocular rivalry. 

However, we also found that the mean time for a key release after an eye-swap deviant was 0.70 (0.06) s for cardinal gratings and 0.70 (0.05) s for oblique gratings. These times were not significantly different from those after an orientation deviant, of which participants must have been conscious: 0.68 (0.04) s for cardinal gratings and 0.68 (0.04) s for oblique gratings (see Table S3). This means that even though dominance durations containing standards and eye-swap deviants did not differ, the onset of an eye-swap deviant caused a change in participants' experience of binocular rivalry that led uniformly to a key release, meaning that participants were conscious of some sort of change in stimulation, even if they did not consciously know anything about the change. 

ERP data

Figure 2 shows the grand-averaged ERPs to the onset of the phase-shift in the grating, as well as their difference waves. Because we found no difference between cardinal and oblique gratings (see below), we collapsed the axis factor. Therefore, the traces are for standards (black traces), eye-swap deviants (red traces), orientation deviants (green traces), eye-swap difference waves (red dotted traces), and orientation difference waves (green dotted traces), and are averaged over left anterior (AF3, F5, F3, F1, FC3), right anterior (AF4, F2, F4, F6, FC4), left central (C5, C3, CP5, CP3, CP1), right central (C4, C6, CP2, CP4, CP6), left posterior (P7, PO9, PO7, PO3, O1), and right posterior (P8, PO4, PO8, PO10, O2) electrode sites. We assessed the mean amplitudes of the ERPs in the time windows of interest using repeated-measures ANOVAs. 

Figure 2

 

ERP data. The graphs show time (ms) on the x-axis, with 0 indicating the onset of the stimulus and voltage (μV) on the y-axis, with positive voltages above 0 and negative voltages below. Each graph represents the ERPs and difference waves averaged over anterior, central, or posterior electrodes for the left or right hemispheres. The gray bars show the analyzed time windows (see text).

What we see in Figure 2 are large ERP components at posterior electrodes, especially from electrodes over the right hemisphere, small components at central electrodes, and moderately-sized components, reversed in polarity, at anterior electrodes (this characterization of the data is supported by ANOVAs in Tables S4 to S8). For example, for the P1 time window, there was a significant interaction for region and hemisphere from orientation and eye-swap ANOVAs, F(2, 22) = 5.54, p = .011, η_p² = .34 and F(2, 22) = 5.67, p = .010, η_p² = .34, respectively (see Table S4). We consistently see this right posterior advantage in many of our ERP studies (e.g., Roeber et al., 2008; van Rhijn, et al., 2013; Veser, O'Shea, Schröger, Trujillo-Barreto, & Roeber, 2008; but see also O'Shea, Kornmeier, & Roeber, 2013), and it is consistent with other approaches that show a right-hemisphere advantage for processing of visual, spatial tasks (e.g., Corballis, Funnell, & Gazzaniga, 2002; Wilcke, O'Shea, & Watts, 2009; for review, see Corballis, 2003). 

Figure 3 shows the grand-averaged ERPs and their difference waves at right posterior electrodes, where the ERP components are most pronounced. They show a typical pattern of activity for phase-reversal stimuli: a peak at about 110 ms (the P1), a negative shift starting at about 200 ms, and a return to baseline at about 350 ms for standards (Odom et al., 2010; O'Shea, Roeber, & Bach, 2009). For eye-swap and orientation deviants, voltages returned to baseline some time after our 500-ms epochs ended. At the P1, standards yielded similar voltages to eye-swap deviants, F(1, 11) = 1.82, p = .205, η_p² = .14, and bigger voltages than orientation deviants, F(1, 11) = 13.02, p = .004, η_p² = .54 (see Table S4). The former result is similar to that found by van Rhijn et al. (2013) but the latter is the opposite. Instead, they found that orientation deviants yielded more activity than standards. We will consider this in the Discussion. 

Figure 3

 

ERP data at right posterior electrodes.

The difference waves at right posterior electrodes show: 

•  
  A positive peak at about 90 ms, in which eye-swap deviants started to go positive for the P1 before the standards, F(1, 11) = 6.34, p = .029, η_p² = .37, which we call the early positivity, and a negative trough at about 110 ms, in which standards gave a larger voltage for the P1 than the orientation deviants, F(1, 11) = 13.64, p = .004, η_p² = .55, which we call the early negativity. There were no other significant differences. In particular, the early positivity for the orientation difference wave was not significant, nor was the early negativity for the eye-swap difference wave (see Tables S5 and S6). van Rhijn et al. (2013) found a similar positivity at about 100 ms, but then their difference waves went negative to a trough at about 150 ms with no quick return to baseline. They called their trough an “early vMMN” (p. 6). Although vMMN as early as 100 ms after stimulus onset is not unprecedented (e.g., Astikainen, Ruusuvirta, & Korhonen, 2000; Iijima, Osawa, Nageishi, Ushijima, & Iwata, 1996), it is earlier than the traditional lower-bound timing of vMMN (Kimura, 2012).
•  
  A trough at about 250 ms for the orientation difference wave, in which orientation deviants gave a bigger voltage than standards, F(1, 11) = 52.64, p < .001, η_p² = .83. We found essentially the same pattern of results at left posterior electrodes, F(1, 11) = 27.74, p < .001, η_p² = .72, as well as at anterior electrodes, F(1, 11) = 17.79, p = .001, η_p² = .62, and at central electrodes, F(1, 11) = 32.85, p < .001, η_p² = .75 (see Table S7). In general, this activity is similar to the “late vMMN” found by van Rhijn et al. (2013, p. 6) at about 210 ms. We call this trough the orientation vMMN.
•  
  Another trough at about 380 ms for the eye-swap difference wave, in which eye-swap deviants gave a bigger voltage than standards, F(1, 11) = 40.61, p < .001, η_p² = .79. Again, we found essentially the same pattern of results at left posterior electrodes, F(1, 11) = 26.05, p < .001, η_p² = .70, as well as at anterior electrodes, F(1, 11) = 11.00, p = .007, η_p² = .50, and at central electrodes, F(1, 11) = 14.62, p = .003, η_p² = .57 (see Table S7). This activity is much later than the “late vMMN” found by van Rhijn et al. (2013, p. 6). We call this trough the eye-swap vMMN.
•  
  A sustained negativity for both difference waves after each vMMN to at least 500 ms. That is, orientation and eye-swap deviants gave a bigger voltage than standards, F(1, 11) = 31.32, p < .001, η_p² = .74 and F(1, 11) = 21.28, p = .001, η_p² = .66, respectively. Orientation deviants also gave a bigger voltage than standards at left posterior electrodes, F(1, 11) = 11.71, p = .006, η_p² = .52, and at central electrodes, F(1, 11) = 29.42, p < .001, η_p² = .73, but not at anterior electrodes. Eye-swap deviants also gave a bigger voltage than standards at left posterior electrodes, F(1, 11) = 16.51, p = .002, η_p² = .60, at anterior electrodes, F(1, 11) = 6.74, p = .025, η_p² = .38, and at central electrodes, F(1, 11) = 14.29, p = .003, η_p² = .57 (see Table S8). Although van Rhijn et al. (2013) found no evidence of this activity, it is similar to a late negativity found by Kobayashi, Kato, and Kuriki (1996), which they thought was associated with the alternation process in binocular rivalry, suggesting that our sustained negativity reflects conscious detection of the deviants.

Figure 4 shows the voltage maps for the orientation and eye-swap vMMN components. We include these maps for comparison with van Rhijn et al. (2013). Similar to their maps, our maps show negative voltages in posterior regions of the brain and positive voltages in anterior and central regions of the brain. Our maps also show a right hemisphere bias in posterior regions of the brain for orientation vMMN, F(1, 11) = 7.09, p = .022, η_p² = .39, but not for eye-swap vMMN. Despite this, in general, the maps show that the two sorts of deviants yield essentially the same pattern of voltages in the brain, but at different latencies. 

Figure 4

 

Voltage maps of the orientation vMMN and of the eye-swap vMMN. The maps show similar patterns of negative voltages over posterior regions of the brain and positive voltages over anterior and central regions of the brain.

To locate the sources that give rise to the orientation and eye-swap vMMN components, we used sLORETA (Pascual-Marqui, 2002). Figure 5 shows the sLORETA maps for both vMMNs and Table 1 shows the best matches for current sources as estimated by sLORETA. According to sLORETA, for orientation vMMN, the best matches for current sources were in occipital, parietal, and frontal areas of the brain, whereas for eye-swap vMMN, the best matches for current sources were in occipital, temporal, and frontal areas of the brain. These results are mostly consistent with Kimura et al. (2010), who found significant differences in brain activity between orientation deviants and equiprobable controls in occipital and frontal areas of the brain. These results suggest that our vMMNs share similar, but not identical, neural generators. 

Figure 5

 

sLORETA maps of the orientation vMMN and of the eye-swap vMMN. Areas colored in red represent the voxels where statistically significant differences (ps < 0.01) between standards and deviants were found by sLORETA. (a) According to sLORETA, the neural generators of orientation vMMN were in occipital, parietal, and frontal areas of the brain. (b) According to sLORETA, the neural generators of eye-swap vMMN were in occipital, temporal, and frontal areas of the brain. (c) Three views to reveal the frontal source of the eye-swap vMMN. Each view is a slice through the brain at the location of significant activity in the right medial frontal gyrus (see Table 1), from left to right, coronally, horizontally, and sagittally.

Table 1

 

Brain areas where significant differences between standards and orientation deviants (240 to 290 ms; orientation vMMN) and standards and eye-swap deviants (350 to 400 ms; eye-swap vMMN) were shown by sLORETA, giving anatomical region, Brodmann area (BA), the x-, y-, and z-coordinates of the brain according to the Montreal Neurological Institute (MNI), and the t-value. Notes: PL = parietal lobe, OL = occipital lobe, FL = frontal lobe, TL = temporal lobe. The critical t-value for p < .01 was 4.08 for orientation vMMN and 4.82 for eye-swap vMMN.

 

Discussion

We found an orientation vMMN that was maximal at about 250 ms and an eye-swap vMMN that was maximal at about 380 ms. We argue here that eye of origin information can be added to the list of stimulus features that yield a vMMN. However, one possible objection arises if we consider each eye separately: an eye-swap deviant is essentially the same as an orientation deviant, except that the rotation is 90° instead of 45°. Could it be that vMMN elicited by changes in eye of origin information is nothing more than an orientation vMMN? We argue not for two reasons: 

1.  
   Czigler and Sulykos (2010) found similar vMMNs to 30° and 60° changes in orientation information, suggesting that the amplitude and latency of vMMN do not correspond with the size of the change between standards and deviants. Similarly, Maekawa et al. (2005) used windmill patterns with varying radial frequencies (i.e., number of windmill arms) and found that increasing radial frequency had no effect on the amplitude of vMMN. Furthermore, different stimulus features are known to yield vMMN at different times after stimulus onset (e.g., Pazo-Alvarez et al., 2003, their table 3). We found that the timing of orientation vMMN was earlier (by about 130 ms) than that of eye-swap vMMN. We also found that orientation vMMN had sources in parietal areas of the brain whereas eye-swap vMMN had sources in temporal areas of the brain. This suggests that an eye-swap deviant is not simply a change in orientation information.
2.  
   Although it is logically possible to consider what is happening in each eye separately, this does not mean that the normal human visual system operates in this way. The normal human visual system is fundamentally binocular (Blake & Wilson, 2011; Howard & Rogers, 2012). An eye-swap deviant presents exactly the same orientations to the visual system as done by a standard, so any adaptation (or “refractoriness”; e.g., Stefanics et al., 2014, p. 1; see also O'Shea, 2015) to those orientations from repeated viewing of the standards will continue, because adaptation to one orientation in one eye partly transfers to the other eye (Blakemore & Campbell, 1969); the only non-adapted neurons to be stimulated by an eye-swap deviant will be those sensitive to eye of origin. An orientation deviant, however, presents new orientations to both eyes so that all new neurons will be stimulated. Therefore, it is unlikely that the visual system would treat an eye-swap deviant as a change in orientation information.

If it can be accepted that eye of origin information can yield a vMMN, then there are six features of it that deserve special mention. 

First, it is perhaps not all that surprising that eye of origin information can be used to elicit the vMMN, because it is a basic feature of binocular vision that yields depth perception via stereopsis when the eyes view similar stimuli or binocular rivalry when the eyes view dissimilar stimuli (Howard & Rogers, 2012; Wheatstone, 1838). Despite this, from a functional perspective, it is surprising that eye-swap vMMN peaked later than orientation vMMN, because it would make sense to process eye of origin information, such as whether the hungry lion is in front of the glass or behind it, without requiring visual consciousness. Indeed, as shown with random-dot stereograms, depth perception precedes any analysis of the form of the objects in each eye (Julesz, 1960); this analysis happens within 120 ms (Over & Long, 1973). However, we suspect that the long latency of eye-swap vMMN is because our eye-swap deviants did not reverse the perception of depth, because binocular rivalry stimuli do not provide any cues for stereopsis. Furthermore, the timing of our eye-swap vMMN is consistent with the fact that it takes trained observers about 350 ms to distinguish a transition from binocular uncorrelation in dynamic random-dot stereograms to complementation, which is a similar, subtle change in the nature of binocular rivalry (Julesz & Tyler, 1976). 

Nevertheless, it is important to point out that the eye-swap difference waves show a small positivity at about 90 ms—the early positivity, which is consistent with the notion that depth perception precedes any analysis of form (Julesz, 1960). The orientation difference waves, on the other hand, show an early negativity at about 110 ms. This prompts us to address two questions: 

1.  
   Why are these deviance-related activities not vMMNs? The early positivity is not a vMMN, because it is positive in polarity instead of negative (Kimura, 2012). The early negativity is not a vMMN, because it is earlier than the traditional lower-bound timing of vMMN (Kimura, 2012). Nevertheless, the early positivity and early negativity are deviance-related components, even though they are not vMMNs as traditionally defined.
2.  
   Why is it that our two deviants yielded very different early deviance-related processing? The early positivity from eye-swap deviants is because its P1 starts earlier than that of the standards, presumably because neurons sensitive to eye of origin can increase their activity faster than neurons sensitive to orientation, because the latter neurons are already adapted from repeated viewing of the standards. The early negativity from orientation deviants is because its P1 is smaller than that of the standards. One explanation for this could be that phase-shifting the gratings allowed negative afterimages to build up during one trial that then summed with the phase-shifted gratings on the next trial, yielding bigger P1s to standards than to orientation deviants for which no summation would occur. Indeed, Campbell and Kulikowski (1972) showed that the amplitude of visual evoked potentials are bigger if sine-wave gratings are phase shifted at 8 Hz than if the same gratings are turned on and off at 8 Hz.

Regardless, our results show early deviance-related processing of eye of origin and orientation information, suggesting that some sort of binocular processing precedes any analysis of form, even if this processing does not qualify as a vMMN. 

Second, we hoped that eye-swap deviants would be useful for showing that conscious experience of the standards and deviants is not necessary to generate vMMN. However, our behavioral data indicate participants were conscious of our eye-swap deviants, contradicting the work of Logothetis et al. (1996). One reason for this could be that their effect occurs within a small range of stimulus parameters (Lee & Blake, 1999), and our stimuli were outside these parameters. However, this does not necessarily limit our conclusion that swapping binocular rivalry stimuli between the eyes yields a vMMN from information of which observers are unaware, because observers are unaware of which eye is viewing which stimulus during binocular rivalry (Blake & Cormack, 1979). Nevertheless, these results leave open the question of whether vMMN can be generated by sequences of stimuli of which we are not conscious. We addressed this in another experiment (Roeber, Jack, O'Shea, Widmann, & Schröger, 2013). In that experiment, we presented luminance decrement deviants to one eye during binocular rivalry suppression and found that visual consciousness of the deviants is not necessary to elicit vMMN. 

Third, these results provide converging operations showing an eye-swap vMMN with those of van Rhijn et al. (2013). They defined their trials by inserting a 100 ms blank interval between presentations; we defined our trials by phase-shifting the stimuli. They mixed up their two types of deviants within each block of trials; we presented only a single sort of deviant within a block of trials. They did not use physically identical stimuli for all of their vMMN analyses; we did. The fact that both approaches yielded rather similar results, an eye-swap vMMN and an orientation vMMN, is testimony to its robustness. For this reason, we are confident that eye of origin information can be considered a stimulus feature that reliably yields a vMMN. 

Fourth, we also set out to extend upon the results of van Rhijn et al. (2013) by using sLORETA to localize vMMN activity in the brain. Our results show that the neural generators of orientation vMMN can be found in occipital, parietal, and frontal areas of the brain, whereas the neural generators of eye-swap vMMN can be found in occipital, temporal, and frontal areas of the brain. Although we are mindful of the uncertain spatial resolution of techniques like sLORETA (Luck, 2012), we would like to take its results at face value and speculate on what we might learn from them. We suspect that the parietal source has something to do with the sudden change in conscious experience caused by the orientation deviant, because the superior parietal lobule is thought to be involved in the processing of cues that contribute to our perception of egocentric space (Lester & Dassonville, 2014). We suspect that the temporal source has something to do with the release of one of the response keys immediately after the onset of the eye-swap deviant. Bogousslavsky, Miklossy, Deruaz, Assal, and Regli (1987) found that a lesion in the fusiform and lingual gyri spared most visual processing (including face and object recognition) but seriously reduced complex visual–verbal learning. That is, activation in the fusiform gyrus might reflect complex visual change detection requiring some verbal self-instruction, such as releasing the response key. Indeed, this is consistent with our behavioral data, because we found that an eye-swap deviant led uniformly to a key release. 

Fifth, one can find converging evidence for neuroimaging (see above) by taking a psychoanatomical approach (Blake, 1995; Julesz, 1971). If neural processing proceeds in a serial fashion, then our finding an eye-swap vMMN shows that eye of origin information is processed by at least one part of the vMMN mechanism prior to the application of binocular rivalry suppression. Binocular rivalry suppression is thought to involve several sites, with the strength of suppression deepening as binocular rival stimuli become more complex (Alais & Melcher, 2007; Freeman, 2005; Nguyen, Freeman, & Alais, 2003). With the simple stimuli we used, gratings, binocular rivalry suppression is thought to operate initially before or at V1. For example, Blake, Tadin, Sobel, Raissian, and Chong (2006) showed that the elevation of contrast thresholds from prolonged viewing of a grating, itself thought to be mediated by adaptation of V1 neurons (Maffei, Fiorentini, & Bisti, 1973; Saul & Cynader, 1989), is reduced if the grating is invisible via binocular rivalry suppression. 

Of course, it is unlikely that the vMMN mechanism involves activity at one site and the binocular rivalry mechanism at another, single site. For instance, vMMN is thought to have mechanisms in visual striate and extrastriate areas as well as in right prefrontal cortex linked by feedforward and feedback connections (Kimura, 2012). Similarly, binocular rivalry suppression is also thought to have mechanisms at various early and late stages of the visual pathway linked by feedforward and feedback connections (Tong et al., 2006). The timing of our eye-swap vMMN is consistent with there being a vMMN mechanism somewhere between V1 and frontal areas of the brain, all of which have been implicated in binocular rivalry suppression in single-cell studies (V1: Leopold & Logothetis, 1996; inferotermporal cortex: Sheinberg & Logothetis, 1997; frontal areas: Panagiotaropoulos, Deco, Kapoor, & Logothetis, 2012). Furthermore, an early site for the initial stages of the vMMN is consistent with recent evidence of deviance-related activity in very early parts of the auditory pathway (Cacciaglia et al., 2015; Leung, Recasens, Grimm, & Escera, 2013). This raises the intriguing possibility that our early deviance-related activities are the result of a similar mechanism in the visual pathway. We should emphasize that we are not saying that prediction error is assessed in the lateral geniculate nucleus, nor that our EEG technique could show it if it were, but only that the time range of our early deviance-related activity at least puts it in the visual cortex. 

Sixth, the present study allowed us to assess the influence of the oblique effect on vMMN. The oblique effect is a decrease in performance for visual tasks when stimuli are oblique instead of cardinal (for reviews, see Appelle, 1972; Howard, 1982; Li, Peterson, & Freeman, 2003). Recently, Takács et al. (2013) reported an oblique effect for orientation vMMN. They found a bigger and more sustained vMMN for changes from horizontal standards to vertical deviants (and vice versa) than for changes from left-oblique standards to right-oblique deviants (and vice versa). However, we found no evidence of the oblique effect for either of our vMMNs. We must confess that we are not sure if this is because the spatial frequency of our gratings were not optimal for revealing it (Arakawa et al., 2000), or if this is because binocular rivalry prevented us from finding it (e.g., Abadi, 1976; Blake, Yu, Lokey, & Norman, 1998; Borra, Hooge, & Verstraten, 2010; Fahle, 1982; Wade, de Weert, & Swanston, 1984). In any case, we are not too worried that Takács et al. observed the oblique effect and we did not, because it does not affect our conclusion: that eye of origin information can elicit vMMN. 

In conclusion, we set out to confirm a new stimulus feature for eliciting vMMN: eye of origin information. We found an eye-swap vMMN that was maximal at about 380 ms and an orientation vMMN that was maximal at about 250 ms, and that orientation vMMN originated in occipital, parietal, and frontal areas of the brain whereas eye-swap vMMN originated in occipital, temporal, and frontal areas of the brain. We also found early deviance-related activity, earlier than would traditionally qualify as a vMMN. We conclude that eye of origin information can be used to elicit vMMN, that eye-swap vMMN is different to orientation vMMN, and that vMMN can be generated from information of which observers are unaware. 

Acknowledgments

This research was supported by a Southern Cross University startup grant awarded to ROS. 

Commercial relationships: none. 

Corresponding author: Bradley N. Jack. 

Email: [e-mail]. 

Address: Discipline of Psychology, School of Health and Human Sciences, Southern Cross University, Coffs Harbour, NSW, Australia. 

Supplementary (ANOVA) tables

Table S1

 

Results of the statistical analysis of the binocular rivalry dominance duration data.

 

Table S2

 

Results of the statistical analysis of the intrusiveness of deviants on the binocular rivalry dominance duration data.

 

Table S3

 

Results of the statistical analysis of the key release after a deviant duration data.

 

Table S4

 

Results of the statistical analysis of the ERP data for the orientation (84 to 134 ms) and eye-swap (84 to 134 ms) P1 time windows. Notes: Values highlighted in bold indicate statistical significance (p < 0.05).

 

Table S5

 

Results of the statistical analysis of the ERP data for the orientation (84 to 94 ms) and eye-swap (84 to 94 ms) early positivity time windows. Notes: Values highlighted in bold indicate statistical significance (p < 0.05).

 

Table S6

 

Results of the statistical analysis of the ERP data for the orientation (110 to 130 ms) and eye-swap (110 to 130 ms) early negativity time windows. Notes: Values highlighted in bold indicate statistical significance (p < 0.05).

 

Table S7

 

Results of the statistical analysis of the ERP data for the orientation (240 to 290 ms) and eye-swap (350 to 400 ms) vMMN time windows. Notes: Values highlighted in bold indicate statistical significance (p < 0.05).

 

Table S8

 

Results of the statistical analysis of the ERP data for the orientation (420 to 470 ms) and eye-swap (420 to 470 ms) sustained negativity time windows. Notes: Values highlighted in bold indicate statistical significance (p < 0.05).