Fixation-related visual mismatch negativity

Abstract

Vision under natural conditions could be studied by combining electroencephalogram (EEG) and eye tracking as well as using saccades as triggers for the onset of the fixation-related potentials (FRPs) and for the oculomotor inhibition (OMI) that follows every saccade. The result of this analysis is thought to be equivalent to the event-related response following a peripheral preview. Previous studies that measured responses to visual deviants in a sequence of flashed stimuli found an increased negativity in the occipital N1 component (visual mismatch negativity [vMMN]), and prolonged saccadic inhibition for unexpected events. The aim of the current study was to develop an oddball paradigm in constrained natural-viewing and determine whether a similar mismatched FRP and prolonged OMI for deviance could be found. To this end, we developed a visual oddball paradigm on a static display to generate expectancy and surprise across successive saccades. Observers (n = 26) inspected, one after the other, seven small patterns of E and an inverted E arranged on the screen along a horizontal path, with one frequent (standard) and one rare (deviant), looking for a superimposed tiny dot target in each 5-second trial. Our results show a significantly larger FRP-N1 negativity for the deviant, compared with the standard and prolonged OMI of the following saccade, as previously found for transient oddballs. Our results show, for the first time, prolonged OMI and stronger fixation-related N1 to a task-irrelevant visual mismatch (vMMN) in natural, but task-guided viewing. These two signals combined could serve as markers of prediction error in free viewing.

Introduction

Our brain computes statistical regularities in the environment, which allows us to anticipate upcoming events and respond to sudden changes (Bubic, von Cramon, & Schubotz, 2010). The event-related potential (ERP), mismatch negativity (MMN), is a known electrophysiological marker, reflecting an automatic change-detection mechanism in the auditory domain (Duncan-Johnson & Donchin, 1977; Jääskeläinen, Ahveninen, Bonmassar, Dale, Ilmoniemi, et al., 2004; Näätänen, Gaillard, & Mäntysalo, 1978). It is manifested by increased negativity, peaking at about 170 ms post-stimulus onset at temporal electrodes. Some studies suggest a similar visual response for visual mismatches, termed visual MMN (vMMN; Stefanics, Stefanics, Kremláček, & Czigler, 2014). Infrequent color patterns (Czigler, Balázs, & Pató, 2004), low spatial frequency gratings (Cleary, Donkers, Evans, & Belger, 2013), and face and house orientations (Zhang, Li, Wang, Zhao, Wang, & Miao, 2018) elicited a posterior vMMN. Even facial expressions and emotions were used in an oddball paradigm and elicited increased negativity, peaking at 100 to 200 ms (Astikainen, Cong, Ristaniemi, & Hietanen, 2013; Gayle, Gal, & Kieffaber, 2012; Kreegipuu, Kuldkepp, Sibolt, Toom, Allik, & Näätänen, 2013). 

Unlike traditional ERPs that use briefly flashed visual transients typically presented at the observer's central visual field, natural vision is based on scanning the image via saccades that follow a peripheral preview. Accumulating evidence from recent free-viewing studies suggest that the brain's response following a saccade, termed fixation-related potentials (FRPs), exhibits electrophysiological components that are consistent with ERPs. For example, the saccade-related occipital lambda response reflects the same information processing as the classic VEP P1 (Kazai & Yagi, 2003). Another example from a recent study that examined the face-selective activity at the lateral temporo-occipital electrodes, N170 (Bentin, Allison, Puce, Perez, & McCarthy, 1996) similarly found an increased negativity for faces in free-viewing conditions (Auerbach-Asch, Bein, & Deouell, 2020). Other classic ERP findings were replicated in free viewing conditions, such as centro-parietal P300, elicited by voluntary target detection in visual search (Hiebel, Ischebeck, Brunner, Nikolaev, Höfler, & Körner, 2018) and the N400 priming effect in natural reading (Dimigen, Sommer, Hihlfeld, Jacobs, & Kliegl, 2011; Niefind & Dimigen, 2016). Combining electroencephalogram (EEG) and eye-tracking measurements enabled us to cross examine eye movement and electrophysiological changes that are influenced by stimulus habituation and by infrequent deviant stimuli. 

Microsaccades are miniature saccades, generated by neural activity in the superior colliculus (SC; Hafed & Krauzlis, 2012). Other cortical regions take part in saccade planning, including the lateral intraparietal area and the frontal eye field (Andersen, Martyn, Bracewell, Barash, Gnadt, et al., 1990; Corbetta, Akbudak, Conturo, Snyder, Ollinger, et al., 1998; Hafed, Yoshida, Tian, Buonocore, & Malevich, 2021; Hanes, Patterson, & Schall, 1998; Schall, Hanes, Thompson, & King, 1995). They occur during fixation, at a rate of one to three per second, and are known to be inhibited momentarily (oculomotor inhibition [OMI]; Bonneh, Adini, Sagi, Tsodyks, Fried, & Arieli, 2016; White & Rolfs, 2016; Ziv & Bonneh, 2021) by stimulus presentation with a later release latency that depends on the stimulus properties, attention, and expectation. Whereas stimulus saliency, such as high contrast, is known to shorten the inhibition (Bonneh, Adini, & Polat, 2015), prolonged inhibition was found in response to deviants (Valsecchi, Betta, & Turatto, 2007). We have recently found that the OMI generalizes to free viewing in terms of fixation-related inhibition that depends on stimulus saliency like OMI in response to flashed stimuli (Kadosh & Bonneh, 2022c). 

To date, most of the electrophysiological studies measuring the event-related deviant response have used auditory stimuli, and only a few studies have focused on visual responses, or have studied the later ERP P300 voluntary response (Hiebel et al., 2018). Here, we studied the fixation-triggered negativity (N1) and the timing of saccades, in response to a visual deviant. As in previous auditory MMN studies that used visual stimuli to draw attention away from the auditory stimuli, we applied a visual oddball paradigm in guided viewing in which participants were instructed to look for a superimposed small target dot presented statically for several seconds, while serially inspecting a line of items in which they could encounter a deviant inverted item irrelevant to their task. We expected to find a longer OMI and increased N1 negativity for the deviant as we previously found for the transient oddballs. 

Methods

Participants

A total of 28 participants were recruited for the study: 15 women and 13 men, ages 18 to 46 years with a mean of 30 years. Two participants were excluded from the study, because one could not maintain fixation long enough to pass the eye-tracking calibration process and the other could not perform the task. All participants had normal or corrected-to-normal vision and were naïve to the purpose of the study. The experiments were approved by the Bar-Ilan University Internal Review Board (IRB) Ethics Committee. All participants gave written informed consent, and all the experiments were conducted according to the IRB guidelines. 

Apparatus

We combined eye tracking and electrophysiological recordings in guided viewing synchronized by a common trigger to both systems. A wireless eight-channel headset with dry electrodes (Cognionics, Inc.; Kim, Yoshimura, & Koike, 2019) was used for the EEG recordings and the Eyelink 1000 plus device (SR Research) was used for eye tracking, both with a sampling rate of 500 Hz. A 100 Hz calibrated 24-inch FHD LCD monitor (Eizo Foris fg2421) was used, and the experiments were run using the in-house developed integrative stimulus presentation and analysis tool (PSY) developed by Y.S. Bonneh. Stimuli were displayed at 0.6 m. A 35 mm lens was used with the eye tracking camera positioned approximately 0.52 m from the participant's head, which was stabilized using a forehead and chin rest. The experiments were conducted in dim light. All recordings were performed binocularly, with analyses on data from the left eye. A standard nine-point calibration was performed before each session. 

Stimuli and procedure

Participants were instructed to serially inspect from right to left a stimulus that was statically presented for 5 seconds per trial and to look for a tiny bright dot that could be present on one of the E letters either in the fifth or sixth positions in 25% of the trials (the target condition). In the standard condition, all the Es in the bright circles face the same direction. Both directions were tested, each in two separate runs, randomly for each participant. Finally, in the deviant condition, a task irrelevant directional mismatched E appeared in one of the fifth or sixth positions, also in 25% of the trials. With a known typical average saccade rate of two to three per second, the participants had no trouble to complete the trials ahead of time. None of them reported seeing the upcoming deviant ahead. The participants completed a total of 160 trials in four runs. At the end of each run, they had to report the number of bright-dot targets that they saw. We analyzed fixations that were induced by leftward saccades and categorized the epochs as standard, deviant, or target only for fixations over the fifth or sixth positions. 

Data analyses

Saccade/microsaccade detection

For the saccade detection, we used the algorithm introduced by Engbert and Kliegl (Engbert & Kliegl, 2003), which is based on eye movement velocity. Saccades were detected as movements exceeding eight SDs of the mean velocity in 2D velocity space, as in Rosenzweig and Bonneh (2019) and Yablonski, Polat, Bonneh, and Ben-Shachar (2017). The detected microsaccades had a velocity range of 8 degrees/second to 150 degrees/second, an amplitude range of 0.08 to 2 degrees, and a minimum duration of 9 ms. The upper limit size of the fixational microsaccades was determined by the size of the circular place holders, which was 2 dva. 

Calculation of the saccade/microsaccade reaction time

Eye tracking epochs were extracted, triggered by the saccade (2–12 dva) landing time in a range of −0.1 second to 0.9 seconds relative to fixation onset, with a possible overlap between epochs. This overlap was taken into consideration when computing the microsaccade reaction time (msRT) of one fixation by ignoring microsaccades that occur later than the following fixation onset. The msRT and the saccade RT (termed here, fixation duration) were calculated for each epoch as the latency of the first microsaccade or saccade relative to the fixation onset. The mean across participants of epochs with a microsaccade occurrence was 74% ±13 SD. The RTs were averaged across the epochs of each condition within observers and then averaged across observers, with error bars computed across observers on demeaned (within observer) data, with a correction factor (multiplied by √ (n/ (n−1)). This method for computing the error bars enables a better representation of within-participant effects (Cousineau & Morey's method; Morey, 2008; see also Bonneh et al., 2015). 

Fixation-related potentials

The EEG data were filtered using a 0.1 Hz high-pass and 30 Hz low-pass cutoffs. The EEG headset had two built-in reference channels placed on both earlobes. FRP epochs were created, as was done for the eye-tracking data, triggered by the saccade (2–12 dva) landing time in a range of −0.15 seconds to 0.25 seconds relative to the fixation onset to minimize overlapping data between epochs. Overlapping data points with neighboring fixations were excluded in both epochs. Extreme value artifacts were rejected by denoting them as “bad blocks”: those time regions around amplitudes exceeding a ±150 µVolt threshold measured at the Mean Global Field Potential (MGFP) and +75 µVolts at the frontal channels (F7, Fz, and F8), and excluding samples at a time range of −200 to 200 and −500 to 200 ms, respectively. We focused on the N1 negativity, which is interpreted here as a mismatch-negativity response. It is calculated within a time range of 100 to 220 ms and is baseline corrected to the mean value of −150 ms to 50 ms, following the fixation-onset data points. This was done to overcome a possible shift in the post fixation-onset baseline resulting from the inducing saccade spike potential (SP). Other baselines, such as −150 to −75 ms, preceding the SP also produced significant N1 differences between standard and deviant stimuli; however, the waveforms were less comparable due to the baseline shift. 

Peak extraction was optimized by setting an individual time range for each observer around their average peak latency, within the predefined time range, from all conditions combined; see Alsufyani, Hajilou, Zoumpoulaki, Filetti, Alsufyani, et al. (2019) for a similar method. This was done to avoid using a long time range to overcome the latency differences across observers, which would increase the false peak discoveries. We also excluded peak magnitudes exceeding ±50 µVolts. An additional N1 peak analysis was performed with the exclusion of epochs containing a saccade or a microsaccade that occurred less than 220 ms post fixation onset (termed MS-free epochs) to avoid peak corruption and to ensure that the increased negativity is not related to the difference in saccade latencies between conditions. 

Statistical assessment

To assess the significance of the differences between the target or deviant from the standard FRP waveforms, we used nonparametric permutation tests (Monte-Carlo, as in Widmann, Engbert, & Schroger, 2014; see also Maris & Oostenveld, 2007). First, we looked for a continuous block of a significant difference between two conditions using paired t-tests at each time point. Then, the condition labels of the participants’ mean at each time point were randomized and the group averages were recalculated to generate 1000 permutations. Next, the first step was repeated. The p value was computed as the fraction of permutations in which the original test statistic was exceeded. The statistical analyses of the N1 group average comparisons were performed using two-tailed paired t-tests using Matlab. We also calculated the Cohen's d effect size and analyzed the “area under the curve” (AUC) of the “receiver operating characteristic curve” (ROC), which shows the balance between the true and false positive rates and is a popular tool for assessing the classification performance. 

Results

Our goal was to study the visual mismatch-negativity response and the accompanying OMI in natural viewing conditions, compared to the un-natural flashed stimuli at fixation. We created a paradigm in which the observer was instructed to shift his gaze over a static image, at a natural pace, between marked stimulus locations with an 8 dva distance between them (see the Stimuli and Procedure, Figure 1). The fixation-related N1 response and the saccade/microsaccade inhibition, were calculated aligned with the fixation onset triggered by large saccades (2-12 dva). 

Figure 1.

 

Stimuli and procedure. Three conditions were tested: the target, standard, and deviant, 25, 50, and 25 percent of the trials, respectively. An example of the deviant condition is presented above, with an inverted E positioned in the fifth bright circle (2 dva), counting from right to left. Standard trials had no inversion, whereas target trials had a tiny bright dot placed on the letter E in the fifth or sixth positions. The small size of the Es (0.34 dva) in the figure conforms with the actual proportions used and is intended to demonstrate the limited visibility of the non-fixated Es.

FRP results

The EEG data were collected using an eight-channel wireless EEG headset (see the Methods), with an electrode configuration illustrated in Figure 2a. Of the 5-second trials, we created epochs triggered by large saccades (2–12 dva) and analyzed the inducing saccade direction to include only leftward saccades in the analyses. We computed baseline-corrected FRPs to illustrate the mismatch response from 26 observers, averaged across observers and normalized within observers by subtracting the observers’ mean and excluding values outside 2 SD (see the Methods section). The FRP MGFP results, presented in Figure 2b₁, show increased negativity in the deviant condition (red line) at around 140 ms post fixation onset. The FRP results for individual electrodes Fz, Cz, P3, P4, O1, and O2 are shown in Figure 2b_2-7, respectively. The main deviant N1 negativity effect was found via electrode P4, Figure 2b₅ (p = 0.018, permutation tests, see the Methods section), and the strongest target (the purple line) N1 negativity effect was found via the Cz electrode, Figure 2b₃ (p = 0.0001, permutation tests). Figure 2c_1-5 illustrates the difference between the standard and deviant conditions in different electrodes, with the most prominent MMN response found via the P4 electrode, Figure 2c₄. 

Figure 2.

 

FRP waveforms. (a) Scalp EEG channel topography. (b) FRP waveforms for all three conditions, standard, deviant, and target, denoted in blue, red, and purple, respectively. These conditions were averaged across observers, but first were demeaned within observers and then adjusted with the grand average of all conditions and observers. The red and purple boxes denote significant differences in the deviant or target compared with the standard assessed via Monte Carlo permutation tests (see the Methods section). The dashed line in (b₇) denotes the fixation onset and the gray labels denote the electrophysiological EEG components. The pre-fixation saccade-induced spike potential is denoted as “SP.” The results show a robust increased negativity for the deviant condition with a significant difference, compared with the standard over the P4 channel (see plot b₅), p = 0.018, Monte Carlo permutation tests. (c) FRP waveforms of the deviant subtracted from the standard, showing the strongest visual MMN response via electrode P4 at around 120 to 160 ms post fixation-onset.

We calculated the fixation-related N1 magnitude with a baseline correction from −150 to +50 ms post fixation-onset (see the Methods section). Using a different baseline, for example, corrected to the period before the saccade-triggered spike potential (Keren, Yuval-Greenberg, & Deouell, 2010) prior to the fixation onset, we found similar results but with reduced significance. The N1 magnitudes were averaged across observers (N = 26) and demeaned within observers, excluding values outside 2 SD. Figures 3a to 3d shows comparisons between the N1 magnitudes of the standard and the deviant conditions. As in the FRP waveforms, the most significant effect was found via the P4 electrode (see Figure 3c), p = 0.003, paired t-test with a calculated large effect size (ES) of 1.09, Cohen's d and an AUC = 0.8 value of the ROC curve. The deviant N1 negativity effect was also significant via the MGFP and O2 electrode, p = 0.025, ES = 0.78, and AUC = 0.71, as well as p = 0.005, ES = 0.9, and AUC = 0.76, respectively. 

Figure 3.

 

The FRP N1 negativity effect. Paired t-test comparisons between the standard, denoted in blue, and the deviant, denoted in red. N1 magnitudes were averaged across observers and demeaned first within observers. N1 magnitudes with all epochs included; excluding epochs with saccade or microsaccade occurrences before 220 ms post fixation-onset. (a) FRP N1 magnitude of the “mean global field potential” (MGFP), showing a significantly larger magnitude for the deviant, p = 0.025. With the exclusion of epochs with saccades in (e), the difference was not significant, p = 0.083. (b) Fz FRP N1 magnitudes show no significant difference, also with the exclusion of epochs with saccades in (f). (c) P4 FRP N1 magnitudes show significantly larger magnitudes for the deviant, p = 0.003 and p = 0.002, in (g). (d) O2 FRP N1 magnitudes also show a significantly larger magnitude for the deviant, p = 0.005 and p = 0.034, in (h). The p values (p), effect size (ES), and “area under the ROC curve” (AUC) are shown for each plot.

To ensure that this significant effect was not affected primarily by distortions triggered by saccades or microsaccades, we excluded epochs contaminated by saccade/microsaccades occurring in the relevant time range for the calculation of the N1 negativity (e.g. before 220 ms). The results are shown in Figures 3e to 3h, with a significant difference between the standard and deviant N1 magnitudes found for the P4 and O2 electrodes, p = 0.002, ES = 1.02, and AUC = 0.77, as well as p = 0.034, ES = 0.62, and AUC = 0.68, respectively. Two observers were removed from the P4 analysis and one from the O2 analysis, due to missing data in the deviant condition, after excluding the contaminated epochs by saccades. The N1 negativity effect was not significant via the Fz electrode (see Figures 3b, 3f). 

OMI results and their relationship with the FRP N1 magnitude

A previous oddball study, which used transient stimuli (Valsecchi et al., 2007; Valsecchi, Dimigen, Kliegl, Sommer, & Turatto, 2009), found prolonged inhibition for the deviant condition. Here, to account for the oculomotor inhibition, we calculated the interval between the current fixation onset and the following saccade (fixation duration) or msRT. As in the previous study, a significantly longer OMI was found for the deviant, compared with the standard condition for both fixation duration and msRT, p = 0.004, ES = 0.74, and AUC = 0.68, as well as p = 0.002, ES = 0.93, and AUC = 0.76, respectively (see Figures 4a, 4b). For the fixation duration results, epochs with microsaccades were excluded from the calculations, and only MS-free fixations were included. Two observers with no MS-free fixations in the deviant condition were excluded. Finally, we examined a possible link between the FRP and OMI measures. Epochs with saccades or microsaccades occurring up to 150 ms after the fixation onset were excluded because of the timing of the N1 negative deflection at around 140 ms. Only epochs from the deviant condition were included. With the standard and the target conditions, which are less relevant, the results also yielded a positive relation, but it was not significant (data not shown). The correlation results for the deviant condition are shown in Figures 4c to 4f, with each dot referring to the average of one observer. A positive correlation was found between N1 measured over the P4 electrode and both the msRT and the fixation duration (saccade RT), R = 0.38 (Pearson's correlation), p = 0.05 (see Figure 4c) and R = 0.55, p = 0.005 (see Figure 4d), respectively. A positive correlation was also found between N1 measured over the O2 electrode and the msRT, R = 0.40 (Pearson's correlation), and p = 0.04 (see Figure 4e); however, the relation with the fixation duration did not yield significance (see Figure 4f). 

Figure 4.

 

OMI results. (a) A paired t-test comparison between the standard, denoted in blue, and the deviant, denoted in red, fixation durations averaged across observers (N = 24), including only MS-free fixations and showing a significantly longer OMI for the deviant, p = 0.004. (b) The same as in a but for microsaccade RT, calculated in a time range of 150 to 500 ms, disregarding corrective saccades, showing significantly prolonged inhibition for the deviant, p = 0.002. (c) A scatter plot with a dot for one observer, showing a positive correlation (R = 0.38, Pearson's correlation, p = 0.05) between the P4 FRP N1 magnitude and the msRT. (d) The same as in c but for fixation duration (R = 0.55, Pearson's correlation, p = 0.005), with one removed outlier. (e) The same as c but for O2 N1 magnitudes, with a significant positive correlation (R = 0.40, Pearson's correlation, p = 0.04). (f) The same as in d but for O2 N1 magnitudes, showing a nonsignificant relation. The p values (p), effect size (ES), and “area under the ROC curve” (AUC) are shown for each plot.

Discussion

We investigated how a task-irrelevant visual deviant, in natural but guided viewing, affected the early fixation-related brain potentials, and the fixation-related oculomotor inhibition. We expected to find increased negativity for the deviant at the time of the N1 component 100 to 200 ms after the fixation onset, as found in previous ERP studies that aimed to establish a vMMN response similar to the auditory MMN (see a review by Stefanics et al., 2014). We also expected increased latencies of the following saccade or microsaccade, as found in previous oddball studies that used transiently flashed stimuli (Valsecchi et al., 2007). Our results revealed, as expected, an increased negativity of the fixation-related N1 magnitude and a longer fixation-related OMI for the deviant, with a positive correlation between the two, suggesting that these two measures could serve as markers of prediction-error in future free-viewing studies. 

FRP-N1 response as a marker of prediction error

Our results showed increased negativity for the deviant stimuli with a latency of around 120 to 160 ms post-fixation onset (see Figure 2, Figure 3). This is consistent with previous vMMN studies (see a review by Stefanics et al., 2014) that investigated the response to infrequent color patterns (Czigler et al., 2004), low spatial frequency gratings (Cleary et al., 2013), and face and house orientations (Zhang et al., 2018). Even facial expressions and emotions were used in an oddball paradigm and elicited vMMN peaking at around 100 to 200 ms (Astikainen et al., 2013; Gayle et al., 2012; Kreegipuu et al., 2013). A more recent study that measured the N170 face response after a preview found a stronger N170 for an invalid preview, which is reminiscent of the vMMN (Huber-Huber, Buonocore, Dimigen, Hickey, & Melcher, 2019). A few studies reported negativity with a later latency beyond the 200 ms post stimulus onset (Kreegipuu et al., 2013; Shtyrov, Goryainova, Tugin, Ossadtchi, & Shestakova, 2013). In our previous attempt to measure the deviant vMMN (Kadosh & Bonneh, 2022b), the participants (N = 16) were instructed to look for the deviant. We used stimuli similar to those in the current study, and the results showed N1 and N2 effects similar to those found here for targets (see Figure 3). However, we did not report these results because they may reflect, at least partially, the allocation of attention to the task-relevant deviant rather than the automatic deviance detection typically linked to MMN. The ERP MMN response is considered an automatic change detection process. In the current study, a task was used to draw attention away from the tested visual deviant stimulus. The few studies that tested the automaticity of vMMN reported that it emerged regardless of whether the deviant was attended to or not (van Rhijn, Roeber, & O'Shea, 2013), and that it was unaffected by the task difficulty (Kremláček, Kuba, Kubová, Landrová, Szanyi, et al., 2013) or had increased latency in relation to the task difficulty (Kimura & Takeda, 2013), which may suggest some dependency on attention. In contrast, another study found increased posterior negativity only for an unattended change or a later response at 250 to 400 ms, suggesting that the task relevance may eliminate earlier differences (Kuldkepp, Kreegipuu, Raidvee, Näätänen, & Allik, 2013), as suggested by hierarchical predictive coding theory, which indicates that the early MMN response may be reduced by higher formed regularities (Dehaene, Meyniel, Wacongne, Wang, & Pallier, 2015; Wacongne, Labyt, van Wassenhove, Bekinschtein, Naccache, & Dehaene, 2011). Here, we did not examine the later responses, such as N2b and P300, partly to avoid using the more complex deconvolution methods to disentangle the visual components from the eye movement artifacts (Dimigen, 2020; Dimigen & Ehinger, 2019; Ehinger & Dimigen, 2019). Those late responses could result from an overlap with the spike potential or the P1 and N1 components induced by the following saccades or microsaccades (Dimigen et al., 2011; Keren et al., 2010; Nikolaev, Nakatani, Plomp, Jurica, & van Leeuwen, 2011), which differed in their latency between conditions in the current study (see Figures 4a, 4b). Instead, we investigated early responses that are less prone to those artifacts and further tested and found that the exclusion of epochs contaminated by microsaccades and saccades at the relevant N1 time range did not affect the results. Moreover, the epoched data were based on saccades that were relatively similar in size and direction, which also eliminated additional noise induced by different saccade sizes. 

OMI as a marker of prediction error

Rare visual or auditory events are known to induce a longer OMI. Our results indicated a robust prolonged fixation-related OMI in response to the deviant (see Figures 4a, 4b), as previously found in oddball studies that used transient flashed stimuli. For example, prolonged microsaccade inhibition has been reported for oddballs in a sequence, a rare blue patch among frequent red patches (Valsecchi et al., 2007), and for auditory oddballs (Kadosh & Bonneh, 2022d; Valsecchi & Turatto, 2009; Widmann et al., 2014). More preliminary evidence of prolonged inhibition was found for high-contrast patches among low-contrast patches (Bonneh et al., 2013) and for temporal oddballs of unpredicted intervals (Bonneh, Polat, & Adini, 2016). Interestingly, our results revealed a positive correlation between the FRP N1 negativity and the fixation-related OMI, as indicated by the observer scatter plots for both saccades and fixational microsaccades (see Figures 4c-4f). We reported a similar correlation in our recent study, where participants freely viewed images of both familiar and unfamiliar face images (Kadosh & Bonneh, 2022a), however, it was achieved by eliminating the interobserver variance of both the FRP N1 and OMI measures. When repeating the same method here (data not shown), by demeaning (subtracting the average across all conditions) within observer, we found a similar correlation as in Figure 4 (for example: msRT and O2N1 correlation, R = 0.43, p = 0.03, similar to Figure 4e), indicating that the covariance of those two measures was due to the deviant response and not due to the variance in participants signal strength or saccadic response speed. This suggests that these measures could serve as markers of prediction-error in future free-viewing studies. 

Conclusions

We have proposed a visual paradigm in which items are fixated serially instead of flashed to the observer's fixation area in order to study the ocular and electrophysiological markers of task-irrelevant deviance. The results showed for the first time in natural, but guided viewing, a significantly increased fixation related N1 negativity magnitude for the deviant at the posterior sites (vMMN), accompanied by longer saccadic inhibition (OMI), with a positive correlation between those measures. Both measures are known to be indicative of prediction error (PE). Overall, the results indicate the sensitivity of the fixation-related potentials and OMI to rare task-irrelevant deviance and suggest that these two signals combined can serve as markers of PE in future free-viewing studies. 

Acknowledgments

Author Contributions: O.K. and Y.S.B. designed the experiments. O.K. collected the data. O.K. and Y.S.B. developed the software used for running the experiments and the data analysis. O.K. analyzed the data and wrote the manuscript, and Y.S.B. reviewed it. 

Data Availability: The experimental datasets generated during the current study will be available from the corresponding author upon reasonable request. 

Commercial relationships: none. 

Corresponding author: Yoram S. Bonneh. 

Email: [email protected]. 

Address: School of Optometry and Vision Science, Faculty of Life Sciences, Bar-Ilan University, Ramat-Gan 52900, Israel.