Detection of object displacement during a saccade is prioritized by the oculomotor system

Abstract

The human eye-movement system is equipped with a sophisticated updating mechanism that can adjust for large retinal displacements produced by saccadic eye movements. The nature of this updating mechanism is still highly debated. Previous studies have demonstrated that updating can occur very rapidly and is initiated before the start of a saccade. In the present study, we used saccade curvature to demonstrate that the oculomotor system is tuned for detecting object displacements during saccades. Participants made a sequence of saccades while ignoring an irrelevant distractor. Curvature of the second saccade relative to the distractor was used to estimate the time course of updating. Saccade curvature away from the presaccadic location of the distractor emerged as early as 80 ms after the first saccade when the distractor was displaced during a saccade. This is about 50 ms earlier than when a distractor was only present before a saccade, only present after a saccade, or remained stationary across a saccade. This shows that the oculomotor system prioritizes detection of object displacements during saccades, which may be useful for guiding corrective saccades. The results also challenge previous views by demonstrating the additional role of postsaccadic information in updating target–distractor competition across saccades.

Introduction

In humans, high-resolution vision is restricted to the fovea, which covers only a small part of the visual world (about 1 visual degree). Fortunately, humans developed a sophisticated eye-movement system capable of quickly, “just in time” orienting the fovea to the objects of interest. Because the visual system is retinotopically organized, the eye-movement system should be equipped with an updating mechanism that can adjust for retinal displacements produced by eye movements. The nature of such an updating mechanism is a matter of a long-standing debate with some researchers arguing for updating occurring slowly after each saccade (Golomb, Chun, & Mazer, 2008; Golomb & Kanwisher, 2012; Lescroart, Kanwisher, & Golomb, 2016; Mathôt & Theeuwes, 2010; Zimmermann, Morrone, Fink, & Burr, 2013) and others proposing that updating occurs rapidly, starting already before the start of a saccade (Arkesteijn, Belopolsky, Smeets, & Donk, 2019; He, Mo, & Fang, 2017; Melcher, 2007; Neupane, Guitton, & Pack, 2016; Rolfs, Jonikaitis, Deubel, & Cavanagh, 2011; Schut, Van der Stoep, Fabius, & Van der Stigchel, 2018, but see Arkesteijn et al., 2019). 

One classic phenomenon suggesting the existence of such anticipatory updating is called saccadic suppression of displacement or SSD (Bridgeman, Hendry, & Stark, 1975; Deubel, Schneider, & Bridgeman, 1996). This phenomenon entails a demonstration that, when objects (including saccade target) are displaced during a saccade, the displacement often goes unnoticed by observers (Atsma, Maij, Koppen, Irwin, & Medendorp, 2016; Bridgeman et al., 1975; Deubel et al., 1996; Deubel, Wolf, & Hauske, 1986; Hollingworth, Richard, & Luck, 2008; McLaughlin, 1967). When the saccade target is displaced during the eye movement, the observer tends to direct the initial saccade to the original target location, followed by a fast (under 100 ms) corrective saccade to the new target location. These results indicate that the oculomotor system is capable of rapidly detecting target displacements. 

The goal of the present study was to understand the mechanism underlying such rapid detection of displacements that allows the oculomotor system to efficiently keep track of objects across saccades. We accomplish this by combining SSD with the method of measuring trajectories of saccadic eye movements, which allows us to implicitly, without interfering, track active representations in the oculomotor system (Belopolsky & Theeuwes, 2011; Boon, Belopolsky, & Theeuwes, 2016; Jonikaitis & Belopolsky, 2014; McPeek, 2003; Tudge, McSorley, Brandt, & Schubert, 2017; van Leeuwen & Belopolsky, 2018). 

Saccade curvature is thought to reflect oculomotor competition in the superior colliculus (Doyle & Walker, 2001; McPeek, 2003; McSorley, Haggard, Walker, Harwood, & Herman, 2006). Research has shown that, when target and distractor are presented simultaneously, saccades tend to curve toward distractors if they are initiated within 200 ms and tend to curve away from the distractor if initiated after more than 200 ms (McSorley et al., 2006). Using a double-step paradigm, Jonikaitis and Belopolsky (2014) found that, if a task-irrelevant distractor was flashed before the onset of the initial saccade, the second saccade curved away from the spatiotopic location, even for saccades that were initiated within 200 ms after the first saccade. Their results suggest that the oculomotor competition is rapidly updated from a retinotopic reference frame to a spatiotopic reference frame when planning multiple saccades (Boon, Theeuwes, & Belopolsky, 2019). Supporting these findings, experiments using a similar paradigm demonstrated rapid spatiotopic updating of oculomotor competition evoked by locations that were kept in memory but not physically present in the external world (Arkesteijn, Smeets, Donk, & Belopolsky, 2018; Boon, Zeni, Theeuwes, & Belopolsky, 2018). Combining the double-step saccade curvature paradigm used by Jonikaitis and Belopolsky with saccadic suppression of displacement of an irrelevant distractor, van Leeuwen and Belopolsky (2018) demonstrated that competition in the oculomotor system initially reflected the original distractor location but, after 120–150 ms, switched to reflect the new displaced distractor location. 

The rapid time course of updating the oculomotor competition suggests that it may start before or during a saccade, which is consistent with the idea of predictive remapping (Duhamel, Colby, & Goldberg, 1992; Walker, Goldberg, Fitzgibbon, & Goldberg, 1995). According to this hypothesis, the receptive fields of a subset of neurons shift their firing preference to the location that will be occupied by a stimulus after saccade is completed. In the context of the double-step paradigm, this would mean that visual information available prior to the first saccade was primarily responsible for rapid updating of the oculomotor competition. Interestingly, this conclusion is inconsistent with some of the results observed in the study by van Leeuwen and Belopolsky (2018). In that study, the onset of saccadic curvature was earlier when the distractor was displaced during a saccade compared to when a distractor remained stationary. Because visual information prior to the first saccade in these two conditions was identical, the onset of saccadic curvature should have also been identical. The earlier onset of saccadic curvature in the distractor displacement condition could indicate that an object changing location during a saccade is processed faster by the oculomotor system compared to objects that remain stationary across a saccade. 

Because these results were not predicted by our original hypothesis and were based on a limited number of trials with a short intersaccadic interval, they were not reported in the original study (van Leeuwen & Belopolsky, 2018). Here we treat these results as exploratory and present their reanalysis as Experiment 1 and the basis for Experiment 2. Experiment 2 was specifically designed to examine whether detection of object displacement during a saccade is prioritized by the oculomotor system. To that end, we replicated the conditions in Experiment 1, in which the distractor could either change location during a saccade or remain stationary. We have added two additional baseline conditions, presaccadic and postsaccadic distractor conditions, with distractor being present exclusively before and after the first saccade, respectively. To increase signal-to-noise ratio for the short intersaccadic intervals, we have collected twice as many trials in Experiment 2 compared to the data in the reanalysis (Experiment 1) and pushed participants to make fast sequences of double-step saccades. If distractor displacements during saccade are prioritized by the oculomotor system, we expected that early saccade curvature would only be observed in the distractor displacement condition. 

Experiment 1: Reanalysis of van Leeuwen and Belopolsky (2018)

Experiment 1 represents a reanalysis of the data from the study by van Leeuwen and Belopolsky (2018). The data from the two experiments in the original study were combined. In this reanalysis, participants performed a sequence of two saccades: one horizontal saccade followed by a vertical saccade. A task-irrelevant distractor was displayed during the trial, and on half of the trials, the distractor was displaced during the initial horizontal saccade. The goal of the current analysis was to determine whether saccade curvature away from the original, presaccadic location of the distractor emerged earlier when the distractor was displaced during a saccade compared to when the distractor remained stationary. 

Methods

Design

The experiment was a double-step saccade task (van Leeuwen & Belopolsky, 2018). The participants had to fixate on a fixation dot; after a random time interval, two black target dots and one white distractor dot were presented (Figure 1A). The participants then made a horizontal saccade to the first target and a vertical saccade to the second target. During the first saccade, the distractor either remained stationary on the screen (no-displacement condition, Figure 1B, second panel from the top) or was displaced across the imaginary straight line between the first and second targets (displacement condition, Figure 1B, first panel from the top). The saccade sequence could start from the left or the right side of the screen, and the final target was either below or above the first target. The initial distractor location was either clockwise (CW) or counterclockwise (CCW) relative to the imagined vertical line between the first and second targets. Therefore, the distractor shifted hemifield during the initial saccade on half the trials. This was the case for all four conditions. 

Figure 1

 

The trial structure for both the experiment by van Leeuwen and Belopolsky (2018) and the current experiment. (A) An example trial, showing the displacement condition. (B) The four conditions used in Experiment 2. In Experiment 1, only the displacement and no-displacement conditions were used. The left column reflects the position (if any) of the distractor dot before the onset of the first saccade. The second column reflects the position (if any) of the distractor dot after the first saccade. The configuration displayed is for illustrative purposes. The initial distractor could appear at either the CW or CCW position. All the distractor position changes (displacements/onsets/offsets) occurred during the initial saccade. The displays in the right column were displayed until the completion of the second saccade. In each condition, there were four possible saccade sequences: saccade sequence could start from the left or the right side of the screen, and the final target was either below or above the first target. The white and black arrows indicate saccade directions. Note that the figure is not drawn to scale. Colors are used for illustration purposes only; all stimuli had the same color in the experiment (black targets, white distractor).

Data processing

Identically to the analysis by van Leeuwen and Belopolsky (2018) saccades were defined using an acceleration threshold of 9.500°/s², a velocity threshold of 35°/s, and were automatically detected by the Eyelink system. Using custom code, all relevant events and data were extracted for each trial. The first saccade was defined as the first saccade that started after the targets were displayed. The second saccade was defined as the first saccade that started at the first target and ended at the second target (any corrective saccades after the first saccade were ignored if they started and ended within 3° of the first target dot). 

The data from the different saccade sequences were transformed to fit into the same reference frame (first saccade from left, second saccade upward). As the measure for saccade curvature is the difference between clockwise and counterclockwise curvature and that saccade curvature might differ for saccades with different saccade configurations, the data from each saccade sequence were analyzed separately before collapsing the data. After rotation, the distractor could either be in the CW or CCW position relative to the second target (see Figure 3B). 

A total of 34 participants were tested in the experiment, of which 19 participants were included in the analysis; 15 participants were rejected because they were rejected in the original data analysis, see van Leeuwen and Belopolsky (2018). Twelve criteria for rejecting trials were used. Combined over all included participants, there were a total of 40,200 trials in the experiment of which a total of 32.5% were rejected, resulting in 27,478 included trials (see Supplementary Table S1 for a detailed list of rejection criteria and the percentage of trials rejected for each criterion). Experiment and analysis code can be found online at: https://osf.io/tkefz/. 

Saccade curvature calculation

Saccade curvature was calculated as the median angle between each sample point in a saccade and an imaginary line between the start and end points of the saccade (Figure 2A). All points within 0.5° of the saccade start/end points were discarded before calculating saccade curvature. The curvature values were split into CW and CCW for each of the two conditions and saccade configurations (left-up, left-down, right-up, right-down). Curvature difference was defined as the difference between the curvature on the trials on which the distractor was presented in the CCW and CW positions, separately for each condition and configuration. After calculating the difference for each configuration, the curvature difference was collapsed for each saccade configuration by using a mean, weighted by the number of trials for each saccade configuration. A positive curvature difference reflects curvature away from the distractor location before the first saccade, and a negative curvature difference reflects curvature away from the postsaccadic distractor location (see Figure 2B for an example of curvature away from distractor). Note that all statistics and plots are done using the curvature difference between CCW and CW trials. 

Figure 2

 

(A) Illustration of saccade curvature calculation. The curvature is the median angle α of all angles between each sample point and the straight line between the start and end points of the saccade (red line), which are further away from the start and end points than 0.5° of Euclidian distance (black circle around the start and end points). (B) Calculating curvature difference, relative to CCW and CW distractors. The single value curvature difference for the conditions is the difference between the curvature in the CCW and CW trials.

Figure 3

 

The results of the reanalysis of the data from van Leeuwen and Belopolsky (2018) when using a σ of 10 ms and smoothing the intersaccadic interval between 50 and 300 ms. (A) The saccade curvature (degrees of arc) smoothed as a function of intersaccadic interval. Positive curvature values reflect curvature away from the presaccadic distractor location, and negative curvature values reflect curvature away from the postsaccadic distractor location; see Figure 1B, left and right panels, respectively. The thick lines reflect points of significant difference from zero. Asterisks indicate significant clusters after cluster-based permutation testing. Error bars show the weighted 95% confidence intervals. The cyan and magenta curves reflect smoothed data for the no-displacement and displacement conditions, respectively. (B) The cluster-based permutation results for each of the four conditions. The cyan and magenta bars reflect the frequency (log) of cluster size in the permutations for the no-displacement and displacement conditions, respectively. The red line indicates the permuted 95th percentile cluster value, and the black lines reflect the clusters found in the smoothed data. (C) The kernel density estimate of the distribution of intersaccadic intervals for all conditions and all participants. Showing the estimated trials per millisecond of intersaccadic interval with a σ of 10 ms. (D) The p values for each individual time point for each of the two conditions. The red horizontal line reflects p = 0.05. Asterisks indicate significant clusters after cluster-based permutation testing. The cyan data plots show the no-displacement condition, and the magenta data plots show the displacement condition. Plots A and C are aligned such that the frequency distribution of intersaccadic intervals follow the smoothed graph in panel A. This clearly shows that there is very little data with intersaccadic intervals less than 100 ms, which is also illustrated with the large confidence intervals in panel A.

Gaussian smoothing

We smoothed the curvature using the SMART method (van Leeuwen, Smeets, & Belopolsky, 2019). To get a precise estimate of the time course of saccade curvature as a function of intersaccadic interval, a moving Gaussian window between 50 and 300 ms (step size 1 ms and σ = 10 ms) was used. A smoothed curvature time series was made for each of the CCW and CW trials in each of the two conditions and saccade configurations, separately for each individual. The smoothed time series were collapsed over saccade configurations using a mean, weighted by the number of trials for each saccade configuration, and then collapsed over participants using a mean, weighted by the number of trials for each participant. This resulted in an averaged smoothed curvature time series for each condition. 95% confidence intervals were calculated for each condition (see van Leeuwen et al., 2019) for a detailed description of the smoothing method. 

Cluster-based permutation testing against baseline

To determine when there was significant curvature, a weighted, one-sample t test was done for each time point of the Gaussian smoothed data. Clusters of significant differences were defined as two or more consecutive time points with p < 0.05, and the size of each cluster was defined as the sum of t values in the cluster. To control for multiple comparisons, cluster-based permutation testing was used to determine cluster significance. The permutations were done separately for each condition and participant. The curvature data was divided into CW and CCW trials for each saccade configuration. Then the data was permuted and smoothed and curvature difference was calculated as previously described. Then the four saccade configurations were collapsed using a mean, weighted by the number of trials for each saccade configuration. This procedure was repeated 1,000 times for each condition and saccade configuration. The sum of t values for the largest cluster size in each permutation was used to build the permutation distribution. The p value for each of the clusters found in the nonpermutated data is the proportion of clusters in the permutation distribution with equal or larger sum of t values than the clusters in nonpermutated data (see van Leeuwen et al., 2019, for a detailed description of the smoothing and permutation method). 

Software

Data processing and statistics were done with Python (Continuum Analytics, 2016). Figures were made using Python (Continuum Analytics, 2016) and Adobe Illustrator (CS5.1). Experiment and analysis code can be found online at: https://osf.io/tkefz/. 

Results

One-sample, weighted t tests and cluster-based permutation testing was used to determine when each of the conditions differed significantly from baseline, which, in the case of saccade curvature, is zero curvature (Figure 3). The results were similar to the results in van Leeuwen and Belopolsky (2018). The no-displacement condition showed one significant cluster; the cluster lasted between the intersaccadic interval of 108 and 300 ms, p < 0.001, with curvature away from the distractor location. The displacement condition shows two significant clusters. The first significant cluster lasted between the intersaccadic interval of 81 and 168 ms, p < 0.001, with curvature away from the presaccadic distractor location. The second significant cluster lasted between the intersaccadic interval of 191 and 300 ms, p = 0.001, with curvature away from the displaced distractor location. 

The goal of the present reanalysis was to determine the earliest significant difference for the two conditions—more specifically, whether the first significant cluster of curvature in the displacement condition occurs earlier than in the no-displacement condition. The results show that the first cluster of significant curvature starts approximately 30 ms earlier in the displacement condition compared to the first significant cluster of curvature in the no-displacement condition. However, the intersaccadic interval distribution shows that the majority of the trials, 25,812, has an intersaccadic interval of more than 100 ms with only 1,586 trials with intersaccadic intervals less than 100 ms (Figure 3C). The difference in onset time of the first cluster of the no-displacement and displacement conditions should, therefore, be interpreted with caution. 

Discussion

First, the reanalysis of the data from van Leeuwen and Belopolsky (2018) showed the same results as the original findings even though different parameters where used for the analysis. The intersaccadic interval range was changed from 100–300 to 50–300 ms, and the Gaussian kernel width was changed from 8 to 10 ms. As expected, the oculomotor system switched from representing presaccadic distractor competition to postsaccadic competition after an intersaccadic interval of approximately 180 ms. Second, the results show that the oculomotor competition in the displacement condition emerges 27 ms earlier compared to the no-displacement condition. This suggests that the oculomotor system is preferentially prioritizing objects that either do not match predictions (presaccadic locations) or are displaced relative to the predicted location. 

Experiment 2

The results from Experiment 1 show that the oculomotor competition for the presaccadic distractor location arises approximately 30 ms earlier in the displacement condition compared to the no-displacement condition. This suggests that the oculomotor system prioritizes detection of object displacement over detection of stationary objects across saccades. However, Experiment 1 was not designed to test this hypothesis, and there were relatively few trials around the initial onset of competition. Additionally, the early onset of competition in the distractor displacement condition might be caused by other aspects of the experimental design, such as the sudden disappearance of the distractor from the initial location or the sudden appearance of the distractor after the first saccade. Experiment 2 was specifically designed to address these issues by focusing the data collection on the early intersaccadic interval by collecting more data per participant and pushing participants to perform fast double-step saccades. The total number of trials per condition was doubled compared to the number of trials per condition in Experiment 1. We also added two new baseline conditions: a condition with only a presaccadic distractor and a condition with only a postsaccadic distractor. 

Methods

Participants

Thirty-four naïve participants and one of the authors, JvL, took part in the experiment. To ensure enough trials with intersaccadic intervals less than 100 ms, the experiment consisted of three sessions, each lasting approximately 1.5 hr. Each of the sessions was done on separate days. One participant chose not to participate in the third session. One participant was excluded due to bad data quality, and 14 participants were either unable to pass the screening (see stimuli, design, and procedure) or were unable to maintain a stable calibration. Nineteen participants, 10 females (mean age: 20) and nine males (mean age: 22), were included in the experiment. Participants received either money (8€ per hour) or credits (60 credits per hour) as compensation for their time. All participants had normal or corrected-to-normal vision; the details of the experiment were explained and they gave their informed consent prior to participating. The experiment was conducted with approval of the local ethics committee of the Vrije Universiteit Amsterdam (VCWE) under protocol number: VCWE-2016-215; all rules, regulations, and guidelines were followed. 

Setup and calibration

All three sessions were conducted in the same dimly lit room. The stimuli were presented on a 21-in. LCD monitor (Samsung 2233RZ) with a resolution of 1,680 × 1,050 with a refresh rate of 120 Hz. Participants viewed the screen from a distance of 70 cm. A chin and forehead rest ensured that the correct head position was maintained during the experiment. Left-eye gaze position was recorded with the Eyelink 1000 (SR Research), sampling at 1,000 Hz. A nine-point automatic calibration was used and repeated until the max validation offset was less than 1° and the average validation offset was less than 0.5°. The participant was alone in the room during the calibration. The calibration and experiment backgrounds were always gray (40 cd/m²). The calibration and experiment target dots were always black (0 cd/m²). 

Stimuli, design, and procedure

The experiment was almost identical to the experiment in van Leeuwen and Belopolsky (2018) but was reprogrammed in Python (Continuum Analytics, 2016) using Psychopy3 (Peirce, 2007). The stimuli consisted of one fixation dot, two saccade target dots, and a distractor dot. The fixation and target dots were black and had a radius of 0.15°. The fixation dot was presented 10° to the left or right of the screen center. The first saccade target was always in the center of the screen. The second saccade target was presented 10° above or below the first saccade target (Figure 2). The distractor was a white dot (122 cd/m²) with a radius of 0.375°. The vertical distance between the distractor dot and the first saccade target dot was 5°. The horizontal distance between the distractor dot and the first saccade target was ±0.75°. In the displacement condition, the distractor dot was displaced 1.5° during the saccade, crossing the imaginary straight line between the saccade targets (see Figure 1). 

We used a 4 (distractor: displacement vs. postsaccadic vs. no-displacement vs. presaccadic) × 4 (saccade direction) × 2 (distractor position: CW vs. CCW, relative to the second saccade target) design, manipulated orthogonally within subjects in each block. The participants were instructed to make a sequence of horizontal and vertical saccades (see Figure 1). They were told that a white distractor dot would appear and were instructed to ignore it. In each session, the participants first performed a practice block (64 trials) followed by 10 experimental blocks. All blocks were identical and consisted of 160 trials. 

Each trial started with participants fixating the dot on either the left or the right side of the screen. The trial started when fixation was detected. The start of fixation was defined as the first gaze sample in a trial that was less than 3° of Euclidian distance from the fixation dot. After fixating the dot for 1,000 ms, a white distractor dot was presented (except for the postsaccadic condition); 50–150 ms after the presentation of the distractor, the first and second saccade target dots were presented simultaneously. The participants were instructed to keep fixating the fixation dot until the target dots were presented, then to look at the first (center) and second (top or bottom) saccade targets as quickly and accurately as possible. The distractor displacement was triggered if the distance between gaze position and fixation dot was greater than 4° and the distance between gaze position and the first saccade target was less than 5.5°. This was done to ensure that the displacement only happened during a saccade toward the first target. If the first saccade was directed toward the distractor at the moment the distractor displacement was triggered, the trial was aborted and the distractor dot was removed. The trial ended when gaze was within 3° of the second saccade target or until trial time-out (1,500 ms after onset of saccade targets). All four conditions where identical except for the following: (a) In the displacement condition, the distractor was displaced during the first saccade. (b) In the postsaccadic condition no distractor was present before the first saccade but was presented during the first saccade. (c) In the no-displacement condition, the distractor remained in the same location across the saccade. (d) In the presaccadic condition, the distractor was removed during the first saccade. See Figure 1A for the illustration of the trial time course and Figure 2B for the illustration of differences between conditions. 

A saccade was considered correct if the gaze position at any time after target onset was within 3° of the target dot. If the trial was too slow (more than 2 s) or one of the saccades was incorrect, the trial was logged as incorrect. After each trial, participants received feedback about saccade accuracy; e.g., if they made correct saccades, the saccade targets turned green (79 cd/m²), and if they were incorrect, they turned red (31 cd/m²). If the trial was correct but the second saccade landed more than 175 ms after the landing of the first saccade, auditory feedback was played immediately after the second saccade landing. The feedback was a sine wave at 2,000 Hz with a decay time of 5 ms and duration of 300 ms. After each block, participants received feedback on their performance (accuracy and saccade latency) in the preceding block. If the number of correct trials was lower than 90% or the average intersaccadic interval (time from the first saccade offset to the second saccade onset) of the correct trials was slower than 150 ms, they were prompted to do better. The first sessions doubled as a screening session. Fourteen participants were excluded after the first session because they had less than 80% correct trials: Their average duration between landing of the first saccade and landing of the second saccade was more than 175 ms, or they were unable to maintain a stable calibration. 

Data preprocessing

The data preprocessing, saccade curvature calculation, Gaussian smoothing, and cluster-based permutation testing against baseline was identical to Experiment 1. Twelve criteria for rejecting trials were used. Combined over all included participants there were a total of 91,200 trials in the experiment of which a total of 27.6% were rejected, resulting in 66,012 included trials (see Supplementary Table S1 for a detailed list of rejection criteria and the percentage of trials rejected for each criterion). In the three conditions in which the distractor location was updated (displaced, removed, or appeared) during the initial saccade, the update always happened before the end of the first saccade (Supplementary Figure S1). 

Software

The experiment was programmed in Python (Continuum Analytics, 2016) using Psychopy3 (Peirce, 2007). Data processing and statistics were done with Python (Continuum Analytics, 2016). Figures were made using Python (Continuum Analytics, 2016) and Adobe Illustrator (CS5.1). Experiment and analysis code can be found online at: https://osf.io/tkefz/. 

Results

The results of saccade curvature as a function of intersaccadic interval of all four conditions are shown in Figure 4. The distribution of intersaccadic intervals shown in Figure 4C, demonstrates that our efforts to focus the data collection on the early part of intersaccadic interval was successful. From comparison to the data distribution in Experiment 1 (Figure 3C), it is clear that the distribution has been successfully shifted to shorter intersaccadic intervals with 14,959 trials having an intersaccadic interval of less than 100 ms compared to 1,586 trials in Experiment 1. It is also clear that there is less data at the later intersaccadic intervals, 5,986 trials for the intersaccadic intervals greater than 180 ms compared to 10,706 trials in Experiment 1. 

Figure 4

 

The results of Experiment 2 when using a σ of 10 ms and smoothing the intersaccadic interval between 50 and 300 ms. (A) The saccade curvature (degrees of arc) smoothed as a function of intersaccadic interval. Positive curvature values reflect curvature away from the presaccadic distractor location, and negative curvature values reflect curvature away from the postsaccadic distractor location (see Figure 1B, left and right panels, respectively). The thick lines reflect points of significant difference from zero. Asterisks indicate significant clusters after cluster-based permutation testing. Error bars show the weighted 95% confidence intervals. The cyan, magenta, pink, and green curves reflect smoothed data for the no-displacement, displacement, presaccadic, and postsaccadic conditions, respectively. (B) The cluster-based permutation results for each of the four conditions. The cyan, magenta, pink, and green bars reflect the frequency (log) of cluster size in the permutations for the no-displacement, displacement, presaccadic, and postsaccadic conditions, respectively. The red line indicates the permuted 95th percentile cluster value, and the black lines reflect the clusters found in the smoothed data. (C) The kernel density estimate of the distribution of intersaccadic intervals for all conditions and all participants. Showing the estimated trials per millisecond of intersaccadic interval with a σ of 10 ms. (D) The p values for each individual time point for each of the two conditions. The red horizontal line reflects p = 0.05. Asterisks indicate significant clusters after cluster-based permutation testing. The cyan data plots show the no-displacement condition, the magenta data plots show the displacement condition, the pink data plots show the presaccadic condition, and the green data plots show the postsaccadic condition. Plots A and C are aligned such that the frequency distribution of intersaccadic intervals follow the smoothed graph in panel A. This clearly shows that there is more data with intersaccadic intervals less than 100 ms compared to Experiment 1.

As in Experiment 1, the data was smoothed with a σ of 10 ms. The results in Figure 4A clearly show that curvature emerges earlier in the displacement condition than in any other condition. The displacement condition had two significant clusters with both clusters showing curvature away from the original presaccadic distractor location. The first cluster lasted from 87 to 135 ms, p = 0.004. The second cluster lasted from 151 to 194 ms, p = 0.004. In the no-displacement condition, there was one significant cluster, which lasted from 136 to 224 ms, p < 0.001, with curvature away from the distractor location (presaccadic and postsaccadic location did not change). There was one significant cluster in the presaccadic condition that lasted from 139 to 184 ms, p = 0.032, with curvature away from the distractor location. The postsaccadic condition had one significant cluster, which lasted from 132 ms to 195 ms, p = 0.001, with curvature away from the distractor location. The difference in cluster onset between the displacement condition and the three other conditions were 49, 52, and 45 ms (M = 49 ms) for no-displacement, presaccadic, and postsaccadic, respectively. 

Unlike in Experiment 1, in the displacement condition, there was no significant cluster with curvature away from the new (postsaccadic) distractor location. However, the results show the same pattern with saccades curving away for the new distractor location during the later parts of the intersaccadic interval. One possibility for this discrepancy is the difference in data available for this analysis across the two experiments. Because in Experiment 2 the data collection was focused on the early part of the intersaccadic interval, less data was available for the late part of the intersaccadic interval. By using a slightly larger σ of 20 ms instead of 10 ms, we were able to compensate for the low trial count in the later intersaccadic interval (Figure 5). The results in Figure 5A clearly show that curvature emerges earlier in the displacement condition than in any other condition. The displacement condition had two significant clusters: the first cluster showing curvature away from the original presaccadic distractor location and the second cluster showing curvature away from the postsaccadic distractor location. The first cluster lasted from 82 to 201 ms, p < 0.001. The second cluster lasted from 221 to 300 ms, p = 0.002. In the no-displacement condition, there was one significant cluster, which lasted from 141 to 236 ms, p = 0.002, with curvature away from the distractor location (presaccadic and postsaccadic location did not change). There was one significant cluster in the presaccadic condition that lasted from 135 to 201 ms, p = 0.019, with curvature away from the distractor location. The postsaccadic condition had one significant cluster, which lasted from 136 to 214 ms, p = 0.007, with curvature away from the distractor location. The difference in cluster onset between the displacement condition and the three other conditions were 59, 53, and 54 ms (M = 55 ms) for no-displacement, presaccadic and postsaccadic, respectively. The estimated temporal onset differences with the change from a σ of 10 to 20 ms was 10, 1, and 9 ms (with an average difference of 7 ms) for no-displacement, presaccadic, and postsaccadic, respectively. These results replicate the findings in the reanalysis concerning the shift from curvature away from the original distractor location to curvature away from the displaced distractor location. The results also show that the main finding of earlier saccade curvature in the displacement condition was not dependent on the chosen σ value. 

Figure 5

 

The results of Experiment 2 when using a σ of 20 ms and smoothing the intersaccadic interval between 50 ms and 300 ms. (A) The saccade curvature (degrees of arc) smoothed as a function of intersaccadic interval. Positive curvature values reflect curvature away from the presaccadic distractor location, and negative curvature values reflect curvature away from the postsaccadic distractor location (see Figure 1B, left and right panels, respectively). The thick lines reflect points of significant difference from zero. Asterisks indicate significant clusters after cluster-based permutation testing. Error bars show the weighted 95% confidence intervals. The cyan, magenta, pink, and green curves reflect smoothed data for the no-displacement, displacement, presaccadic, and postsaccadic conditions, respectively. (B) The cluster-based permutation results for each of the four conditions. The cyan, magenta, pink, and green bars reflect the frequency (log) of cluster size in the permutations for the no-displacement, displacement, presaccadic, and postsaccadic conditions, respectively. The red line indicates the permuted 95th percentile cluster value, and the black lines reflect the clusters found in the smoothed data. (C) The kernel density estimate of the distribution of intersaccadic intervals for all conditions and all participants. Showing the estimated trials per millisecond of intersaccadic interval with a σ of 20 ms. (D) The p values for each individual time point for each of the two conditions. The red horizontal line reflects p = 0.05. Asterisks indicate significant clusters after cluster-based permutation testing. The cyan data plots show the no-displacement condition, the magenta data plots show the displacement condition, the pink data plots show the presaccadic condition, and the green data plots show the postsaccadic condition. Plots A and C are aligned such that the frequency distribution of intersaccadic intervals follow the smoothed graph in panel A.

Discussion

The results clearly replicate the findings from Experiment 1 by showing that the oculomotor competition in the displacement condition emerges earlier than in the no-displacement condition: about 50 ms. The results also show that onset of oculomotor competition in the presaccadic and postsaccadic distractor conditions emerges around the same time as in the stationary distractor condition. These findings suggest that distractor displacements occurring during a saccade were preferentially processed compared to distractors that did not change spatial location during a saccade. It appears that, when the distractor is displaced, the oculomotor competition is driven by early postsaccadic visual information, compared to later postsaccadic visual information when the distractor does not change location. 

General discussion

The present results clearly show that the oculomotor system is especially tuned for detecting displacements of objects across saccades. The results from Experiment 1 demonstrate earlier emergence of saccade curvature in the condition in which the distractor was displaced during a saccade relative to the condition in which the distractor remained stationary. Experiment 2, which was specifically designed to examine this conjecture, replicated this result and demonstrated that saccade curvature away from the presaccadic location of the distractor emerged as early as 80 ms when the distractor was displaced during a saccade: about 50 ms earlier than when it remained stationary. Interestingly, saccade curvature emerged around 130 ms independently of whether the distractor was only present before a saccade, only present after a saccade, or remained stationary across a saccade. 

The results challenge previous views on how oculomotor competition is updated during a saccade. Previous studies have suggested that updating of the oculomotor competition was based solely on the information available prior to a saccade (Arkesteijn et al., 2018; Boon et al., 2018; De Vries, Hooge, & Verstraten, 2014; Jonikaitis & Belopolsky, 2014; Silvis, Solis, & Donk, 2015; van Leeuwen & Belopolsky, 2018). For instance, by showing that saccades curved away from the original presaccadic location of the distractor (Jonikaitis & Belopolsky, 2014; van Leeuwen & Belopolsky, 2018). Contrary to that interpretation, here we show that the information available rapidly after saccade landing has a clear impact on the time course of updating the oculomotor competition. The oculomotor system seems to be sensitive to small discrepancies between the anticipated distractor location and the actual location available upon saccade landing. Remarkably, detection of location discrepancies across a saccade, as detected with saccade curvature, is possible already at 80 ms. It appears that later emergence of saccade curvature (around 130 ms after the saccade) in the other conditions reflects the slower influence of new postsaccadic visual information. 

Experiments using single saccades have revealed that saccades curve toward distractors if they are initiated within approximately 200 ms of target onset and away from distractors if they are initiated more than 200 ms after target onset (Hickey & Van Zoest, 2012; McSorley et al., 2006; Mulckhuyse, Van der Stigchel, & Theeuwes, 2009). Although the initial saccade curvature in the displacement condition is interpreted as curvature away from the presaccadic distractor location, an alternative interpretation could be that it reflects initial curvature toward the displaced distractor, which then shifts to curvature away from the displaced distractor. However, although curvature toward is typically found in the single saccade paradigms, it has not been found in the double-step paradigms (Boon et al., 2018; Jonikaitis & Belopolsky, 2014; van Leeuwen & Belopolsky, 2018). This might be due to the increased time and predictability that the oculomotor system has to determine the location of the distractor when making a sequence of saccades compared to a single saccade (R. Walker, McSorley, & Haggard, 2006). Additionally, if the initial effect in the displacement condition is indeed curvature toward the distractor, the same initial curvature should be found in the postsaccadic condition, which was not the case. Therefore, we interpret the pattern of saccade curvature in the displacement condition as curvature away from the initial distractor location that shifts to curvature away from the new distractor location. 

It is difficult to directly compare the current estimates of the onset of the oculomotor competition to the previous estimates obtained using single saccades simply because the typical latency of a single saccade lies around 200 ms. Most studies that examined the time course of oculomotor competition have estimated the onset of the oculomotor competition to occur around 150 ms (Born & Kerzel, 2008; Godijn & Theeuwes, 2002; Mulckhuyse et al., 2009; R. Walker & McSorley, 2006). For instance, Godijn and Theeuwes (2002) show that a remote distractor affects saccade direction as early as 155 ms after onset. Similarly, experiments examining the time course of saccade curvature for single saccades find saccade curvature at the fastest saccade latencies, approximately 150 ms (McSorley et al., 2006; R. Walker et al., 2006). All these estimates (around 150 ms after saccade) are more in line with the late emergence of saccade curvature observed in our study and may be attributed to the time that new visual information takes to influence oculomotor competition through V1. 

Neurophysiological studies have demonstrated that the effects of visual salience are apparent already in the feed-forward sweep of information processing (Lamme & Roelfsema, 2000; White, Kan, Levy, Itti, & Munoz, 2017). Interestingly, although the earliest visual information reaches V1 after only 40 ms, it takes approximately 140 ms before salience has apparent effects in V1 (White et al., 2017). In contrast to V1, the superficial layers in the superior colliculus show slightly later initial visual responses at approximately 49 ms, but significantly earlier effects of salience at approximately 60 ms (Marino, Levy, & Munoz, 2015; McPeek & Keller, 2002; White et al., 2017). This is important because saccade curvature can be artificially induced by electrically stimulating the superior colliculus prior to a saccade onsets, showing that the superior colliculus is intimately connected to oculomotor competition (McPeek, 2003). Neuropsychological evidence shows that stimulus onsets can cause salience effects in the superior colliculus either directly and very rapidly, around 60 ms, or indirectly, through V1, and slower, more than 120 ms (White et al., 2017). 

Based on the studies discussed above it seems plausible that oculomotor competition can emerge as early as 60 ms after the initial saccade. Interestingly, in our study, the oculomotor competition emerged as early as 80 ms after the initial saccade, but only when the distractor was displaced during a saccade. Oculomotor competition emerged at approximately 130 ms when the distractor was available only before saccade (presaccadic condition), only after the saccade (postsaccadic condition), or remained stationary (no-displacement). Although previous studies suggest that oculomotor competition is updated to a spatiotopic reference frame before or during the first saccade (Arkesteijn et al., 2018; Boon et al., 2018; Jonikaitis & Belopolsky, 2014; van Leeuwen & Belopolsky, 2018), the current results paint a slightly different picture. Although the competition might be updated, it also seems to be resolved before the second saccade as indicated by the lack of saccade curvature at the shortest intersaccadic intervals. The subsequent reemerging oculomotor competition (+130 ms) might be caused by the salience signals arriving from V1 based on postsaccadic visual information (Lamme, 1995; Self, Van Kerkoerle, Supe, & Roelfsema, 2013; White et al., 2017). For the postsaccadic condition, there was no initial competition and no distractor was expected, but the appearance of a distractor after the first saccade evoked target–distractor competition. For the no-displacement condition, the competition was initially resolved and then built up again as the distractor remained present and was processed anew after the first saccade. The competition in the presaccadic condition was also initially resolved, but as the visual system expected a distractor after the first saccade, the absence of the distractor (distractor offset) might have evoked of new wave of competition. Because in the presaccadic condition, the competition was likely caused by the discrepancy between the expected distractor and the absence of the distractor, this condition showed overall smaller oculomotor competition compared to the other conditions (postsaccadic and stationary distractor) during which a distractor was present after the first saccade. 

One possible explanation for the rapid buildup of oculomotor competition in the distractor displacement condition is that the superior colliculus detects distractor displacement as motion, which can skip cortical processing (Davidson & Bender, 1991; Yoshida et al., 2012). The rapid detection of object motion during saccades might be a mechanism used for detecting and correcting saccade errors. It is a well-known fact that saccades often undershoot the target, and the undershoot is subsequently corrected by a corrective saccade with a very short latency. These corrective saccades tend to occur between 0 and 150 ms after the first saccade and often go unnoticed (Biscaldi, Weber, Fischer, & Stuhr, 1995; Deubel, Wolf, & Hauske, 1982; Kowler & Blaser, 1995; Weber, Dürr, & Fischer, 1998). The current results indicate that there is a qualitative difference between early oculomotor competition when a distractor is displaced compared to distractors that stay in the same spatiotopic location. Specifically, the early competition in the displacement condition might arise from early salience and motion detection in the superior colliculus, and the later competition in the other three conditions might arise due to salience being projected from V1 to the superior colliculus. This very rapid early oculomotor competition might, therefore, reflect the oculomotor system keeping track of relative object positions across saccades. 

In conclusion, we propose a revised interpretation of how the oculomotor system updates competition across saccades. Previous research indicated that target–distractor competition is updated to a spatiotopic reference frame before or during the first saccade, thereby causing immediate competition after the first saccade (see Boon et al., 2019, for review). However, the current data showed that the updated competition was resolved at the shortest intersaccadic intervals but reemerged rapidly when new visual information became available after the saccade. Interestingly, when objects were displaced during a saccade, the oculomotor competition emerged at the shortest possible time it takes for visual information to reach the superior colliculus. The results demonstrate that the oculomotor system prioritizes detection of object displacements during saccade, which may be useful for guiding corrective saccades upon landing of the eye. 

Acknowledgments

This research was supported by an Open Area Research Grant from the Netherlands Organization for Scientific Research to Artem Belopolsky (ORA 464-15-193). The authors are not aware of any competing financial interest that might be perceived as affecting the objectivity of this work. J.v.L. and A.V.B. designed research; J.v.L. performed research; J.v.L. and A.V.B analyzed data; J.v.L, A.V.B. wrote the paper. 

Commercial relationships: none. 

Corresponding author: Jonathan van Leeuwen. 

Email: . 

Address: Department of Experimental and Applied Psychology, Vrije Universiteit, Amsterdam, The Netherlands.