Retinal error signals and fluctuations in eye velocity influence oculomotor behavior in subsequent trials

Abstract

The oculomotor system makes use of an integration of previous stimulus velocities (the prior) and current sensory inputs to adjust initial eye speeds. The present study extended this research by investigating the roles of different retinal or extra-retinal signals for this process. To test for this, participants viewed movement sequences that all ended with the same test trial. Earlier in the sequence, the prior was manipulated by presenting targets that either had different velocities, different starting positions, or target movements designed to elicit differential oculomotor behavior (tracked with or without additional corrective saccades). Additionally, these prior targets could vary in terms of contrast to manipulate reliability. When the velocity of prior trials differed from test trials, the reliability-weighted integration of prior information was replicated. When the prior trials differed in starting position, significant effects on subsequent oculomotor behavior were only observed for the reliable target. Although there were also differences in eye velocity across the different manipulations, they could not explain the observed reliability-weighted integration. When comparing the same physical prior trials but tracked with additional corrective saccades, the eye velocity in the test trial also differed systematically (slower for forward saccades, and faster for backward saccades). The direction of the observed effect contradicts the expectations based on perceived speed and eye velocity, but can be predicted by a combination of retinal velocity and position error signals. Together, these results suggest that general fluctuations in eye velocity as well as retinal error signals are related to oculomotor behavior in subsequent trials.

Introduction

Directing our gaze at a flying ball to successfully catch it imposes significant challenges on our oculomotor system. Due to the ball's movement and our inherent processing delays, the ball will already be at a different position than where it was when the planning of eye movement started. It has been shown before that saccadic (Keller & Johnson, 1990; Gellman & Carl, 1991; Engel, Kevin, Anderson & Soechting, 1999; Guan, Eggert, Bayer & Büttner, 2005; Schreiber, Missal & Lefèvre, 2006; Fleuriet, Hugues, Perrinet & Goffart, 2010; Daye, Blohm & Lefèvre, 2014; Goettker, Braun & Gegenfurtner, 2019) and smooth pursuit eye movements (Bahill & McDonald, 1983; Orban de Xivry, Bennett, Lefèvre & Barnes, 2006) can successfully account for target movements and processing delays by using the currently available sensory information to estimate correct target positions at the end of eye movements. 

However, eye movements are not only based on current sensory input, but can also be refined by previous experience (the prior). This integration can improve oculomotor control in the form of anticipatory saccades (e.g. the ability to predict where a ball will be after a bounce; Diaz, Cooper, Rothkopf & Hayhoe, 2013) or anticipatory pursuit (Hayhoe, McKinney, Chajka & Pelz, 2012; Kowler, Aitkin, Ross, Santos & Zhao, 2014; Kowler, Rubinstein, Santos & Wang, 2019), which begins even before a target starts to move. The use of the Bayesian framework, which proposes a reliability-weighted integration of two signals (e.g. prior experience of where a ball will go and the available visual signals) has had a lot of success with explaining human eye movement behavior (Yang & Lisberger, 2012; Bogadhi, Montagnini & Masson, 2013; Orban de Xivry, Coppe, Blohm & Lefèvre, 2013). Recent studies (Darlington, Tokiyama & Lisberger, 2017; Deravet, Blohm, de Xivry & Lefèvre, 2018; see Kim, Park, & Lee, 2019 for an example with target direction) demonstrated that the velocity of a target in the previous trial has a systematic effect on the oculomotor behavior in the following trial. When comparing the pursuit response to a target movement based on a faster or slower previous trial, the faster previous trial led to a faster pursuit response in the next trial, despite the same sensory input. Additionally, the magnitude of this influence was modulated by the reliability of the sensory input. If there was less reliable sensory information, the influence of the previous trial was higher, providing direct evidence for a reliability-weighted integration of the prior information with the available sensory input. Although these results are interesting, they leave two open questions. 

First, is target velocity the only parameter that can have an influence on oculomotor behavior in subsequent trials? It has been demonstrated that target position can also influence pursuit behavior (Blohm, Missal & Lefèvre, 2005; Orban de Xivry & Lefèvre, 2007). Pursuit initiation differs depending on target position (Lisberger & Westbrook, 1985) and position errors during pursuit can increase or decrease pursuit speed (Seagraves & Goldberg, 1994). Therefore, information about the target position could also serve as a potential prior signal that can affect pursuit behavior in subsequent trials. 

Second, which signal processing mediates the influence of the prior trial? When comparing the influence of a prior target that is fast or slow, the effect could be based on a difference in perceived speed between the two, differences in the eye velocity in the prior trial, which presumably is encoded in an extra-retinal eye velocity signal (Brenner & van Berg, 1994; Ono, 2015; Goettker, Braun, Schütz & Gegenfurtner, 2018), or differences in the retinal velocity, which can serve as an error signal that needs to be minimized (Lisberger, 2010, 2015). These three possibilities are typically difficult to distinguish as they are directly related to each other. However, a recently demonstrated effect of corrective saccades on perceived speed can help to disentangle those signals (Goettker et al., 2018; Goettker, Brenner, Gegenfurtner & de la Malla, 2019b). If one tracks a moving object with the eyes, the object will be almost stationary on the retina, while the stationary background shifts in the opposite direction (Freeman, 2009). To account for this effect, retinal and extra-retinal eye velocity signals are combined to reconstruct a valid percept (Haarmeier, Thier, Repnow & Petersen, 1997; Freeman, Champion & Warren, 2010; see Spering & Montagnini, 2011, Schütz, Braun & Gegenfurtner, 2011). Goettker and colleagues (Goettker et al., 2018, 2019b) demonstrated that the occurrence of corrective saccades during the tracking of a moving target distorts this integration. The occurrence of a forward saccade (a saccade in the direction of the target movement) led to a faster percept than the tracking of the same physical stimulus with pure pursuit. Crucially, the influence on perceived speed seems to be based on an integration of retinal and extra-retinal information during the small corrective saccades. Assuming such an integration, when computing the average eye velocity for these trials, a proxy for the extra-retinal eye velocity signal, the eye velocities are affected in a similar way as the perceived speed: a forward saccade leads to a higher eye velocity than for a comparable pursuit trial due to the saccadic epoch. However, the average retinal velocity (in the following defined as the signed difference between eye velocity and target velocity) gets lower as the target on the retina is slowed down, and even moves in the opposite direction of the target movement during the forward saccade. The occurrence of a backward saccade (a saccade in the opposite direction of the target movement) leads to a slower perceived speed, and has the opposite effect on average eye and retinal velocity when compared to a pure pursuit response. Thus, the influence of corrective saccades allows for dissociation of the retinal velocity from the eye velocity and perceived speed. 

This study aims to answer these two questions to provide a more detailed description of the relevant information for oculomotor priors. In a first step, the goal will be to replicate the reliability-weighted effect of prior target velocities on a trial-by-trial basis. Then, based on similar logic, the influence of target position will be quantified. The influence of a corrective saccade in the prior trial on oculomotor behavior in the next trial will be characterized by comparing them to a situation with the same physical target movement, but a pure pursuit response in the prior trial. The observed effects should then help to disentangle the contributions of perceived speed, eye velocity, and retinal error signals on updating oculomotor behavior on subsequent trials. 

Methods

Participants

Sixteen volunteers (mean = 23.7 years old, SD = 3.0; 12 women) participated in the experiments. All participants, except the author, A.G., were naïve to the purpose of the study and received money as compensation. They all had normal or corrected-to-normal vision. Before the start of the experiments, they all gave informed consent (per the Declaration of Helsinki) and all experiments were approved by the local ethics committee (Giessen University LEK 2017-0029). 

Setup and experimental conditions

Participants sat at a table facing a gray monitor (Display ++, LDC; Cambridge Research Systems Ltd.) in an illuminated room. A chin and forehead rest was used to stabilize the head of each participant and minimize head movements. In this setup, participants’ eyes were 90 cm away from, and at approximately the height of the screen center. Eye movements were recorded from each participant's right eye with a desk-mounted eye tracker (EyeLink 1000 Plus, SR Research), with a sampling frequency of 1000 Hz. The experiments were controlled by MATLAB using the Psychtoolbox (Kleiner, Brainard, & Pelli, 2007). Each participant completed a total of eight blocks, four with a low contrast prior target and four with a high contrast disk (see Paradigm for further information). The order of the conditions was randomized. Before each experimental block, a calibration was run until a calibration value of at least 0.35 degrees of visual angle (deg) was reached. Additionally, an online drift-correction was performed at the beginning of each trial. 

Paradigm

Overall, participants completed eight experimental blocks, each containing 64 movement sequences. Four of the blocks had low contrast prior targets, the other four had high contrast prior targets. Participants randomly started with the low or high contrast condition, and completed all four blocks of the respective condition before switching to the other one. Each movement sequence participants saw consisted of three different trials (see Figure 1). Participants started each movement sequence with a button press while fixating on a central fixation dot. 

At the beginning of each sequence, participants saw the passive trial. The passive trial was used to replicate the design of Deravet and Colleagues (2018) and was thought to minimize carry over effects from the last movement sequence. Each passive trial started with the presentation of a red dot (radius of 0.2 deg). The red dot remained on the screen through the entire passive trial. After a random delay between 650 and 1000 ms, the standard target (a white Gaussian blob with SD = 0.3 and max contrast = 0.6) appeared either to the left or right of the initial fixation dot and moved across the screen at a constant velocity of 15 deg/s for a random time between 650 and 1000 ms. Targets that jumped to the left continued to move to the right, and vice versa. Half of the trials moved to the left. Note here that all three trials in a movement sequence moved in the same direction. The initial position of the target was determined by the relevant target crossing time. The target crossing time (TX) was defined as the time the target needed until it crossed the center of the screen after the initial position step. For the passive trials, it was always set to 220 ms. To ensure that participants kept fixating on the red dot, eye position was monitored online and if it deviated more than 2 deg from the red dot, participants were notified, and the passive movement was repeated (happened in 9% of trials). 

Once the passive trial was finished, the prior trial started with the presentation of a green dot (radius of 0.2 deg) at the center of the screen. A new random fixation duration between 650 and 1000 ms was selected before the target movement started. As soon as the target movement started, the green dot disappeared and participants were instructed to track the appearing target with their eyes. Across different blocks, the prior trial could be either presented with a high contrast white disk (radius of 0.33, contrast = 1) or a low contrast Gaussian distribution (SD = 0.52, max contrast = 0.2) to manipulate the reliability of the prior. Within each block there were four possible prior trials, which were grouped into two manipulations. For the first manipulation, trials differed in TX. Based on previous experience (Gelman & Carl, 1991; de Brouwer, Yuksel, Blohm, Missal & Lefèvre, 2002; Goettker et al. 2018, 2019b) and some pilot data, TX values were chosen that led to a mixture of trials tracked with either pure pursuit or pursuit and mostly additional forward saccades for the short TX and a mixture of pure pursuit and pursuit with mostly additional backward saccades for the long TX (see Figure 2) due to trial-by-trial variability in oculomotor behavior. For the high contrast targets, the short TX was 180 ms; the long TX was 240 ms. For the low contrast targets, the short TX was 200 ms, and the long TX was 300. Together, these trials allowed to compare the influence of the same physical target movement but tracked with different oculomotor behavior within the short and long TX conditions, as well as the comparison of the influence of initial starting position when comparing the short and long TX for trials with pure pursuit. The second manipulation was a variation in target speed. One prior trial moved at 10 deg/s, and one prior trial moved at 20 deg/s. Both had the same TX of 220 ms for the high contrast targets and 250 ms for the low contrast targets, leading to tracking with mainly pure pursuit. The comparison of these conditions allowed to measure the influence of prior target velocity. 

Figure 1.

 

Depiction of the paradigm. Each movement sequence consisted of three trials. All sequences started with each participant viewing passive motion indicated by a red fixation dot. The dot stayed visible throughout the trial, and participants needed maintain fixation on the dot while the target passed. After the passive trial, the prior trial started with a green fixation dot. The fixation dot disappeared as soon as the movement started. Note that the position of the initial fixation in the prior trial is still depicted with an open green circle to show where the eye was fixated at the moment of movement onset. At the moment the target appeared, the fixation dot was not visible anymore. There were four different possibilities that could be presented during the prior trial, all of which could be either presented at high or low contrast. Trials varying with short or long target crossing times (TX) were presented with the same velocity, but had different starting positions. Trials varying in target speed could either be presented at 10 deg/s or 20 deg/s. Immediately after the prior trial, the test trial was presented. It also started with a green dot that disappeared at movement onset. Then the target moved again across the screen. Note that passive and test trials were always constant throughout the experiment and only the prior movement was manipulated, allowing assessment of the influence of varying prior information.

Figure 2.

 

Depiction of different oculomotor behaviors. (A) Depiction of horizontal target position, black dotted line, and eye position for a movement with short target crossing time (TX). The gray dotted line shows eye position during fixation, and the start of the black line shows the start of the pursuit response. The blue segment indicates the forward saccade. Note that the same physical movement is sometimes tracked with pure pursuit and in some cases with an additional corrective saccade. (B) Depiction of two movements in response to targets with long target crossing time. Eye position is shown in the same way as in panel A, but for long TX the corrective saccade was typically a backward saccade, as shown in green.

The distribution of the different types of prior trials was as follows: In each block of 64 movement sequences, there were 24 trials with short TX, 24 with long TX, and eight trials each of 10 or 20 deg/s, which were presented randomly. This unequal representation of prior trials was chosen, because for the short and long TX the comparison of trial with pure pursuit and trials with additional corrective saccades relied on trial-by-trial variability in oculomotor behavior and therefore needed more trials for a reliable estimate this effect. Regardless of the prior condition, the target kept moving for a random time between 650 and 1000 ms and then disappeared. 

At the end of each movement sequence, the test trial was presented. The movement again started with the presentation of a green dot at the center of the screen. After a random delay between 650 and 1000 ms the standard target appeared with a TX of 220 ms and moved at 15 deg/s across the screen. The test movement was always the same and resembled the passive movement, except that participants were asked to follow the movement with their eyes. In this way, it was possible to directly compare the effect of all prior conditions when analyzing the oculomotor responses in the test movement. 

Data analysis

Eye movement data were digitized on-line and analyzed off-line using Matlab software. As target movements always moved horizontally, only the horizontal-position was used for the analyses. First, eye position was filtered with a second-order Butterworth filter, with a cutoff frequency of 30 Hz, and eye velocity was calculated as the first derivative of the filtered position traces. Saccades were identified based on the EyeLink criteria with a speed and acceleration threshold of 30 deg/s and 4000 deg/s², respectively. After the detection of saccades, a linear interpolation of the eye movement velocity around the time of the saccade (from 25 ms before saccade onset to 25 ms after saccade offset) was performed for some parts of the analysis, and the eye movement velocity was filtered with an additional low-pass Butterworth filter with a cutoff frequency of 20 Hz. Pursuit onset was calculated by looking for the first time eye velocity was higher than two times the standard deviation of the eye velocity during fixation (−50:50 ms after target movement onset) and stayed above 30% of the target velocity for 10 consecutive frames (10 ms). To analyze all data independent of their movement directions, all eye movements were projected in one direction. 

The first step in the analysis was to classify the prior trials for short and long TX into different groups depending on their oculomotor behavior. Trials were considered as pure pursuit trials when no saccade occurred until 400 ms after motion onset (de Brouwer et al., 2002). If a saccade was detected in that interval, the direction of the saccade was used to classify the trial as a trial, including a forward or backward saccade. If saccades went in the direction of the target movement, they were considered a forward saccade. Additionally, we computed position error, velocity error, and eye velocity, including saccadic epochs for each sample between motion onset and 400 ms after motion onset, and then computed the average value in our analysis window of 100 ms to 400 ms after motion onset. Typically, during the first 100 ms, the subject's eyes are still in fixation and no difference between the conditions was expected. 

For the test trial, saccade-interpolated eye velocity traces were aligned on the target motion onset for each subject for the different conditions. Then, for each subject and target contrast, the velocity profile was computed by taking the median of all movement sequences with relevant prior trials. Note here that aligning the velocity profile on motion onset and not on pursuit onset allows the capture of potential differences in latency, as well as acceleration and peak speed into one measure when comparing different velocity profiles. The difference between two velocity profiles was quantified as the summed difference for each relevant frame within in the analysis window of 100 to 400 ms after motion onset normalized by the number of frames. To assess the influence of prior target velocities and target positions, the difference between velocity profiles was computed for each subject for both target contrast either between prior trials moving at 10 deg/s or 20 deg/s or between trials with short and long TX tracked with pure pursuit. To quantify the influence of corrective saccades, we compared the velocity profile in the test trials, depending on whether the prior trial was tracked with pure pursuit or an additional corrective saccade. This comparison was done for trials with short and long TX separately. 

Exclusion criteria and statistical analysis

Trials were excluded based on three criteria: (1) if a subject blinked in any of the trials in the time window of 150 ms before motion onset and 400 ms after motion onset, (2) if it was not possible to determine a pursuit onset within 300 ms, and (3) if a saccade was detected with a saccadic latency smaller than 50 ms, because these saccades were probably planned based on the information present before motion onset. Based on these criteria 7035 of 8192 (approximately 86%) movement sequences were included in the further analysis. 

To quantify the influence of target velocity, paired t-tests against zero were used to quantify the difference in test trials depending on prior velocities and Bonferroni corrections for multiple testing was used. To test for reliability-weighted integration a paired t-tests was used to compare the magnitude of the difference between the velocity profiles. Because there is a clear hypothesis for a reliability-weighted integration, namely a stronger influence with a more reliable prior in the high contrast condition, a one-sided t-test was used. The comparison based on reliability-weighted integration was performed with a significance level of p < 0.05. The analysis of the influence of target position relied on trial-by-trial variability in oculomotor behavior, because only trials with pure pursuit were compared to rule out additional influences by corrective saccades. Only data points were taken into account where at least eight valid trials for both conditions (short and long TX) were available. Based on this restriction, we could use 25 out of 32 (16 participants * 2 contrast conditions) data points and again first used paired t-tests again against zero (again corrected for multiple comparisons), and then a one-sided two sample t-test to compare the effects between the different contrast conditions to test for a reliability-weighted effect. 

To test for the influence of eye velocity, movement sequences were randomly selected for each participant, so that the difference in pursuit eye velocity between prior trials moving at 10 and 20 deg/s was between 2 and 3 deg/s. The random pairing of movement sequences ended when no other pair of trials was available that fulfilled this criterion. Because the selection of trials is dependent on chance, we repeated this procedure 50 times for each participants and then averaged across the repetitions to obtain one measurement per subject and target contrast for a difference in prior eye velocity as well as a difference in eye velocity for test trials. Then, the influence of prior target contrast on these two measurements was tested with paired t-tests. 

To quantify the influence of corrective saccades, differences between the velocity profiles in the test trial were computed, and trials were grouped based on the oculomotor response in the prior trial: pure pursuit or pursuit with an additional corrective saccade. This was done separately for short and long TX. To quantify the influence of forward saccades, only participants with at least eight valid trials for both velocity profiles (short TX pure pursuit, short TX pursuit + corrective saccade) were chosen. Additionally, more than 50% of the saccades needed to be identified as forward saccades. To quantify the effect of backward saccades, only participants with at least eight valid trials for both velocity profiles (long TX pure pursuit, long TX pursuit + corrective saccade) were chosen. Additionally, more than 50% of the saccades needed to be identified as backward saccades. Based on these criteria, 23 comparisons for the low contrast condition (16 for forward saccades and 7 for backward saccades) and 25 comparisons for the high contrast condition (10 for forward saccades and 15 for backward saccades) were included. To quantify the effect of corrective saccades, two-sample t-tests were used to compare the influence of forward and backward saccades. To test for an influence of target contrast, the magnitude of the differences between forward and backward saccades for the low and high contrast target was compared. 

Modeling

To evaluate the influence of retinal error signals, extra-retinal eye velocity signals or differences in perceived speed, a simple model was built to predict the influence of additional corrective saccades. This effect was especially interesting, because here the same physical target movement was presented and so any differences can only be related to variations in the relevant variables. To make predictions based on retinal errors, we computed estimates of the retinal velocity during prior trials in the analysis time window of 100 to 400 ms after motion onset. Retinal velocity was defined as the signed difference between eye and target velocity. Negative values indicate a movement in the target direction and positive values indicate a movement in the opposite direction of the target direction. Please note here that target directions were projected in one direction at the beginning of the analysis, so that this calculation works for targets moving to the left and right. To predict the influence of additional corrective saccades during the prior trials, the average retinal velocity was computed for prior trials moving at 10 or 20 deg/s. The difference in retinal velocity between these prior trials was then related to the observed influence of the different prior target speeds on test trials. For example, for the high contrast prior trials, the retinal velocities (including saccadic epochs) were on average across subjects −3.57 deg/s for a prior target moving at 10 deg/s, and −10.72 deg/s for a prior target moving at 20 deg/s. This average difference of −6.42 deg/s in retinal velocities in the prior trials was related to an average difference of the velocity profiles during the relevant test trials of 1.13 deg/s. The ratio between these two values led to a transfer weight (which was computed for each subject individually) that then was used to predict the influence on the test trial. To estimate the effect the transfer weight was multiplied with the difference in retinal error signals for prior trials only differing in the execution of a corrective saccade. With the same method we computed the influence of retinal position errors, for the comparison of prior trials with different TX and their respective influence on the test trial. Then retinal errors for the prior trials with or without additional corrective saccades were computed and the transfer weights estimated for each subject was used to predict the expected effect. For the retinal prediction the effect of retinal velocity and retinal position error was summed. 

The prediction based on eye velocity was calculated in a similar fashion. Here, the difference in pursuit eye velocity was computed between prior trials moving at 10 or 20 deg/s and related to the observed effect on test trials via a transfer weight for each observer. Then differences in pursuit velocity between prior trials with or without additional corrective saccades were used to predict the effect on test trials. Because perceived speed was not assessed in the present study, the differences in perceived speed estimated in Goettker et al. (2018) were used. A forward corrective saccade led to an increase in perceived speed by roughly 7%, whereas a backward saccade led to a reduction of perceived speed by around 4%. To estimate the influence of such a change in perceived speed, the observed effect of prior targets moving at 10 or 20 deg/s was multiplied by this percentage. Because there were no individual estimates for each observer or for the different target contrasts, this was only done once to obtain comparison values. 

Data availability

All data are available under https://osf.io/j9x6y/. 

Results

The goal of this study is to describe relevant information that mediates the influence of priors on oculomotor behavior in subsequent trials. Based on previous results, it has been established that prior target velocity is integrated for oculomotor control in a reliability-weighted manner, but whether additional signals, like position errors or additional corrective saccades in the previous trial, affect the influence of the prior signal on oculomotor behavior in the next trial is unknown. By varying the available prior information across these dimensions, this study sought to quantify these effects. First, the results of the different comparisons will be presented starting with the differences caused by different prior target velocities, followed by the effect of different target position. Then the role of variations of prior eye velocities and the influence of additional corrective saccades will be investigated. After presenting the individual effects, the different possible mediators, like perceived speed, eye velocity signals, or retinal error signals, will be tested. 

Effect of prior target velocities on the initial pursuit response in the next trial

To establish the influence of prior target velocity on the oculomotor behavior in the next trial, movement sequences were sorted based on target velocity as well as target contrast in the prior trial. This allowed the comparison of pursuit responses in the test trial depending on the prior information (see Figure 3). For each of the groups, oculomotor responses were aligned at motion onset and the median eye velocity was computed for each participant for the different conditions. The influence of the prior target velocity was quantified by computing the average difference between velocity profiles in the test trial for a prior trial of 20 deg/s and 10 deg/s in the interval 100 to 400 ms after target movement onset (see Methods for more details). For both target contrasts in the prior trial, values obtained were positive and significantly different from zero (t(15) = 6.04, p < 0.001 for low contrast; and t(15) = 7.85, p < 0.001 for high contrast, see Figure 3C), showing a strong influence of prior target velocities on initial pursuit response in test trials. Despite the same sensory information being available in test trials, if a prior trial moved at 20 deg/s the oculomotor response was faster than if the prior trial moved at 10 deg/s. By comparing the magnitude of the difference based on the prior target contrast, it was possible to check for reliability-weighted integration. There was a significantly larger influence of the prior target velocity for the high contrast target (t(15) = 2.84, p = 0.006), which demonstrates a stronger influence of the more reliable high contrast prior trial. Thus, replicating the reliability-weighted integration of prior information reported in earlier studies. 

Figure 3.

 

Effect of prior target velocity and reliability. (A, B) The average eye velocity across subjects in the test trial aligned on target motion onset. The lighter red line shows the oculomotor response in the test trial for a prior trial moving at 10 deg/s, the darker red velocity profile shows the eye velocity for a prior trial moving at 20 deg/s. The black dashed line indicates the actual target velocity of 15 deg/s in the test trial. A shows the results when the prior targets were at low contrast, B shows the results when the prior target was at high contrast. (C) The effect of the prior velocity based on the average difference between the two velocity profiles. The open square shows the result from the low contrast condition, and the filled circle the result from the high contrast condition. Positive values indicate a higher velocity in test trials when the prior trials moved at 20 deg/s. Grey lines depict individual results per subject. Error bars and shaded areas around the velocity profiles indicate the standard error of the mean.

Effect of prior target positions on the initial pursuit responses in the next trial

Based on a similar logic as for the influence of target velocity, it was possible to test for potential influences on oculomotor behavior in the test trial based on target position in prior trials. To do this, the comparison was made between prior trials with short and long TX, which had the same target velocities, but started at different positions. Short TX had a smaller initial position error than long TX (2.7 deg vs. 3.6 deg for the high contrast target and 3 deg vs. 4.5 deg for the low contrast condition). To make sure that there was no influence by additional corrective saccades in the prior trial, only trials with a pure pursuit response were compared for both cases. Because there was some intersubject variability in the number of pure pursuit trials available, participants only were included in the comparisons when at least eight valid trials were present for both TX (see Methods for more details). Based on these criteria for the prior trials, the different velocity profiles were computed for the test trials (see Figure 4). Although there was no significant influence of the prior target position for the low contrast prior (t(13) = 0.05, p = 0.96), there was a significant effect for the high contrast prior (t(10) = 2.81, p = 0.02). If there was a high contrast prior, the pursuit response in the test trial was significantly faster for a prior trial with short TX. Comparing the effect for high and low contrast prior targets, again revealed a significantly stronger effect for the more reliable high contrast prior (t(23) = 2.08, p = 0.049). Thus, indicating again a reliability-weighted influence of the prior target position. 

Figure 4.

 

Effect of prior target position and reliability. (A, B) The average eye velocity across subjects in the test trial aligned on target motion onset. The lighter purple line shows the response in the test trial for prior trials with short target crossing times, the darker purple shows the velocity profile for a prior trials with long TX. The black dashed line indicates the actual target velocity of 15 deg/s in the test trials. A shows the results when the prior target was at low contrast, B shows the results when the prior target was at high contrast. (C) The effect of the prior TX based on the average difference between the two velocity profiles. The open square shows the result from the low contrast condition, the filled circle the result from the high contrast condition. Positive values indicate a higher velocity for trials where the prior had a short TX, thus the eye was more likely of lagging behind the target. Grey dots depict the individual results for each subject. Error bars and shaded areas around the velocity profiles indicate the standard error of the mean.

The influence of prior eye velocity on the initial pursuit response in the next trial

When manipulating the initial starting positions for the comparison of different TX, we did not only introduce different position errors, but also the eye velocity changed during prior trials (see Figures 5A, 5B). The effect was in the same direction as the observed influence on the subsequent test trials: the eyes moved faster for the short TX condition in comparison to the long TX condition. The difference in eye velocity during prior trials was also significantly larger in the high contrast condition (t(23) = 3.90, p < 0.001), which again resembles the effect on the subsequent trial (compare Figure 5C and Figure 4C). However, there was one interesting difference: although there was no effect on test trials for the low contrast target, there was still a significant difference in eye velocity during the prior trial when comparing short and long TX. This could suggest that the eye velocity might not be responsible for the observed differences depending on prior target contrast. 

Figure 5.

 

Variations in eye velocity. (A, B) The average eye velocity across subjects in the prior trial aligned on target motion onset. The lighter purple line shows the response in the prior trial with targets at a short target crossing times, the darker purple shows the velocity profile in the prior trials with the target at long TX. The black dashed line indicates the actual target velocity of 15 deg/s in the test trials. A shows the results when for the low contrast prior target, B shows the results for the high contrast prior target. (C) The difference in eye velocities in the prior trials depending on the target crossing time. The open square shows the result from the low contrast condition, the filled circle the result from the high contrast condition. Positive values indicate a higher velocity for trials with a short TX. Grey dots depict the individual results for each subject. (D) Difference in eye velocities for prior targets moving at 20 and 10 deg/s. The presented values show the difference between the high and low contrast condition. Here, trials were selected in order to have the same eye velocities for prior trials (left data point), but interestingly the data still showed a significant difference in the oculomotor behavior in the associated test trials (right data point). (E, F) Average eye velocities in prior trials moving at 20 deg/s E and test trials F for a high contrast target. Two groups have been built based on a median split for the prior eye velocities. The solid line shows the fast trials, the dashed line shows the slow trials. All error bars and shaded areas around the velocity profiles indicate the standard error of the mean.

To investigate this in more detail, a similar analysis was performed for the differences in eye velocity depending on prior target speed. Here, a similar pattern was present for the comparison of prior trials moving at different speeds. Here also, the difference in eye velocity between prior trials moving at 10 and 20 deg/s was larger for the high contrast in comparison to the low contrast condition (t(15) = 2.28, p = 0.040). Thus, again indicating that the difference in prior eye velocity could explain the difference between the prior target contrast. However, when matching the difference in prior eye velocities for the two target contrasts, the reliability-weighted effect remained (t(15) = 2.42, p = 0.029). The effect is visualized in Figure 5D by subtracting the difference in eye velocity from the low contrast priors from the difference in eye velocity for the high contrast prior targets. Thus, indicating that differences in prior eye velocity cannot explain the observed differences depending on target contrast. 

Therefore, does that mean that eye velocity during the prior trials has no influence? Although it cannot explain the reliability-weighted integration, natural variability in prior eye velocities was related to the eye velocity during test trials. When the same prior trials (e.g. moving at 20 deg/s) were separated based on a median split depending on the eye velocity (see Figure 5E for an example) there was a significant influence across both target contrasts and priors moving at 10 or 20 deg. Higher eye velocities during the prior trial led to higher eye velocities in the test trial (Figure 5F). Thus, indicating that general fluctuations in prior eye velocity do have an influence on oculomotor behavior in test trials, there must be an influence by other signals. 

Effect of additional corrective saccades on the initial pursuit response in the next trial

To investigate the influence of variations in oculomotor behavior in more detail, the influence of additional corrective saccades during prior trials was tested. For these prior trials with the same physical movements of the target, but one time tracked with pure pursuit response and another time with pursuit with an additional corrective saccade were compared. Additionally, the direction of these corrective saccades might play a crucial role: previous work has demonstrated that trials tracked with additional forward corrective saccades are perceived to move faster than comparable trials with pure pursuit, whereas trials with additional backward saccades are perceived to move slower (Goettker et al., 2018, 2019b). Based on the oculomotor behavior and target crossing times in the prior trials, the velocity profile in the test trials were computed and compared (see Figure 6). Interestingly, trials with prior forward saccades led to a slower pursuit response in the test trials compared to the same physical trials tracked with pure pursuit, whereas trials with prior backward saccades led to the opposite effect on the test trials (Figure 6C). The difference between the influence of the two types of saccades was significant for the low contrast (t(21) = 2.24, p = 0.04) as well as the high contrast condition (t(23) = 3.17, p = 0.004), however, the size of the effect was comparable across both prior target contrasts (0.77 deg/s for low contrast and 0.74 deg/s for high contrast). 

Figure 6.

 

Effect of prior oculomotor behavior. (A) The average eye velocity across subjects in the test trial aligned on target motion onset. The black velocity profile shows the response for trials where the prior trial was tracked with pure pursuit, the blue velocity profile shows the velocity profile tracked with an additional forward saccade. Note that the physical target movement for both of these prior trial was the same. The black dashed line indicates the actual target velocity of 15 deg/s in the test trial. (B) Same depiction as in A, however, here, the green velocity profile shows trials that were tracked with additional backward saccades. (C) The effect of the occurrence and direction of a corrective saccade based on the average difference between the two velocity profiles. The open square shows the result from the low contrast condition, the filled circle the result from the high contrast condition. Blue data points depict the influence of forward saccades, green data points the influence of backward saccades. Positive values indicate a higher velocity in the test trials with a corrective saccade during the prior trials. Grey dots depict the individual results per subject. Error bars and shaded areas around the velocity profiles indicate the standard error of the mean.

A simple model to predict the influence of differences in oculomotor behavior

After establishing all of the different influences, a simple model was built to directly compare the influence of retinal error signals, prior eye velocity, and differences in perceived speed (values based on Goettker et al., 2018). The model aimed at predicting the effect of prior trials with or without additional corrective saccades on the test trial, because this comparison was especially interesting, as here, always the same physical target movement was presented, so no additional factor could influence the estimated effect. The model was built based on transfer weights that turned estimates of different variables into an estimate of an expected influence on the test trial. For example, to estimate the influence of retinal velocity errors, the average retinal velocity error for prior trials moving at 10 or 20 deg/s was computed and this difference was related to the observed effect on the test trial for such a comparison (see Methods for more details). Overall, predictions based on retinal errors (the sum of retinal velocity and position errors) were the closest to the observed effect (average error between means across conditions: 0.18 deg/s), followed by the predictions based on eye velocity (average error: 0.28 deg/s) and the predictions based on perceived speed (average error: 0.41 deg/s). In addition, on the individual level, the correlation between the observed data and the retinal prediction (Figure 7B) was the highest with r(23) = 0.62, p < 0.001 for the high contrast condition and r(21) = 0.41, p = 0.051 for the low contrast condition. In addition, the predictions based on eye velocity correlated significantly with the observed effect in the high contrast condition: r(23) = 0.44, p = 0.026 (low contrast: r(21) = 0.38, p = 0.072). 

Figure 7.

 

Retinal signals predict the observed effects of oculomotor behavior. (A) Model predictions plotted against the average of the observed effects of prior oculomotor behavior. Black lines depict the model predictions based on retinal errors with shaded areas showing the 95% CI based on variations across subjects. The pink lines depict the mean predictions of the model based on eye velocity, and yellowish lines show the predictions based on differences in perceived speed, which could be expected given the data from Goettker et al. (2018). (B) Predicted values plotted against observed values for all subjects. Filled circles depict high contrast prior targets, open squares show data with low contrast prior targets. The black line depicts a regression fitted to the high contrast condition. CI, confidence interval.

Because differences in prior eye velocities and retinal errors are directly related, a partial correlation was performed to test whether the influence of retinal errors can be explained by variations in eye velocity. However, also the partial correlation between the retinal predictions and the observed effect remained significant when controlled for predictions based on eye velocity (r(23) = 0.51, p = 0.011 for high contrast), indicating a systematic influence of retinal error signals. Interestingly, the correlation between the retinal predictions and the observed effect was substantially reduced when the saccadic epochs are interpolated for the retinal velocity error (r(23) = 0.16, p = 0.457 for high contrast and r(21) =.030, p = 0.172 for low contrast) and also the average error of the estimated effect is increased to 0.42 deg/s (0.18 deg/s before). These results indicate that retinal error signals, even those during small corrective saccades, play an important role in updating subsequent oculomotor behavior. 

Interactions of saccadic and pursuit eye movements

Although, so far, the focus was on the saccades in the prior trials and average velocity profiles in the test trials, it is also interesting to look at saccades that occurred during test trials. Therefore, the average amplitudes of the first saccade in each test trial was computed and averaged based on the prior trials. For the comparison of saccade amplitude in the test trials based on variations of oculomotor behavior in the prior trials, the average saccade amplitude in the test trials was larger for prior trials with forward saccades than for the comparable prior pursuit trials (Figure 8A). In contrast, when the prior trial contained a backward saccade, the saccade amplitude in the test trial was smaller compared to the respective prior pure pursuit trials. The change in saccade amplitudes in the test trials is in contrast to the observed effects on the initial pursuit responses, suggesting an interaction between pursuit and saccadic eye movements. To quantify this relationship, the change in saccade amplitude was computed similarly to the computation carried out on the influences on pursuit behavior, that is, as the difference between the respective prior trials. This was also possible for the comparison of different velocities and different positions in the prior trials. When plotting the observed effect on pursuit behavior against the change in saccade amplitude, there was a significant relationship across all prior comparisons of r(51) = −0.40, p = 0.003 for the low contrast condition and r(50) = −0.44, p = 0.001 for the high contrast condition (see Figures 8B, 8C). This relationship indicates two things: first, it validates our measurement of pursuit behavior by demonstrating that changes in pursuit behavior are compensated for by scaled saccadic eye movements. Second, this relationship also indicates a tight interaction of saccadic and pursuit eye movements when used to track a moving target, where saccadic eye movements seem to correct for the influence of the prior, if the prior leads to additional error signals. 

Figure 8.

 

Interaction of saccadic and pursuit eye movements. (A) The average amplitude of the first saccade in the test trials based on the oculomotor behavior in the prior trials. Open squares show the data for the low contrast prior target, filled circles the data for the high contrast prior target. (B, C) The comparison of the influence of different prior comparisons on the differences between velocity profiles in the test trials against differences in the amplitudes of the first saccade in the test trial. Red data points depict comparisons of the prior trial moving at 10 or 20 deg/s, purple data points depict the comparison of pure pursuit trials for short and long TX, blue and green data points depict the effect of forward or backward saccades with respect to the comparable pursuit trials. The black line depicts a regression fitted to the data. B shows the data for the low contrast prior, and C shows the data for the high contrast prior.

Discussion

The goal of the present study was to investigate which signals are used as prior information that influence oculomotor control in subsequent trials. The influence of the prior was established with different target velocities in the prior trial affecting the oculomotor behavior in the test trial, which replicated the reliability-weighted integration of prior and current sensory inputs (see Figure 3). However, not only target velocities but also initial starting positions of prior pursuit targets had a significant influence on the test trials, but only when prior targets were more reliable (see Figure 4). Although the different prior movements also led to differences in eye velocity, this difference was not able to explain the differential effect of the target contrast in the test trials (see Figure 5). Interestingly, when investigating the same physical prior movements, but with trial-by-trial variability in prior oculomotor behavior, there were also systematic changes in the oculomotor behavior in the test trials: prior trials tracked with additional forward saccades led to slower pursuit responses compared to pure pursuit trials. In contrast, additional backward saccades led to the opposite pattern (see Figure 6). Because this influence of corrective saccade is in the opposite direction of previously reported results on perceived speed (Goettker et al., 2018, 2019b) or the use of a potential extra-retinal eye velocity signal, this suggests that the oculomotor prior is based on retinal error signals (see Figure 7). The changes in initial pursuit responses in test trials were directly related to accompanying changes in saccade amplitudes. The change in saccade sizes scaled with the influences on pursuit initiations, to adjust for the influence of the prior on pursuit to allow for optimal tracking (see Figure 8). 

Reliability-weighted integration

As the test trials were always the same throughout the whole experiments, in theory, participants could just ignore the prior trials and just use the sensory information during the test trials. However, there were clear systematic differences in the test trials depending on the type and contrast of prior trials, which adds to the view of a reliability-weighted integration of prior information with the current sensory input in the oculomotor system. Such an integration of prior information cannot only reduce the noise and lead to more accurate pursuit responses (Kim et al., 2019), but over multiple trials can also produce anticipatory behavior to minimize delays and error signals within the oculomotor system (Kowler, Aitkin, Ross, Santos & Zhao, 2014; Kowler, Rubinstein, Santos & Wang, 2019). Previous related studies investigated this reliability-weighted integration by varying the reliability of the current sensory input and investigating the influence of comparable priors (Darlington et al., 2017; Deravet et al., 2018). The results from the current study demonstrate that the same logic also holds when keeping the relevant sensory information constant, but modulating the reliability of the prior information (see Figure 3). Evidence for stronger influence of a more reliable prior was also found in a control experiment of Darlington et al. (2017) and for multiple repetitions of the same prior movement, which presumably also leads to a more reliable prior (Deravet et al., 2018). Together, this demonstrates that there is a reliability-weighted integration of prior target velocity and current sensory input. 

Interestingly, not only prior target velocities, but also prior target positions influenced oculomotor behaviors in subsequent trials. It is known that whereas motion signals are the dominant input for pursuit and pursuit is not even possible without the percept of a moving target (Steinbach, 1976), there is also evidence that position-related signals influence pursuit eye movements (Orban de Xivry & Lefèvre, 2007): pursuit initiation is affected by relative target positions (Lisberger & Westbrook, 1985) and pursuit velocities during tracking change depending on the relative position of the eye to the target (Seagraves & Goldberg, 1994). When comparing short and long TX prior trials, eye velocities in the test trials were faster for the short TX condition for the high contrast prior (see Figure 4). Short TX prior trials led to an average position error which was slightly negative and it is known that when the eyes lag behind the target, pursuit accelerates (Seagraves & Goldberg, 1994; Goettker et al., 2018; Goettker, Braun & Gegenfurtner, 2019a). In contrast, long TX prior trials led to positive position error, and thus a lead of the eye in front of the target, which is known to be related to a deceleration of the pursuit response (Seagraves & Goldberg, 1994). Therefore, the direction of this effect of the prior target position matches the influence of target positions from previous studies during pursuit. The current results demonstrate that not only target velocities but also target positions are used as prior signals, which are used to update the oculomotor behavior in subsequent trials. The prior information for target position is also integrated in a reliability-weighted manner, as the effect was only present for the more reliable high contrast prior trials. 

Although there was reliability-weighted integration of signals related to target velocity and position, the influence of prior trials with forward and backward saccades was comparable across both contrast conditions. This lack of a reliability-weighted effect might be explained by the way the influence of the corrective saccades was computed. The impact on the test trial of corrective saccades was always measured with respect to pure pursuit movements with the same physical prior target movement and contrast. Such a relative effect attenuates the differences between the high and low contrast prior trials, and, therefore, makes it difficult to observe a reliability-weighted integration of the influence of corrective saccades. In addition, although the difference in reliability of retinal error signals is relatively small during small corrective saccades (see the next section for more detailed discussion), if it is similar for the high and low contrast condition, it could also explain the comparable effect. 

Two factors influencing oculomotor behavior in the next trial

With the present study there is evidence that prior target velocities, prior target positions, as well as the occurrence of additional corrective saccades can influence oculomotor behavior in subsequent trials. However, the question that remains is which kind of signals mediate this influence on subsequent trials. Remember that, for example, the difference in target velocity can be explained by multiple factors (perceived speed, differences in eye velocities, or higher retinal velocity errors). However, the combination of the present results can shed some light on the contributions of these factors. 

Differences in perceived speed can be ruled out as a factor influencing subsequent oculomotor behavior. First, when comparing the effect of prior trials with different initial starting positions, this position change should not lead to a difference in perceived speed. However, there still was a difference in test trials for the high contrast target. Second, the influence of additional corrective saccades was in the opposite direction of the previously measured effect on perception: whereas forward saccades led to the percept of a faster movement (Goettker et al., 2018; Goettker et al., 2019b), the oculomotor response in test trials was slower after prior trials with forward saccades when compared to prior trials tracked with pursuit. The opposite pattern was true for trials with backward saccades. Thus, indicating that perceived speed is unlikely to mediate the influence on subsequent oculomotor behavior. 

As the results presented in Figures 5E and 5F indicate, fluctuations in prior eye velocities have a significant influence on oculomotor behavior in the test trials. However, when matching the difference in eye velocity during the prior trials for the high and low contrast condition, the influence on test trials was still significantly different (see Figure 5D). This suggests that variations in prior eye velocity play a role, but cannot explain the reliability-weighted integration. However, this logic is based on an assumption: although it is accepted that the reliability of a visual signal can vary depending on the target contrast, in theory, it would be possible that also the reliability of a representation of prior eye velocities (e.g. in the form of efference copies), could vary in reliability. However, to the best of my knowledge, so far, no study has demonstrated that the reliability of a visual signal is reflected in an efference copy signal about the executed eye movement. Therefore, the relation between prior eye velocity and test eye velocity could be related to other more general factors, for example, concentration or attention to the task (e.g. Hutton & Tegally, 2005). Throughout the block, the level of attention participants paid to the task presumably varied and this could explain why variations in pursuit velocity are correlated between prior and test trials. 

This leaves retinal error signals as a candidate for mediating the influence on subsequent oculomotor behavior. The reliability of retinal error signals can be directly related to target contrast, and are therefore the easiest explanations for the influence of prior target position and velocity. In addition, retinal error signals were also the best predictor of the influence of additional corrective saccades on test trials (see Figure 7). Furthermore, a theoretical explanation based on previous work also points in that direction: variations in eye velocity can be compensated when judging perceived speed, but this mechanism is distorted by the integration of velocity signals of small corrective saccades (Goettker et al., 2018). Assuming the retinal velocity error and the eye velocity during these small corrective saccades is indeed integrated, the direction of the observed influence on test trials can even only be explained by retinal error signals. For example, prior forward saccades led to a slower oculomotor response during test trials. However, the prior eye velocity (with the saccade integrated) for trials with forward saccades was on average higher than for the comparable pursuit trial and would therefore predict the opposite of the observed effect on the test trials. Only the retinal velocity, which shows the reverse pattern to the eye velocity, predicts the direction of the influence on test trials correctly. In line with this hypothesis, when interpolating the saccadic epochs in the velocity traces, the predictions based on retinal error signals got significantly worse, suggesting that retinal errors during the saccade are indeed affecting the oculomotor behavior in the test trials. 

Such an integration of retinal signals during saccadic eye movements seems to be at odds with the view that visual sensitivity is reduced around the times of saccadic eye movements to maintain perceptual stability (see Binda & Morrone, 2018, Wurtz, 2018). Recent work has even demonstrated that this reduction in sensitivity might already start at the level of the retina (Idrees, Baumann, Franke, Münch, & Hafed, 2020). So why should the brain rely on such unreliable retinal information? There are two main arguments that this is probably still the case. First, following the classical studies by Castet and Masson (2000) who demonstrated that we are still able to perceive motion during saccadic eye movements, recent work by Schweitzer and Rolfs demonstrated convincingly that information during saccadic eye movements is still processed and used to link object locations and prepare follow-up movements (Schweitzer & Rolfs, 2020a, 2020b). Therefore, when intra-saccadic retinal information can be used within one trial, it should also be able to affect subsequent oculomotor behavior. Second, and more importantly, the reduction in visual sensitivity scales with saccade velocity (Mitrani & Yakimoff, 1970; Stevenson, Volkmann, Kelly & Riggs, 1986; Schütz, Braun & Gegenfurtner, 2007). Remember here, that the saccades in the present study were all small corrective saccades with amplitudes of around 1 degree (mean peak speed for forward saccades: 81.09 deg/s; mean peak speed for backward saccades −54.02 deg/s). Therefore, also the produced retinal motion stayed roughly within the range that can in theory be covered by pursuit (up to 100 deg/s, see Meyer, Lasker & Robinson, 1985). Therefore, the presumably relatively small reduction in sensitivity during corrective saccades is probably negligible for influence on oculomotor behavior in subsequent trials. 

Together, this suggests that there are two factors that influence oculomotor behavior on subsequent trials: general fluctuations in eye velocity as well as retinal error signals. The additional role of retinal error signals is also supported by previous results on anticipatory pursuit, which also seems to be based on a retinal velocity memory (Murdison, Pare-Bingley & Blohm, 2013). How these two factors interact is an interesting question for future experiments. For example Watamaniuk, Bal, and Heinen (2017) demonstrated that a fixation trial with no target movement minimizes anticipatory pursuit in the following trial. This could suggest that active pursuit in the prior trial is needed to boost pursuit behavior in the following trial. However, if the updating based on retinal error signals is independent of the prior eye velocity, fixating and seeing trials with different target movements should also affect the following oculomotor response. Although there are fixation trials in the current paradigm (passive trials), targets always moved at the same velocity, so the investigation of this hypothesis is not possible. 

Priors on different time scales

Although using the retinal error signals to update oculomotor behavior in subsequent trials seems to be useful to minimize error signals in subsequent trials, trial-by-trial adaptation can also be maladaptive. The integration of a fast prior movement for a slow test target does not minimize, but actually produces more initial error signals. An interesting maladaptive example is also present in the current results: a forward corrective saccade during the prior trial led to a slower initial pursuit response in the test trial. The slower pursuit response in turn led to the occurrence of a larger forward corrective saccade (see Figure 7A), which then should have an even stronger influence on the following trial. Such an integration of prior information is unstable over time. However, the fast trial-by-trial adaptations we investigated here might not be the only learning mechanism. Hall and colleagues (Hall, Yang & Lisberger, 2018) demonstrated that pursuit learning happens with at least two different learning components, which differ in their time course. After a first rapid learning component that quickly adapts the pursuit response during the first few repetitions, there is a second slow learning component which gradually affects the pursuit up to 1000 trials. A similar separation of learning mechanisms also has been observed for saccade adaptation (Ethier, Zee & Shadmehr, 2008). Here, also a fast system, which is much more sensitive to error signals, leads to the initial correction, where a slower, more gradual system corrects the saccade on the longer time scale. Thus, whereas the initial, fast adaptation might be maladaptive, there is a more conservative mechanism in the long run which stabilizes the system. 

Although the different learning processes described here work on the scale up to 1000 trials, recent evidence has shown that the integration of prior information for oculomotor behavior also changes over the lifetime (Hemptinne, Deravet, de Xivry, Lefèbre & Yüksel, 2020). Hemptinne and colleagues studied the influence of prior visual experience on oculomotor behavior in subsequent trials for children and adults and observed that adults showed a larger modulation of the integration based on the quality of the visual information. This demonstrates that while both groups used an integration of prior information, adults made more use of a reliability-weighted integration depending on the quality of visual information of the current sensory stimulus than children. Thus, suggesting that these integration mechanisms are fine-tuned on different time scales ranging from the trial-level, across multiple hundreds of trials or even lifetime. 

Neural basis of reliability-weighted integration

Neural implementation of the reliability-weighted integration of prior and current sensory information was proposed by Darlington et al. (2018). They demonstrated convincingly that the smooth eye movement region of the frontal eye field (FEF_sem) had all properties necessary for reliability-weighted integration. The preparation activity that occurred before new sensory information arrived in FEF_sem differed depending on what happened in previous trials, showing a potential representation of the prior. If sensory information became available, the amount of preparatory activity influenced the activity for the incoming sensory information according to a reliability-weighted average of prior and sensory evidence. Darlington et al. (2018) proposed that the input of sensory information is based on motion signals from the middle temporal area (MT). The population of neurons in MT is thought to code velocity (e.g. Perrone & Thiele, 2001), which can be used to represent retinal velocity errors. These retinal velocity errors are then integrated in the FEF to affect the oculomotor behavior in subsequent trials. Although Darlington et al. (2018) only manipulated target velocities, one could think of similar mechanisms for retinal position errors that could be integrated into the oculomotor system as well. For example, the superior colliculus (SC) encodes retinal positions of targets relative to eye position, and this signal is used to control saccadic as well as smooth pursuit eye movements (Basso, Krauzlis & Wurtz, 2000; Krauzlis, Basso & Wurtz, 2000). Such a position-related signal could be used in a similar fashion as the velocity-related signals from MT to update oculomotor behavior in subsequent trials. 

The open question is how the above-described neural integration framework could explain the observed influence of prior corrective saccades on oculomotor behavior in subsequent trials. Due to the small size of corrective saccades, retinal velocities stay in a large proportion of the movements somewhere between −40 and 40 deg/s, a range in which MT neurons still respond (Inaba & Kawano, 2009). Neurons in MT have no information that the present velocity is caused by fast, saccadic movement, so the population of neurons probably provides a general representation of retinal velocities, irrespective of oculomotor behavior (Ono, 2015). MT signals which code, on average, slightly lower velocities for trials with forward saccades and slightly faster velocities for trials with backward saccades, could then be used as a prior to influence subsequent oculomotor behavior. Therefore, this effect might be unique to small corrective saccades, as the velocities of larger saccades quickly bring the retinal velocity outside the range of −40 to 40 deg/s where the MT response starts to attenuate (Inaba et al., 2009) and therefore probably not affect the prior representation. 

Saccade and pursuit interactions

Results in the current study are similar to the results observed by Deravet et al. (2018), in that not only initial pursuit responses, but also saccadic eye movements in the test trials were affected by prior information. However, due to the step-ramp paradigm used in the present study, the relationship between the occurrence and direction of corrective saccades to the prior behavior is not as clear as in the paradigm developed by Deravet and colleagues. Because in the paradigm by Deravet and colleagues the target started to move without the initial step, they could directly interpret the size of the initial saccade as a proxy for the expected retinal velocity in the current trial. They demonstrated that the saccadic and the pursuit system seem to use the same kind of representation of the target movement (Hainque, Apartis & Day, 2016). In the present study, the size and direction of the needed corrective saccades was already directly affected by the initial pursuit response, which happened before the corrective saccade. If the initial pursuit response was faster due to the prior information, the corrective saccade was smaller, and vice versa (see Figure 7). This negative correlation demonstrated that observed saccades directly corrected for potential influences on pursuit eye movements (Orban de Xivry et al., 2006). This relationship was present across all comparisons of different prior information, suggesting that oculomotor behavior in the test trials was mediated along continuous variations in retinal error signals. 

Differences between perceptual and oculomotor effects

Previous research has shown that not only oculomotor behavior is affected by the sensory input of previous trials, but also perceptual judgments depend on previously seen stimuli (Fischer & Whitney, 2014; Cicchini, Mikellidou & Burr, 2018; Kim et al., 2020). For example, the orientation of previously seen stimuli biases the percept of the observers in the same direction on the next trial (Fischer & Whitney, 2014), and this effect is also strongest when the current sensory input is less reliable (Cicchini et al., 2018). However, whereas the serial dependence shown for perceptual judgments looks similar to the effects shown for oculomotor behavior (a fast trial leads to a faster oculomotor response in the next trial), there might be potentially different mechanisms needed for perceptual judgments and actions. 

The present study shows a dissociation between the influences of prior oculomotor behavior on subsequent trials and the influence of corrective saccades on perceived speeds (Goettker et al., 2018; Goettker et al., 2019b). Based on the results, the major goal of using prior information to adapt oculomotor behavior in subsequent trials seems to be to reduce retinal error signals (position and velocity errors). Reducing motion blur and position errors by a more accurate pursuit response can lead to more accurate and reliable representations of relevant objects (Schütz, Braun & Gegenfurtner, 2009; Schütz et al., 2011). Thus, the relevant signals for oculomotor control seem to be retinal signals. In contrast, an accurate percept of target speeds across eye movements is not only based on retinal velocities, but based on a combination of retinal motions and extra-retinal signals about eye velocities (Freeman et al., 2010). This is necessary, as the retinal motion during eye movements is not a valid cue to physical motion, and pure extra-retinal eye velocity information is conflicted by noise (Osborne, Lisberger & Bialek, 2005; Rasche & Gegenfurtner, 2009), and can also be altered by the relative position of the target (Lisberger & Westbrook, 1985, Seagraves & Goldberg, 1994; Blohm et al., 2005). Thus, combining the two types of information increases the reliability of speed estimates. The use of primarily retinal signals for oculomotor priors and the integration of retinal and extra-retinal information for the reconstruction of perception can explain the observed dissociation between the effect of corrective saccades on perceived speed and their influence on subsequent oculomotor behavior. A similar dissociation between effects on oculomotor control and perception was observed between anticipatory pursuit and velocity judgments in a study published by Maus, Potapchuk, Watamaniuk and Heinen (2015). Seeing a previously faster movement makes the perceived velocity of the next movement slower, while at the same time increasing the velocity of anticipatory pursuit. Please note that, although such a repulsion effect could in theory explain the influence of corrective saccades on subsequent oculomotor behavior, this explanation breaks down when considering the effect of different prior target velocities. In this case, the faster prior trial leads also to a significant increase in eye velocity in subsequent trials. Similar results were recently presented by Wu, Rothwell, Spering and Montagnini (2021). In their work, they demonstrated that anticipatory pursuit follows the expected direction of a target, whereas perceptual results followed the opposite direction. Together, this set of studies suggest that oculomotor control is driven by more low-level retinal error signal, whereas perceptual reports typically are based on an integration of different signals. This differential use of signals can lead to dissociable behaviors for oculomotor behavior and perceptual reports depending on the task. 

Conclusion

The present study investigated the relevant signals for oculomotor priors, and which signals are mediating the influence on oculomotor behavior in the next trial. The results demonstrated an influence of prior target velocities as well as target positions on the oculomotor behavior in the next trial in a reliability-weighted manner. Although eye velocity during the prior trial was correlated with the eye velocity in the test trials, differences in prior eye velocities were not able to explain the reliability-weighted effect. Interestingly, also variability in the type of oculomotor behavior in the prior trial (pure pursuit vs. pursuit and corrective saccade) had a systematic effect on the oculomotor behavior in subsequent trials, and the direction of the effect suggested that the prior is mediated over retinal velocity and position error signals. This suggests that for short-term adaptation of oculomotor behavior, retinal error signals are used to minimize these error signals for subsequent movements to improve acuity and reliability of retinal information (Schütz et al., 2009, 2011). 

Acknowledgments

The author thank Annelie Göhler for help with data collection and Karl Gegenfurtner, Laurence Henry, and two anonymous reviewers for helpful comments and discussions on previous versions of the manuscript. 

Supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)—project number 222641018—SFB/TRR 135 Project A1, and by the DFG International Research Training Group 1901. 

Commercial relationships: none. 

Corresponding author: Alexander Goettker. 

Email: [e-mail]. 

Address: Justus Liebig University, Otto Behaghel Str. 10F, 35394, Giessen, Germany.