Different extrapolation of moving object locations in perception, smooth pursuit, and saccades

The ability to accurately perceive and track moving objects is crucial for many everyday activities. In this study, we use a “double-drift stimulus” to explore the processing of visual motion signals that underlie perception, pursuit, and saccade responses to a moving object. Participants were presented with peripheral moving apertures filled with noise that either drifted orthogonally to the aperture’s direction or had no net motion. Participants were asked to saccade to and track these targets with their gaze as soon as they appeared and then to report their direction. In the trials with internal motion, the target disappeared at saccade onset so that the first 100 ms of the postsaccadic pursuit response was driven uniquely by peripheral information gathered before saccade onset. This provided independent measures of perceptual, pursuit, and saccadic responses to the double-drift stimulus on a trial-by-trial basis. Our analysis revealed systematic differences between saccadic responses, on one hand, and perceptual and pursuit responses, on the other. These differences are unlikely to be caused by differences in the processing of motion signals because both saccades and pursuits seem to rely on shared target position and velocity information. We conclude that our results are instead due to a difference in how the processing mechanisms underlying perception, pursuit, and saccades combine motor signals with target position. These findings advance our understanding of the mechanisms underlying dissociation in visual processing between perception and eye movements.

(Note that the model was fit to control trials where there was no bias on saccadic targeting and therefore we can assume that the direction taken into account by the saccadic system corresponded, except for internal noise, to the physical direction of motion.)This multivariate model was fit separately for each participant and direction (inward vs. outward).The horizontal dashed line represents the average error obtained by using the raw saccadic velocity compensation metrics (SVC) to estimate the target direction.The numbers underneath the dots indicates the number of degrees of freedom for each dimension (vertical and horizontal) of the models.The model used in the analysis presented in the main text is the one that obtained the smallest error and can be formally notated as where ⃗ Vx and ⃗ Vy are the horizontal and vertical components of the target motion up to saccade landing; ⃗ Sx and ⃗ Sy are the horizontal and vertical components of the raw SVC metrics; θ the direction angle of the saccade, and ϵ1, ϵ2 normally distributed residual errors.We used this analysis to reduce the variance of the estimated trial-bytrial direction errors and increase the sensitivity of our analysis to trial-by-trial correlations between direction errors measured from saccades, perception and pursuit.We point out that analyses performed on raw SVC measures, or with different models all converged in showing the same pattern of results presented in the main text.S1 for details) influence the estimated target direction, for one example participant.The grey arrows represents (hypothetical) raw SVC measures, shown for both inward and outward direction, and equally spaced around the visual field.The black arrows represents the predictions of the model, which was cross-validated on control trials to maximize the accuracy in predicting the target velocity vector.Note that the models for inward and outward saccades were fit independently (using data from different trials), yet they are highly consistent in how the vectors are displaced -see for example how the estimated vectors (black arrows) in the top-right quadrant are both shifted toward the left relatively to the raw measures (gray arrows).This suggests that the model-based estimates are correcting a calibration inaccuracy that affected the top-right part of the display, regardless of the target direction (inward vs. outward).Each point in the plots represents the direction of the pre-saccadic gaze movements on a given trial, with respect to the physical direction of the target envelope (which correspond in the plot to 0 degrees); the histogram and the circular density represent the distribution of these points, and the black arrow, the average direction.B. Double-drift condition.Using the same conventions as in the plots in the main text, a positive angular difference indicates a shift toward the direction of the internal motion.Only trials where the speed of the gaze movement exceeded 2 standard deviations of the gaze speed in the pre-stimulus baseline are included.These distributions suggest that there was, at least on some trials, a pre-saccadic pursuit response broadly tuned to the direction of the target.However, these pre-saccadic responses in our dataset were too small and variable to discern whether the internal motion of the target in the double-drift condition biased the responses.We estimated an extrapolation interval, corresponding to how far in time the saccadic system predicts future target locations.This analysis assumes that the saccade sistem extrapolates the target position using a biased motion equivalent to that driving early pursuit, and that this extrapolation begins only after a certain time has elapsed since the target onset.For each participant we found the interval t which minimizes the squared error between the observed direction bias, estimated by our analysis of saccadic landings, and the expected direction bias that would be obtained in the same analysis if the saccade extrapolated target position along the pursuit direction for an interval t * (starting from an unbiased position).The expected saccadic direction bias Ŝ (shown as the green angle in the figure above) can be expressed as a function of the interval t and the pursuit direction bias P (shown as the blue angle in the figure above), Ŝ = arcsin sin P •(v•t * ) v•(latency+duration) , where v is the physical or external speed of the target, and latency and duration are the parameters of the saccadic eye movements.We found values of t * numerically using data of the Experiment 2, separately for each participant; the mean value of t * was 110 ms and the standard deviation across observers was 2 ms.(Heller et al, 2021).Blue lines represent model predictions, while black dots accompanied by error bars depict the observed data (averaged across observers) and their respective standard errors.The relative weights, denoting the ratio of internal to external motion weighting, were estimated to match the bias in pursuit direction (left panel).Additionally, we introduced a parameter that determines the gain of smooth pursuit.This was estimated by minimizing the mean squared difference between the predicted and observed speeds (right panel).The right panel emphasizes the mismatch between the predicted steep increase in pursuit speed with rising internal speed, predicted by the weighted average model fit on direction data, and the actual observed pursuit speed.

Figure S1 :
Figure S1: A. Each point represents a different multivariate model; the Y-axis represents the average error on the left-out trials (error bars represents the standard error across participants and conditions) measured in the crossvalidation procedure.(Notethat the model was fit to control trials where there was no bias on saccadic targeting and therefore we can assume that the direction taken into account by the saccadic system corresponded, except for internal noise, to the physical direction of motion.)This multivariate model was fit separately for each participant and direction (inward vs. outward).The horizontal dashed line represents the average error obtained by using the raw saccadic velocity compensation metrics (SVC) to estimate the target direction.The numbers underneath the dots indicates the number of degrees of freedom for each dimension (vertical and horizontal) of the models.The model used in the analysis presented in the main text is the one that obtained the smallest error and can be formally notated as B. Example of the model fit for one participant.The two panels show the residual error as a function of the predicted velocity components ⃗ V x (black dots) and ⃗ V y (gray dots); the lower panels show real and estimated values of target direction angle, calculated as arctan2 ⃗ V y , ⃗ V x (where 'arctan2' denotes the four-quadrant inverse-tangent function).deg.visual angle] vertical coordinates [deg.visual angle]

Figure S2 :
FigureS2: Example of correction of the estimated target direction from saccade landing.Raw saccade velocity compensation (SVC) measures can be affected by idiosyncratic, location-specific biases, possily due to changes in head position after the calibration or imperfect eyetracker calibration.This figure demonstrates how the model used in the analysis of saccade landings (see Main text and Fig.S1for details) influence the estimated target direction, for one example participant.The grey arrows represents (hypothetical) raw SVC measures, shown for both inward and outward direction, and equally spaced around the visual field.The black arrows represents the predictions of the model, which was cross-validated on control trials to maximize the accuracy in predicting the target velocity vector.Note that the models for inward and outward saccades were fit independently (using data from different trials), yet they are highly consistent in how the vectors are displaced -see for example how the estimated vectors (black arrows) in the top-right quadrant are both shifted toward the left relatively to the raw measures (gray arrows).This suggests that the model-based estimates are correcting a calibration inaccuracy that affected the top-right part of the display, regardless of the target direction (inward vs. outward).

Figure S3 :
FigureS3: Direction of pre-saccadic pursuit.A. Control condition.Each point in the plots represents the direction of the pre-saccadic gaze movements on a given trial, with respect to the physical direction of the target envelope (which correspond in the plot to 0 degrees); the histogram and the circular density represent the distribution of these points, and the black arrow, the average direction.B. Double-drift condition.Using the same conventions as in the plots in the main text, a positive angular difference indicates a shift toward the direction of the internal motion.Only trials where the speed of the gaze movement exceeded 2 standard deviations of the gaze speed in the pre-stimulus baseline are included.These distributions suggest that there was, at least on some trials, a pre-saccadic pursuit response broadly tuned to the direction of the target.However, these pre-saccadic responses in our dataset were too small and variable to discern whether the internal motion of the target in the double-drift condition biased the responses.

Figure S4:
Figure S4:We estimated an extrapolation interval, corresponding to how far in time the saccadic system predicts future target locations.This analysis assumes that the saccade sistem extrapolates the target position using a biased motion equivalent to that driving early pursuit, and that this extrapolation begins only after a certain time has elapsed since the target onset.For each participant we found the interval t which minimizes the squared error between the observed direction bias, estimated by our analysis of saccadic landings, and the expected direction bias that would be obtained in the same analysis if the saccade extrapolated target position along the pursuit direction for an interval t * (starting from an unbiased position).The expected saccadic direction bias Ŝ (shown as the green angle in the figure above) can be expressed as a function of the interval t and the pursuit direction bias P (shown as the blue angle in

Figure S5 :
Figure S5: Relationship between trial-by-trial measures.Each panel illustrates a correlation betwen two of the variables, plotted separately for each observer in Experiment 1. Gray dots are individual trials; black dots are averages binned according to quantiles of the variables plotted in abscissa; and black lines are ordinary linear regression fits.

Figure S6 :
FigureS6: Discrepancy between the observed pursuit velocity in Experiment 2 and the predictions of a weighted average model(Heller et al, 2021).Blue lines represent model predictions, while black dots accompanied by error bars depict the observed data (averaged across observers) and their respective standard errors.The relative weights, denoting the ratio of internal to external motion weighting, were estimated to match the bias in pursuit direction (left panel).Additionally, we introduced a parameter that determines the gain of smooth pursuit.This was estimated by minimizing the mean squared difference between the predicted and observed speeds (right panel).The right panel emphasizes the mismatch between the predicted steep increase in pursuit speed with rising internal speed, predicted by the weighted average model fit on direction data, and the actual observed pursuit speed.