Trajectory prediction of saccadic eye movements using a compressed exponential model

Abstract

Abstract

Abstract:

Abstract  Gaze-contingent display paradigms play an important role in vision research. The time delay due to data transmission from eye tracker to monitor may lead to a misalignment between the gaze direction and image manipulation during eye movements, and therefore compromise the contingency. We present a method to reduce this misalignment by using a compressed exponential function to model the trajectories of saccadic eye movements. Our algorithm was evaluated using experimental data from 1,212 saccades ranging from 3° to 30°, which were collected with an EyeLink 1000 and a Dual-Purkinje Image (DPI) eye tracker. The model fits eye displacement with a high agreement (R² > 0.96). When assuming a 10-millisecond time delay, prediction of 2D saccade trajectories using our model could reduce the misalignment by 30% to 60% with the EyeLink tracker and 20% to 40% with the DPI tracker for saccades larger than 8°. Because a certain number of samples are required for model fitting, the prediction did not offer improvement for most small saccades and the early stages of large saccades. Evaluation was also performed for a simulated 100-Hz gaze-contingent display using the prerecorded saccade data. With prediction, the percentage of misalignment larger than 2° dropped from 45% to 20% for EyeLink and 42% to 26% for DPI data. These results suggest that the saccade-prediction algorithm may help create more accurate gaze-contingent displays.

Introduction

Gaze-contingent display paradigms are commonly used in vision science research, in which the gaze direction is detected by an eye tracker, and image manipulation on the display is applied synchronously according to the gaze direction (Duchowski, Cournia, & Murphy, 2004; Perry & Geisler, 2002; Reder, 1973). As examples, a superimposed mask on a background scene can simulate loss of central vision (Pidcoe & Wetzel, 2006; Varsori, Perez-Fornos, Safran, & Whatham, 2004), whereas a gaze-contingent window with the rest of the scene masked can simulate tunnel vision (Castelhano & Henderson, 2007). These simulations allow researchers to investigate a variety of visual phenomena, including eye movement guidance in reading (Bernard, Scherlen, & Castet, 2007; Scherlen, Bernard, Calabrese, & Castet, 2008), stability of vision (McConkie & Currie, 1996; Poletti, Listorti, & Rucci, 2010), visual search strategies (Bertera, 1988; Geisler, Perry, & Najemnik, 2006), and scene perception (Loschky & McConkie, 2002). 

The spatial alignment of gaze location between the measuring eye tracker and the responding display system is obviously crucial in gaze-contingent studies. However, a misalignment is almost inevitable during eye movements due to time delays between the eye tracker and the display equipment. This system latency includes the time for data transmission from the eye tracker to the display, the processing time for image manipulation, and the time for refreshing the display. The total delay varies based on the equipment and software, and reported values have ranged between 9 and 50 ms (Dorr & Bex, 2011; Saunders & Woods, in press; Schumacher, Allison, & Herpers, 2004; Triesch, Sullivan, Hayhoe, & Ballard, 2002; Yang, Wang, Tong, & Rayner, 2012). For fast saccadic eye movements, that time delay may cause a large misalignment. Take a 20° saccade as an example: the peak velocity can be 350°/s–450°/s (Enderle, Blanchard, & Bronzino, 2005), which may lead to 8° displacement on average within 20 ms. This misalignment is often overlooked, probably because it is often assumed that there is no visual perception during saccades due to visual suppression. However, it has been shown that there is visual perception during saccades (Campbell & Wurtz, 1978; Castet & Masson, 2000; Garcia-Perez & Peli, 2001). Even if perception during saccades does not matter, this misalignment can create a “preview” time at the end of a saccade, allowing the viewer a brief glimpse of areas of the display that are meant to be masked or altered. Even postsaccadic previews that are less than 10 ms could affect perception: for example, Bodelón, Fallah, and Reynolds (2007) reported an average time to perceive grating orientation of 8.4 ms, while McConkie and Loschky (2002) showed detection of global image-resolution changes above chance with 6 ms viewing times. Therefore, minimizing the latency, and therefore the duration of these glimpses, is critical for creating valid simulations of visual experiences such as scotomas and many other uses of gaze-contingent systems. 

Recently, Aguilar and Castet (2011) suggested a misalignment reduction method that updates the display based on eye position at a time as close as possible to the next monitor redraw to minimize the latency. Figure 1a illustrates an example for a system that placed a stimulus (scotoma) at the gaze location with 20 ms latency (Saunders & Woods, in press), which caused a large misalignment (the vertical difference between the red and blue lines in Figure 1a). Aguilar and Castet (2011) reported that they obtained an average of 7–9 ms time delay for saccadic eye movements using their 100 Hz system, though they did not make clear how they determined the delay. 

Figure 1

 

(a) Position of the center of a simulated scotoma (stimulus) relative to the gaze position during one left-to-right saccade using current best practice updating (Aguilar & Castet, 2011). Recorded on an iMac Intel Core i7 (60 Hz display) connected to an Eyelink 1000 (Saunders & Woods, in press). (b) Two alternative strategies to improve updating using the gaze position (blue line). The orange line (triangles) uses prediction of the location of the gaze in the middle of the next frame. The red line (squares) uses prediction of the landing point of the saccade.

To further reduce the impact of the update latency, we propose that the prediction of the gaze location at a future time may be beneficial for gaze-contingent systems that try to maintain a stimulus close to the gaze. Instead of placing the stimulus at the last known gaze location (Aguilar & Castet, 2011), we propose using a prediction about the trajectory of the saccadic eye movement to place the stimulus either at the estimated location of the gaze during the next frame (orange triangles in Figure 1b) or placing it at the estimated landing location (red squares in Figure 1b). While we do not know which approach is best, and the best approach may vary with the application, for the rest of this paper we will report the first approach (next frame prediction) as its benefits are easiest to interpret. The general conclusions of our analyses are the same for the landing-position prediction approach. This approach will provide no benefit to gaze-contingent systems that change only when a saccadic eye movement is detected (i.e., the saccade path is not relevant to the change). 

Prediction of saccade trajectories has been studied before. The main sequence, which denotes the relationship among peak velocity, saccade duration, and saccade amplitude, shows great variability among saccades. The peak velocity, for example, can vary from 200°/s to 500°/s for saccades with the same amplitude. Anliker (1976) presented a method to predict the saccade trajectories after peak velocity by mirroring the data points before the peak velocity, based on the assumption that the velocity function is symmetrical. However, it is well known now that this assumption is not correct, especially for large saccades (Van Opstal & Van Gisbergen, 1987). More recently, Komogortsev and Khan (2009) built a model in Kalman filter form to predict eye movements, including fixations, pursuits, and saccades. The main assumption of the Kalman filter is that the error terms have a Gaussian distribution. However, this assumption does not hold for saccades, since the prediction error is strongly associated with the dynamic procedure of saccades, which include large acceleration and deceleration phases. In addition, Komogortsev and Khan (2009) assumed that saccade duration and amplitude are highly correlated, which is not true as discussed earlier regarding the main sequence. Therefore, their results showed that prediction error for saccades was much worse than for fixations and pursuits. 

In this work, we predict the trajectories of saccadic eye movement by using a predefined model. The plant model of saccades has been investigated for years. The early models date from the suggestion by Westheimer (1954), and they have been well developed (Bahill, Latimer, & Troost, 1980; Enderle et al., 2005; Enderle, Wolfe, & Yates, 1984; Robinson, 1964) as additional features of eye movement have been discovered. In these models, the Voigt elements (a pair of viscosity and elasticity elements in parallel) are used to represent the ocular muscles. Although these models can fit some saccade trajectories very well, they are too computationally intensive to be used in real-time prediction. Generally, they are formulated as a high order differential equation (typically fourth order), in which more than 25 parameters are adjustable (Zhou, Chen, & Enderle, 2009). Determination of these parameters is time consuming. Directly using these models is not feasible for gaze-contingent (i.e., real time) applications, because the regression of so many parameters would require many samples, and would increase latency. In addition, these models do not have elementary function forms for data fitting. 

In this paper, we present a new function to model the trajectories of saccadic eye movements. The main part of this function is a compressed exponential function that has only three adjustable parameters. Based on this function, a new algorithm is proposed to predict the trajectories of saccadic eye movement by curve fitting. The validity of our saccade-prediction algorithm has been evaluated experimentally. We believe that this work can help improve the accuracy of gaze-contingent displays. 

Method

Compressed exponential model

The mathematical model we used for predicting the saccadic eye movement trajectories is:  

 

where p₁, p₂ and p₃ are three parameters that need to be determined in data fitting, and t is the time since saccade onset. It can be seen that the exp[− Display Formula ] function is the key part of the above model (Equation 1). This function is called the generalized exponential function because of the additional parameter p₃. With p₃ = 1, the generalized exponential function becomes the standard exponential function. If 0 < p₃ < 1, the function is called a stretched exponential function, while if p₃ > 1, it is called a compressed exponential function. The introduction of p₃ provides an additional control of the decay pattern of the exponential function. Figure 2a shows typical curves of generalized exponential functions with three different values of p₃ while p₂ = 1. It can be seen that the curve of p₃ = 0.5 decays faster than that of p₃ = 1 at the beginning, and slower at the end. While for p₃ = 2, the decay pattern is reversed. 

Figure 2

 

(a) Curves of three generalized exponential functions with p₁ = 1 and p₂ = 1. (b) An example of fitting (red line) of saccadic eye movement data (blue stars) using the compressed exponential function.

As described by Cardona, Chamberlin, and Marx (2007), the generalized exponential function was first introduced by German physicist Rudolf Kohlrausch in 1854 (Anderssen, Husain, & Loy, 2003), and it was rediscovered in 1970 by Williams and Watts (1970). Since then, this function has found its application in many fields, such as economics (Laherrere & Sornette, 1998) and population models (Murase, Shimada, & Ito, 2010). Here we use this function to model saccadic eye movement trajectories. Specifically, we employ the compressed exponential function, where p₃ > 1, because its curve is similar to the trajectories of saccadic eye movements. 

The three adjustable parameters in the model are nicely associated with different attributes of saccades. The parameter p₁ corresponds to the amplitude of saccade. The parameter p₂ can be used to rescale the saccade duration, and parameter p₃, as mentioned above, is used to control the decay trend (i.e., the asymmetry). A typical curve produced by the model is shown in Figure 2b by the solid line. Comparison with the trajectory of a saccadic eye movement obtained experimentally, shown plotted in Figure 2b as the blue stars, shows good agreement. 

The differential of Equation 1 represents the velocity profile of the saccadic trajectory:  

 

 

This equation is very similar to the probability density function of the Gamma distribution:  

 

where α, β, and γ are three adjustable parameters. Equation 3 was used by Van Opstal and Van Gisbergen (1987) to investigate the skewness of saccadic velocity profiles. The only difference between Equations 2 and 3 is in the exponential component. Equation 3 is a normal exponential function (p₃ = 1), whereas a generalized (compressed) exponential function is employed in Equation 2. These two similar equations seem to fit the velocity profile well, and the similarity indirectly supports the validity of using the compressed exponential model. However, we do not recommend predicting saccades using velocity. The reason is that the velocity is calculated from gaze displacement by the differentials, in which the time-independent noise can be greatly amplified in speed computation if the sampling interval is very short. For instance, a 0.1° random error in gaze point tracking would cause a 6°/s error in speed data for a 60 Hz eye tracking system, and a 100°/s error for a 1000 Hz system. So in our method, the raw data, which is the gaze displacement obtained from eye tracker, are fitted to Equation 1 directly. It should be noted that the integral of Equation 3, which corresponds to the cumulative distribution function of the Gamma distribution, is not an elementary function. Therefore, it is not suitable for real-time saccade trajectory prediction. 

The idea of our method is to fit the eye tracking data to Equation 1, so a non-linear regression is needed. There are many nonlinear regression algorithms that can be used here, but the time consumption of the algorithm is a crucial factor to be considered for real-time prediction. Nonlinear regression functions available in MATLAB computing language (Mathworks, Natick, MA), such as lsqcurvefit( ), take about 15 ms for a 30-data-point fitting, which is too long for our prediction. We developed our own MATLAB code based on the Levenberg–Marquardt algorithm. For 50 data points, the typical time consumption of our code is less than 0.3 ms (CPU 2.8 GHz), which is sufficiently short for our application. 

It should be mentioned that Equation 1 can only predict the distance between the start point of a saccade and a point in the future. In other words, it is a one-dimensional prediction. For a two-dimensional prediction of a point in the future, we use the direction from the start point of the saccade to the most current point. Curvature in saccade trajectories (Minken, Van Opstal, & Van Gisbergen, 1993; van Zoest, Donk, & Van der Stigchel, 2012) is beyond the scope of the present study. 

Algorithms

The flowchart of the current best-practice algorithm used in gaze-contingent displays (Aguilar & Castet, 2011) can be simplified as shown in Figure 3a. At the last possible moment, leaving enough time for any processing, the latest gaze location is sampled, and used to render the display in time for the next frame. 

Figure 3

 

(a) Simplified flowchart of algorithm used in current practice gaze-contingent displays. (b) Flowchart of our saccade-prediction algorithm.

We present a new algorithm as shown in Figure 3b by adding trajectory prediction. The differences between the predictions of the two algorithms are marked by dashed rectangles I to III. The first difference is a buffer for storing gaze samples that will be fitted to our model. The second difference is saccade detection. If the current speed of eye gaze position is larger than a threshold (20°/s in our work), a saccade starting point is flagged. In a real-time gaze-contingent display implementation, blinks may be misinterpreted as downward saccades, at least in the beginning, when using only eye movement speed as the criterion. Aguilar & Castet (2011) advised a solution to exclude blinks based on pupil size. Because the algorithm does not need to provide prediction until the next frame is going to be rendered, the eye movement samples are just stored in the buffer. Once the current speed is lower than the threshold, which indicates the end of the saccade, the buffer is emptied. 

If the speed is still larger than the threshold (ongoing saccade) at the moment the next frame is coming, the prediction will be employed. The rectangle III denotes the process of the prediction. The number of data points in the buffer is checked first to make sure that there is sufficient data for curve fitting. Since our model has only three parameters to be determined, the minimum number of samples required for prediction is three. If there are not enough eye movement samples in the buffer by the time of the next frame refresh, the algorithm has to wait for one more frame. The number of additional samples depends on the display refresh rate and the sampling rate of the eye tracker. In the data analysis conducted for this paper, we arbitrarily limited prediction to when there were at least 10 samples in the buffer. The quality of prediction will be also monitored by the residual of curve fitting of our model. Only if the residual is less than a threshold will the predicted eye position be used (the threshold for average residual was 0.3° per sample in our experiment). Otherwise, the algorithm will simply output the latest gaze position, just like the best-practice algorithm shown in Figure 3a. 

Equipment and participants

Data were collected using an EyeLink 1000 eye tracker (SR Research, Mississauga, ON, Canada) and a Dual-Purkinje Image (DPI) eye tracker (Fourward Technologies, Gallatin, MO). Both eye trackers worked at a 1000 Hz sampling rate. A chinrest was used during the experiments to reduce head movements. Three normally sighted participants (aged 32, 36, and 44 years) were recorded using the EyeLink eye tracker, and two of them were also recorded using the DPI eye tracker. During the eye movement data collection with the EyeLink system, movie clips and static natural scene images were displayed on a monitor while the subjects viewed the images freely. Also, with the EyeLink system, eye movements were monitored while subjects switched their gaze between two stimuli that were 4° or 18° apart. With the DPI system subjects viewed static images of natural scenes. The monitors were 33° by 18° for EyeLink eye tracker data and 19° by 14° for DPI eye tracker data. 

The study followed the tenets of the Declaration of Helsinki and was approved by the institutional review boards at Schepens Eye Research Institute and Boston University. 

Results

To evaluate our saccade-prediction algorithm, saccades were detected offline from the eye movement recording using following criteria: speed >20°/s; the duration >20 ms. In total, 1,212 saccades were detected (Figure 4). 

Figure 4

 

Histogram of the amplitude of detected saccades in the (a) EyeLink eye tracker (n = 546) and (b) DPI eye tracker data (n = 666).

To check the overall goodness of fit of our model for saccadic eye movement trajectories, we used all data points of each detected saccade. Figure 5 shows three typical examples of curve fits of our model for saccades of different amplitudes. The calculated coefficients of determination, R², as shown in each plot, were very close to 1; and the residual error, as shown in the figure, was typically smaller than 0.5°. 

Figure 5

 

Upper panels: Three examples of curve fitting using the compressed exponential model. The blue asterisks are experimental data, while the solid red lines are fitting results. Lower panels: Residual errors associated with the fits in the upper panel.

The coefficients of determination (R²) of all 1,212 saccades as a function of different amplitudes are shown in Figure 6. The R² of all saccades was larger than 0.96, with 98% having an R² > 0.99. These results prove that our model can fit the saccade trajectories well. Overall, R² for smaller saccades tends to be a little lower than for large saccades, presumably because there were fewer data points for small saccades and measurement noise was a greater proportion of the saccade amplitude, reducing the quality of the fit. Nevertheless, all the R² were very high. 

Figure 6

 

The quality (R²) of saccade trajectory fitting using the compressed exponential function for (a) the EyeLink (n = 546) and (b) the DPI data (n = 666). Almost all (98%) R² values were above 0.99. In general, large saccades were fit slightly better.

Figure 7a illustrates the misalignment between the eye position and a stimulus that should be at the gaze location caused by system update latency. The asterisks represent the gaze position detected by the eye-tracking system, while the circles represent the stimulus that could be displayed on the monitor. To reduce congestion in the figure, data points are plotted every 2 ms in Figures 7a and 7b, and in the other plots of the same type. For convenience, the time between commencing the saccade-prediction algorithm and the display update (the system latency) was assumed to be 10 ms, which is shorter than most gaze-contingent systems. The locations of the stimulus shown in Figure 7a were determined by the best-practice method illustrated in Figure 3a, that is, the latest sample data from the eye tracker was used to render the display (Aguilar & Castet, 2011). Figure 7b shows that this 10 ms latency can cause a large misalignment between gaze and stimulus position; up to 2.5° in this example 7° saccade. The error varies with the phase of the saccade. At the beginning of the saccade, the error is relatively small, while in the middle of the saccade, this misalignment is largest, then decreases as the saccade approaches its end (Figure 7b). If the stimulus is a gaze-contingent scotoma, then the latency may result in a 10 ms preview, which might be long enough for visual perception (Bodelón et al., 2007; Clark, Winkielman, & McIntosh, 2008; McConkie & Loschky, 2002; VanRullen & Thorpe, 2001). Longer system latencies, as are common (Saunders & Woods, in press), will result in greater errors and greater risk of preview at the end of the saccade. 

Figure 7

 

Updating using the current-practice approach (Aguilar & Castet, 2011): (a) The gaze position (asterisks) and the location of a gaze-contingent stimulus that should be at the gaze location (red circle) for a representative (7°) saccade. The vertical gap between the two curves is the misalignment. Data points are plotted every 2 ms for clarity. (b) The error (misalignment) for that 7° saccade. (c) A summary of the misalignment errors for all the saccades measured using the EyeLink and (d) the DPI system. The average error in degrees (colored scale to the right) is shown for each saccade-amplitude bin (x-axis) by proportion-of-saccade amplitude bin (y-axis). Contour lines indicate bins that are 10, 20, 30, and 40 ms after saccade onset.

Figure 7c and d shows the misalignment error for all saccades using the best-practice method as a function of saccade amplitude (x-axis) and proportion of saccade amplitude (y-axis), specifically the saccadic displacement used in the prediction divided by the total saccade amplitude. In this analysis, errors were evaluated without considering the frame rate of the display. In other words, it is assumed the display can be updated at every gaze sample. As noted earlier, the refresh cycle can be considered as a source of noise that occurs on top of these predictions. A consideration of the refresh cycle is included below (gaze-contingent simulation for a 100 Hz display). Figure 7c and d shows that the average misalignment exceeds 2° during most portion of the saccade for a wide range of saccade sizes. Figure 7b shows an example of a moderate 7° saccade, for which the error is larger than 1° during most of the saccade. In general, the largest errors occur in the middle of the saccade (amplitude), and were greater for larger saccades. The error increases quickly in the early part of the saccade, then, after it reaches a maximum, the error decreases relatively slowly. This asymmetrical property is consistent with the skewed speed profile of saccades, and it is most pronounced for large amplitude saccades. 

Figure 8 shows the effect of employing our saccade-prediction algorithm. Figure 8a illustrates the same sample saccade as that in Figure 7a, with the prediction shown as the black squares. As we assume the time lag of display updating to be 10 ms, to predict eye position at time t, data from saccade onset to t − 10 ms were used. For example, to predict the saccadic displacement at 32 ms, only data from the start of the saccade to 22 ms were used for the curve fit, and then the location at 32 ms was predicted. The errors found using current practice gaze-contingent updating (Figure 7b) were much larger than those found using saccade prediction (Figure 8b) for that example 7° saccade, especially at about 25 ms, which is around the maximum speed point. 

Figure 8

 

Updating using our saccade-prediction approach: (a) Illustration of the misalignment between gaze position and a gaze-contingent stimulus that should be at the gaze location, for a representative (7°) saccade. (b) The error (misalignment) for that 7° saccade. (c) and (d) The misalignment error for all saccades measured using the EyeLink and the DPI systems. As in Figure 7, average error in each bin is shown. Compared with the current-practice approach shown in Figure 7, the errors were generally smaller after 10 ms from saccade onset, which is when the prediction was first available.

Figure 8c and d shows the average misalignment errors when using the saccade-prediction algorithm, for all 1,212 detected saccades. In general, the errors are smaller than the current best-practice approach (Figure 7c, d). The percentage of bins with an error >3° is 17% in Figure 8c and d compared to 50% in Figure 7c and d. As marked by the contour lines, the errors started to drop after 10 ms from saccade onset, which was when our prediction started to work. 

To show the improvement with saccade prediction more clearly, for EyeLink tracker the differences between Figure 8c and Figure 7c are plotted in Figure 9a. The relative improvement is shown in Figure 9c as the ratio of misalignment reduction (Figure 9a) to misalignment without prediction. Similarly, Figure 9b and d shows the absolute and relative improvement for the DPI tracker. These figures show that the improvements were mainly on the upper-right side of the plots, indicating that the prediction did not reduce the misalignment error shortly after saccade onset and provided little benefit for very short saccades. This is because our algorithm did not perform prediction until at least 10 sample points (corresponding to 10 ms) were received, as marked by the contour lines in Figure 9. For large saccades, the displacement relative to saccade amplitude is still small at 10 ms after saccade onset, but for small saccades it may be already as high as 80%. For saccades larger than 8°, the relative improvement with the EyeLink system was 30% to 60% and with the DPI tracker was 20% to 40%. 

Figure 9

 

The upper panels show the absolute improvement of misalignment with saccade prediction over the last available approach (Aguilar & Castet, 2011) for (a) the EyeLink data and (b) the DPI data. The lower panels show the relative improvement with saccade prediction for (c) the EyeLink data and (d) the DPI data. The contour lines show that the improvement occurred from 10 ms after saccade onset.

To further investigate the effect of our saccade-prediction method in practice, we performed a simulation of a gaze-contingent scotoma for a 100 Hz monitor, with current best-practice and with saccade prediction. The latency between output of the next stimulus location and the display refresh was assumed to be 10 ms. Since asynchrony between frame refresh and saccade initiation can be random, our simulation tested asynchrony of 0 to 9 ms for each saccade. Another factor in an actual gaze-contingent display is the location of the stimulus. A stimulus in the middle of the screen is not immediately rendered when a frame starts. The misalignment errors that could occur due to differences in the time to refresh the stimulus location related to the position of the stimulus on the monitor were not considered. As illustrated in Figure 10a, the simulated scotoma (stimulus) position during a frame (10 ms in duration) remained the same while the eye kept moving. The misalignment errors were analyzed for each sample of eye position. 

Figure 10

 

(a) An illustration of stimulus location compared to the gaze location (asterisks), when considering the display refresh cycle, for current practice (red circles) and saccade-prediction (black squares). Asynchrony between frame refresh and saccade onset can be random. For the 100 Hz display, simulation was conducted with asynchrony of 0 to 9 ms for each saccade. (b) Histogram of misalignment errors in the gaze-contingent display simulation with current practice and with saccade prediction for the EyeLink system. (c) Same as panel b, for the DPI system. For both tracking systems, saccade prediction resulted in more small errors, and fewer large errors than current practice.

Figure 10b and c is a histogram of the misalignment errors in the simulation experiment with the current-practice and saccade-prediction methods (corresponding to Figure 3a and b, respectively) for the two gaze-tracking systems. Overall, the prediction resulted in more small errors, and fewer large errors. For instance, the percentage of errors larger than 2° was 20% with saccade prediction and 45% with current practice for the EyeLink system, and 26% with saccade prediction and 42% with current practice for the DPI system. 

Discussion

We have shown that the compressed exponential function can model the trajectories of saccadic eye movements. Using this model, the latency-induced misalignment errors between the actual gaze position and the updated stimulus location can be reduced, as shown in Figures 9 and 10. The benefit of saccade trajectory prediction is confirmed with data collected using EyeLink and DPI eye tracking systems. While overall the results obtained from EyeLink and DPI are similar and consistent, there is a slight difference between the two data sets, and improvement for EyeLink data seemed to be slightly better than for DPI data. It might be due to the different noise level in the data we collected. As the R² results in Figure 6 show, the residual error (probably due to noise) of DPI data was slightly higher (more data points below R² = 0.99) than that of EyeLink data. Because we used raw eye tracking data as ground truth, any noise would diminish improvement. If it is indeed noise, it would not necessarily mean DPI is less accurate than EyeLink. Such noise is normally a measure of high frequency fluctuation rather than validity. 

The model fitting requires a certain minimum amount of history data (theoretically, at least three samples with our model). By the time there are enough history data and prediction needs to be employed, the eye is often in the middle of a saccade (in our experiment we assume at least 10 ms has passed). So for small saccades, our prediction offers benefit only later in a saccade, but for large saccades, our prediction can reduce misalignment from an early stage. In a real-time gaze-contingent display, the prediction does not have to wait for 10 ms. As long as three samples are available for calculating the three parameters, the prediction can be performed if needed, although the accuracy may not be as good as when more samples are available. 

We observed three factors that are not included in our model but affected prediction accuracy. The first was a dynamic overshoot or glissade near the end of many saccades (Bahill, Clark, & Stark, 1975; Weber & Daroff, 1972). In a dynamic overshoot, the eye moves beyond the target, and then quickly returns with no time delay. A glissade is similar to a dynamic overshoot, but the return to the target is slower. Such postsaccade phenomena are visible in the saccade example shown in Figure 1a. As this is not considered in the compressed exponential model, there may be small errors around the end of a saccade, even when 90% of the saccade is used in the curve fitting. 

The second factor was curvature of saccade trajectories (Minken et al., 1993). A recent study suggested that distraction may cause a curved trajectory in cases when saccade latency is short (van Zoest et al., 2012). By visual examination we found many curved-saccade trajectories in our experimental data with both gaze-tracking systems. Curved saccades were present when viewing both static images and videos. Although we updated the trajectory direction dynamically, using the direction from the start point to the latest data point, it could not fully account for the deviation caused by curvature. To better account for the curved trajectories, a method of curvature regression using the history gaze points would be needed. 

The third factor was irregular velocity profiles, as illustrated in Figure 11. By visual inspection, we found such saccades when subjects were viewing complex moving stimuli (videos), viewing static images, and while performing simple step eye movements, both with the EyeLink and with the DPI system. Since these irregular velocity profiles were found with both systems, and the EyeLink system has been shown to produce eye-movement traces similar to a scleral search coil (van der Geest & Frens, 2002), these irregular profiles are unlikely to be errors in our measurement systems. 

Figure 11

 

Examples of saccades with a conventional (black) and two nonconventional velocity profiles (green and orange).

Despite these potential sources of error that are not accounted for in our model, we have shown that this algorithm can predict the gaze position 10 ms into the future during a saccade, with an average error of 2° or less. To implement the algorithm in a gaze-contingent system it is necessary to know the system latency as this determines how far in the future the saccade prediction must be made. Saunders and Woods (in press) have recently reported a low-cost method for direct measurement of system latency that is easy to implement. We expect the prediction will allow more accurate simulations of ocular conditions by reducing inappropriate glimpses. This hypothesis needs to be validated by further psychophysical studies. 

Acknowledgments

This work is supported in part by State Scholarship Fund of China and NIH funding AG041974 and EY019100. We thank Claudia Cherici and Michele Rucci for helping with DPI eye tracking data collection. 

Commercial relationships: none. 

Corresponding author: Peng Han. 

Email: [e-mail]. 

Address: School of Physics and Telecommunication Engineering, South China Normal University, Guangzhou, Guangdong, China.