Davida Teller Award Lecture 2013: The importance of prediction and anticipation in the control of smooth pursuit eye movements

Abstract

Abstract

Abstract:

Abstract  The ability of smooth pursuit eye movements to anticipate the future motion of targets has been known since the pioneering work of Dodge, Travis, and Fox (1930) and Westheimer (1954). This article reviews aspects of anticipatory smooth eye movements, focusing on the roles of the different internal or external cues that initiate anticipatory pursuit. We present new results showing that the anticipatory smooth eye movements evoked by different cues differ substantially, even when the cues are equivalent in the information conveyed about the direction of future target motion. Cues that convey an easily interpretable visualization of the motion path produce faster anticipatory smooth eye movements than the other cues tested, including symbols associated arbitrarily with the path, and the same target motion tested repeatedly over a block of trials. The differences among the cues may be understood within a common predictive framework in which the cues differ in the level of subjective certainty they provide about the future path. Pursuit may be driven by a combined signal in which immediate sensory motion, and the predictions about future motion generated by sets of cues, are weighted according to their respective levels of certainty. Anticipatory smooth eye movements, an overt indicator of expectations and predictions, may not be operating in isolation, but may be part of a global process in which the brain analyzes available cues, formulates predictions, and uses them to control perceptual, motor, and cognitive processes.

Introduction

We need eye movements in order to see clearly. Saccadic eye movements bring selected images to the fovea, the region of highest visual resolution, and smooth eye movements ensure that image velocity stays within a range that is best for visual acuity and visibility. Eye movements perform these functions by relying on an elegant combination of bottom-up sensory signals and top-down high-level signals, where top-down signals include decisions, goals, selective attention, learning, and expectations. 

The question of how bottom-up signals work in concert with top-down signals to control eye movements is a central issue in contemporary research. Although most of the attention has been focused on saccades (e.g., Epelboim et al., 1995; Johansson, Westling, Backstrom, & Flanagan, 2001; Hayhoe & Ballard, 2005; Najemnik & Geisler, 2005; Kowler, 2011; Tatler, Hayhoe, Land, & Ballard, 2011), the interplay of sensory and high-level signals is as important for the control of smooth pursuit (Krauzlis, 2004; Barnes, 2008; Ilg & Their, 2008; Lisberger, 2010; Kowler, 2011; Schütz, Braun, & Gegenfurtner, 2011; Spering & Montagnini, 2011). 

A role for high-level signals in smooth pursuit at first might appear unlikely given two classic observations that illustrate the dependence of pursuit on sensory motion. First, it is not possible to generate smooth pursuit at will across a stationary scene, and, second, it is not possible to fully suppress pursuit in a scene consisting solely of moving targets. Even a small, single point of light moving across an otherwise dark field will elicit pursuit after a brief latency interval—a clear sign of the operation of reflexive sensorimotor circuitry. Yet, despite these failures of volition, top-down signals can intervene in surprising ways. For example, when the direction of motion of the point target is known in advance, pursuit begins in the expected direction of motion before the target itself starts to move. Thus, the mystery: It is not possible to generate pursuit voluntarily, but a different high-level signal, the anticipation of motion, is sufficient to produce the response. 

Anticipatory effects during pursuit raise many fundamental questions, for example: What types of internal signals evoke the anticipatory response, and how does the effect of these signals interact with the effect of the immediate motion signals on the retina? Is anticipation typical of pursuit, or does it only arise in special cases, such as after considerable opportunity to learn the pattern of motion? Is anticipatory pursuit an isolated phenomenon, or is it an unavoidable result of the operations of a brain that is continually predicting upcoming events and planning behavior on the basis of the predictions? 

After a brief review, we will describe some new results on anticipatory smooth eye movements evoked by different types of cues that may shed light on some of these fundamental issues. 

Anticipatory smooth eye movements

Anticipation during smooth pursuit was discovered by Dodge, Travis, and Fox (1930) and by Westheimer (1954) in their pioneering studies of pursuit of periodic target motions. An example is shown in Figure 1, where the eye can be seen pursuing the target with little lag, including a cycle in which the eye changed direction ahead of the target. 

Figure 1

 

Horizontal eye movements (bottom) while pursuing several cycles of sinusoidal motion (top). Arrow shows a cycle in which the eye led the target at the change of direction.

We should not underestimate the significance of these observations in bringing the now-familiar bottom-up versus top-down debate to the study of smooth pursuit. Westheimer (1954) concluded that in addition to control by psycho-optical reflexes, the anticipatory response during pursuit indicates the involvement of high-level influences, such as learning. Incorporating the influence of such high-level processes into models of pursuit posed a major challenge decades ago, when there was far less known about predictive behavior, or the possible neural sources of predictions. Thus, it is not surprising that the earliest models of pursuit elected to focus on the role played by sensory motion signals, with anticipatory effects seen as emerging only under special circumstances (e.g., Dallos & Jones, 1963). 

The view of the importance of prediction during smooth pursuit has changed in more recent times because we now know that anticipation during pursuit is not limited to the tracking of predictable, repetitive motions (e.g., Figure 1). Anticipatory smooth eye movements—smooth pursuit in the direction of the future motion of a target—occur before the expected onset of predictable or unpredictable target motions (e.g., Kowler & Steinman, 1979a, 1979b; Kowler, 1989; Boman & Hotson, 1992; Barnes & Schmid, 2002). When the motion is unpredictable—i.e., the direction, velocity, or onset times are randomly chosen—anticipatory smooth eye movements depend on the properties of the motions seen or tracked in the recent past (Kowler, Martins, & Pavel, 1984, Kowler, 1989; Heinen, Badler, & Ting, 2005; de Hemptinne, Nozaradan, Duvivier, Lefevre, & Missal, 2007; Collins & Barnes, 2009; Maryott, Noyce, & Sekuler, 2011; Santos, Gnang, & Kowler, 2012). 

Anticipatory responses have also been observed during pursuit of targets that are intermittently occluded. Anticipation is involved in that the pursuit during the period of occlusion undergoes changes in velocity or direction that correspond to the motion expected at target reappearance (Becker & Fuchs, 1985; Barnes & Collins, 2008; Orban de Xivry, Missal, & Lefèvre, 2008). 

Anticipatory smooth eye movements can also be evoked by cues that signal the direction of future target motion. The types of cues that have been studied include the color of the target (de Hemptinne, Lefevre, & Missal, 2006, 2008), the location of a stationary fixation target prior to the onset of motion (Krauzlis & Adler, 2001; Santos et al., 2012), the frequency of an auditory tone (Santos et al., 2012), visual symbols that designate the path of motion (Kowler, 1989; Ladda, Eggert, Glasauer, & Straube, 2007; Eggert, Ladda, & Straube, 2009) and expectations derived from knowledge of physical forces governing motion (Suoto & Kerzel, 2013). When the information about future target motion provided by cues conflicts with expectations derived from previously seen motions, the effect of the cues dominates (Kowler, 1989). 

Anticipatory smooth eye movements have been found in rhesus monkeys (Missal & Heinen, 2004; Medina, Carey, & Lisberger, 2005; Badler & Heinen, 2006; de Hemptinne et al., 2006; Yang & Lisberger, 2010), which has facilitated investigations of the possible neural origins of the anticipatory pursuit response. Interest has focused on the supplementary eye field (SEF), an area within dorsomedial frontal cortex that is part of the pursuit pathway. Stimulation of SEF while a monkey is expecting a target to move results in faster anticipatory smooth eye velocities (Missal & Heinen, 2004). In addition, neurons within SEF show firing patterns that correlate with the onset time and the direction of anticipatory smooth eye movements (de Hemptinne et al., 2008). SEF would appear to be a good candidate for integrating sensory motion with expected motion because it receives signals from motion centers, such as the medial superior temporal area (MST), and it plays a role in high-level decisions associated with the planning of saccades (Schall, Stuphorn, & Brown, 2002; Yang, Hwang, Ford, & Heinen, 2010; Berdyyeva & Olson, 2011). 

Developing models of anticipatory smooth eye movements

Anticipatory behavior is, of course, not limited to smooth pursuit. Prediction, and its neural basis, has generated enormous interest over the past years in many domains. Models have been proposed to describe how predictions are generated on the basis of past history or a priori beliefs, and these models have been influential in accounting for key aspects of perceptual phenomena (Kersten & Yuille, 2003), cognitive decision-making (Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010) and motor control (Kording & Wolpert, 2006; Bosco, Delle Monache, & Lacquaniti, 2012), including the planning of saccades (Diaz, Cooper, Rothkopf, & Hayhoe, 2013; Ross & Kowler, 2013; Wu & Kowler, 2013). 

Developing models of anticipatory smooth eye movements, however, presents special challenges that go beyond the formidable task of accounting for the content of the expectation itself. A model of anticipatory smooth eye movements needs not only to account for the generation of the expectation (as a model of decision-making might), but must also include the processes downstream that operate continuously over time to convert the expectation (which also may be changing continuously over time) into the command for a particular eye velocity. The role of retinal motion signals must also be considered because of the inherent conflict between the generation of the anticipatory pursuit (which creates retinal motion by moving the eye away from the target), and the sensori-motor reflexes that act to counteract retinal motion. A model must be compatible with the present-day understanding of the neurophysiological circuitry generating pursuit, while at the same time deal with the many unanswered questions about the neural circuitry underlying the generation of an expectation. Finally, attempts at developing models should be open to the possibility that anticipatory pursuit responses can result from many different and perhaps independently-controlled processes, for example: rote learning of motion, short-term memory for previous motions, learned associations between cues and motion, and the interpretation of symbolic cues disclosing the future motion path. Given the complexity, it is not surprising that very few detailed models of anticipatory smooth eye movements have appeared, and those models that have been proposed have focused mainly on cases where the expectation is generated on the basis of a short-term memory for the motions presented in the recent past (Barnes & Collins, 2008; Orban de Xivry, Coppe, Blohm, & Lefèvre, 2013; Bogadhi, Montagnini, & Masson, 2013; see also Pavel, 1990, for comparisons of different modeling approaches to prediction in eye movements, perception and motor control). 

Cues

The development of models of anticipatory smooth eye movements would be facilitated by a better understanding of the factors that generate anticipatory pursuit. To that end the remainder of this paper focuses on new results that compare the anticipatory smooth eye movements evoked by different types of cues. Previous work has shown that anticipatory smooth eye movements are sensitive to properties of cues, including modality (Santos et al., 2012) and visual configuration (Eggert et al., 2009), but, other than these reports, systematic comparisons of different types of cues are lacking. 

Anticipatory smooth eye movements evoked by cues are interesting for several reasons. One reason is that cues may be the most frequently encountered source of information about the likely pattern of future motion, given that natural environments are replete with cues about motion, embedded in the flow of visual events. Another interesting aspect of cues is that our skills in decoding meaningful symbols can allow information about future motion to be conveyed immediately, without the need for trial-by-trial learning, although trial-by-trial learning may still influence the pursuit response. Finally, although the results presented here are based on cues that were always valid, the paradigm is readily adaptable to variations in cue validity or reliability (Santos, 2014), just as has been done in studies of the effect of cues on perception (Trommershäuser, Kording, & Landy, 2011). 

We tested the role of different types of cues, all within the same stimulus configuration, and all indicating the direction of future motion with perfect validity. Thus, the cues were identical in what information was conveyed, but differed in how the information was conveyed. A process that relies purely on monitoring cue validity would yield identical responses, regardless of the type of cue. Finding differences among the effects of the cues, on the other hand, would suggest the involvement of other processes, such as the perceptual representation of the cue, or the inherent or learned beliefs about the persuasiveness of the information conveyed by different types of cues. 

Symbolic cues structurally linked to the motion path

The first type of cue we tested was an easily interpretable symbol, namely, the visual barrier shown in Figure 2A (inset), which is depicted as blocking access to one of the paths. In the experiment, the target moved down the inverted Y-shaped tube shown in Figure 2A, and then followed the unblocked path. The critical interval for testing the effect of the cue is the interval immediately before the onset of the horizontal component of motion, when the target enters the unblocked oblique arm. An effect of the cue would be shown by finding horizontal pursuit in the expected direction of motion prior to the time the target enters the oblique path. Anticipatory smooth eye movements have been observed previously with this (Kowler, 1989) and similar (Ladda et al., 2007; Eggert et al., 2009) configurations. 

Figure 2

 

Mean horizontal eye velocity (average of n = 21 subject means) when (A) the visual barrier cue indicated the direction of motion (left or right) down the oblique branch, or (B) no cue was present. The dotted line indicates the horizontal velocity of the disc. The disc entered the oblique branch at time = 0 and traveled either to the right (positive values) or left (negative values) Shading indicates ±SE. Insets show the stimuli.

The effect of the visual barrier cue was examined in a group of 21 naive observers (see Details of methods, below). Their smooth eye velocities are shown in Figure 2A, which plots mean eye velocity over time (mean of subject means), where time = 0 is the onset of horizontal target motion, i.e., when the target entered the oblique arm of the path. Horizontal anticipatory smooth eye movements in the direction of the cued motion were evident by about 150 ms before the target entered the horizontal path, as shown by the separation of the average eye velocity traces for the two cued directions. 

Testing the relatively large group of 21 subjects made it possible to do two additional analyses. The first examined individual differences. This was done by first computing an index of the strength of the anticipatory smooth eye movements. We used the ratio of eye velocity to target velocity during the interval ±50 ms around the onset of horizontal target motion. The average ratio (mean of individual subject means) was 0.35 (SD = 0.15; N = 21 subjects). Individual subjects' average ratios (Figure 3) show that while most ratios clustered about the mean, some subjects showed very small (0.1) and large (0.8) ratios. 

Figure 3

 

Mean ratio of eye velocity:target velocity at the time of motion onset (±50 ms) for individual subjects (±1 SD), plotted in ascending order.

We also examined the anticipatory smooth eye movements on each subject's first trial with the cued motion in order to find out whether experience in the laboratory viewing or tracking the cued motion was required to produce the anticipatory response. The ratio of eye velocity to target velocity, averaged over the first trial of each subject, was 0.42 (SD = 0.45). This high ratio on the first trial shows that the relationship between the barrier cue and the direction of target motion had already been established before testing began. Similar results have been found in a group of observers with high-functioning autism spectrum disorder, whose anticipatory smooth eye movements did not differ from those of a typical group of subjects (Aitkin, Santos, & Kowler, 2013). 

The anticipatory eye movements found with the cues (Figure 2A) were not simply a time-shifted version of the pursuit response found without any cues (Figure 2B). Instead, anticipatory eye velocity increased gradually over time. One advantage of a gradual increase in velocity is that it avoids periods of high retinal velocities, either before or after the onset of target motion, that would occur if the onset of pursuit were abrupt and the estimate of the onset time of the motion was either too early or too late. Given that misestimates of the timing, direction, or speed of upcoming motion are inevitable, the gradual increase in eye velocity provides a way to keep image velocities within acceptable ranges despite uncertainties about the upcoming motion (de Hemptinne et al., 2007). 

Barriers, bars, crashes, and repetitions

The barrier cue tested so far (Figure 2A, inset) not only conveys unambiguous information about the path of the target, but also allows an easy and immediate visualization of the motion path. To find out whether this perceptual quality is important in determining cue effectiveness, we compared the performance with the barrier cue to a set of three other cues, all of which disclosed the direction of future motion with perfect validity, but none providing the clear visualization of the path that is conveyed by the barrier cue. In all cases subjects were told about the significance of the cue before any of the trials were run, thus all cues were equally capable of generating equivalent expectations of the future motion. All cues, including the barrier cue, were tested in blocks of 40 trials on a group of four observers. 

One cue was a large, high-contrast filled bar located near the top of the motion path, on the side corresponding to the direction of the horizontal target motion (Figure 4B, inset). This cue evoked anticipatory eye movements, but with a lower velocity than the barrier cue. (Eggert et al., 2009, reported a similar result when comparing the effectiveness of two different types of visual cues). The anticipatory smooth eye movements in response to the bar cue were quite similar to those found in two other situations in which the direction of upcoming target motion was disclosed with perfect validity, namely, (a) a block of 40 trials with no visual cue in which the motion path, right or left, remained the same, and subjects were informed of the direction prior to the trials (Figure 4C); and (b) a block of 40 trials in which the significance of the barrier cue was reversed, so that the moving target crashed through the barrier, instead of following the unblocked path (Figure 4D) (subjects were informed to expect the crash). 

Figure 4

 

Mean horizontal eye velocity (average of n = 4 subject means) when (A) the visual barrier cue indicated the path of the disc (left or right) down the oblique branch; (B) the filled red bar cue indicated the direction of motion (left or right); (C) direction remained the same for a block of 40 trials; and (D) same as (A) except the disc always crashed through the barrier. The dotted line indicates the horizontal velocity of the disc. The disc entered the oblique branch at time = 0 and traveled either to the right (positive values) or left (negative values). Shading indicates ±SE. Insets show the stimuli.

Figure 5 summarizes the anticipatory responses for all four conditions—barrier cue, bar cue, blocked directions, and crash—using the average ratio of eye velocity to target velocity around the onset of the horizontal motion (as was done in Figure 3). Figure 5 shows that the anticipatory responses with the barrier cue exceeded that for any of the other three conditions. Differences among conditions were reliable, F(3, 24) = 6.31, p = 0.0026, and post hoc tests showed that these differences were due solely to the differences between the barrier cue and the three other conditions tested (p < 0.05). 

Figure 5

 

Mean ratio of eye velocity:target velocity at the time of motion onset (±50 ms), averaged over subject means (N = 4 subjects) for the four cuing conditions shown in Figure 4: barrier cue, bar cue, blocked directions, and disc crashing through the barrier. Bars show ±1 SE.

More cues: Alternating directions of motion, cues at motion onset

The barrier cue was compared to a different set of cues in a group of six subjects. Changing the direction of motion between right and left on alternate trials constitutes an internal cue that relies on keeping track of the directions. The alternating directions resulted in anticipatory responses that were slower than the barrier cue (Figure 6B), even though observers were told ahead of time that motion direction would change on alternate trials and were told which direction would be tested on the first trial. Anticipatory responses were also weak when the same visual barrier cue was used, but rather than indicating a change in motion direction, it indicated where a stationary target would move immediately after the onset of motion (Figure 6C; see also Suoto & Kerzel, 2013, for differences in anticipatory processes during and prior to motion). Adding a second moving stimulus to this stationary target configuration to act as a launcher (Badler, Lefèvre, & Missal, 2010) increased anticipatory eye velocity slightly (Figure 6D). The differences among the eye velocity/target velocity ratios (Figure 7) for the four cue types were reliable, F(3, 40) = 14.15; p < 0.001, again due solely to the difference between the barrier and the other cue types (p < 0.01). 

Figure 6

 

Mean horizontal eye velocity (average of n = 6 subject means) when (A) the visual barrier cue indicated the path of the disc (left or right) down the oblique branch; (B) the direction of motion alternated between left and right on consecutive trials; (C) the barrier cue indicated the path and the disc was stationary prior to the onset of oblique motion; and (D) same as (C) except a second disc moved down the tube to act as a launcher. The dotted line indicates the horizontal velocity of the disc. The disc entered the oblique branch at time = 0 and traveled either to the right (positive values) or left (negative values). Shading indicates ±SE. Insets show the stimuli.

Figure 7

 

Mean ratio of eye velocity:target velocity at the time of motion onset (±50 ms), averaged over subject means (n = 6 subjects) for the four cuing conditions shown in Figure 6: barrier cue, alternating directions, disc stationary prior to onset of oblique motion, and disc stationary prior to oblique motion plus launcher. Bars show ±1 SE.

In summary, the seven types of cues tested (Figures 4 through 7) were not equally effective in evoking anticipatory smooth eye movements, even though subjects were told the significance of the cues, and told that all the cues disclosed the direction of future motion with perfect validity. The strongest anticipatory smooth eye movements were found with the barrier cue when it signaled the change in the direction of ongoing motion, the situation that provided the clearest visualization of the motion path. 

Anticipatory responses with self-generated target motion

One type of cue to the direction of future motion, which we may encounter frequently in natural situations, is the internal signal derived from our own motor commands whenever we control the motion of a target by ourselves. This situation was recently studied by Ross and Santos (2014). 

There is prior evidence that the motor signals generated when controlling the motion of a target ourselves are able to trigger anticipatory pursuit responses (Mather & Lackner, 1981; Vercher & Gauthier, 1992). (Interestingly, not all tasks with self-moved targets require pursuit. Many rely primarily on saccades, with the eye leading and the hand catching up [Ballard, Hayhoe, & Pelz, 1995; Epelboim et al., 1995; Johansson et al., 2001; Ko, Poletti, & Rucci, 2010; Zhao & Marquez, 2013]. An example is shown in Figure 8, which shows intersaccadic eye velocities were slow during a task that required guiding a mouse cursor through a displayed maze [Zhao & Marquez, 2013].) 

Figure 8

 

Distributions of velocities of the eye during pauses between saccades while moving a mouse through an overhead maze shown on a display screen. Velocities on the left are taken from pauses during which the mouse was in motion through the maze. Velocities on the right are taken from pauses during which the mouse was relatively stationary and saccades were used to explore the maze in search of the correct path. Data are taken from the experiment of Zhao and Marquez (2013). Velocities are about equal in both, showing there was no pursuit of the moving mouse.

Ross and Santos (2014) studied the role of the motor signals used to control a target that was moved along the same cued or uncued paths illustrated in Figure 2. In their experiment a mouse was used to control the horizontal trajectory of motion of the target (the vertical motion was controlled by the software). The target was moved either along the oblique arm indicated by the cue, or in absence of a cue, along either oblique path, freely chosen by the subjects. 

Evidence for anticipation during pursuit came from finding little or no lag between the change in direction of the mouse and the change in direction of the eye, regardless of whether the mouse was taken down the cued path (Figure 9A), or the path was freely chosen without any visual cue (Figure 9C). The anticipatory pursuit generated by the internal motor signals was about the same as the anticipatory pursuit from the visual barrier cue by itself. This can be seen by comparing the pursuit of the self-moved target (Figure 9A, C) to pursuit of the same visual motion trajectories tested on a subsequent day in the presence of the visual barrier cue, but with no involvement of the mouse (Figure 9B). Substantial lags between eye and target occurred only in the absence of any cues, visual or motor (Figure 9D). Thus, cues derived from one's own motor commands can be as effective as a visual cue, at least for motion trajectories like these, in which the turns were gradual, rather than abrupt. 

Figure 9

 

Mean horizontal eye velocity (average of n = 5 subject means) (shading shows ±1 SE) for eye (blue and green) and mouse (red and magenta) for target motion down the right or left oblique branch of the tube (Figure 2a, inset). (a) Active cued: Subject pursued a disc that was moved by the mouse in the direction indicated by the barrier cue (Figure 2a, inset). (b) Passive cued: Subjects tracked the disc motions recorded previously in the active cued condition. The barrier cue was present. Each subject was tested using only his or her own motions. (c) Active uncued: Same, except no barrier cue was present and subjects choose either branch under instructions to distribute choices approximately equally between both right and left directions. (d) Passive uncued: The stimuli were the disc motions recorded previously in the active uncued condition. No cue was present. Each subject was tested using his or her own motions. Time = 0 indicates when the vertical position of the disc reached the beginning of the oblique branch. Negative velocities represent leftward motion. From Ross, N. M., & Santons, E. (2014). The relative contributions of internal motor cues and external semantic cues to anticipatory smooth pursuit. Proceedings of the Symposium on Eye Tracking, Research and Applications, 183–186, doi:10.1145/2578153.2578179. Copyright 2014, Association for Computing Machinery. Reprinted by permission.

Summary: Anticipatory smooth eye movements were found with all types of cues, but eye velocity depended on cue type

All the types of cues we tested produced anticipatory pursuit. This included the visual cues that were structurally linked to the motion path (the visual barriers), the cues that were linked to the motion path by means of arbitrary associations (the filled bars), and the internal cues that relied on memory for prior motions, namely, sequences in which the direction of motion remained the same, and sequences in which directions alternated on successive trials. Anticipatory smooth eye movements with at least one type of cue, the barrier, were present on the first trial. 

There were differences in the anticipatory eye velocity found with the different types of cues, despite the fact that the cues were always valid, and subjects were given all the information needed to understand the significance of each type of cue. The visual barrier cue produced faster velocity anticipatory eye movements than any of the other types of cues, including the crash condition in which the link between the location of the barrier and the direction of motion was reversed, and the condition in which motion direction remained the same across an entire sequence of 40 trials. The only case where the effect of the barrier cue was small was when the target was stationary prior to the onset of horizontal motion. 

The internal signals that trigger anticipatory smooth eye movements

What might these results suggest about the origin of the signals that generate anticipatory pursuit? One possible origin of the anticipatory signals is within the sensory motion system itself (e.g., areas MT or MST). The involvement of motion signals is consistent with the finding that the cue that evoked the highest velocity anticipatory smooth eye movements, the visual barrier, provided an immediate perceptual visualization of the motion path. The representation of the motion path was hard to override, even during sequences of trials in which the target kept crashing through the barrier. The possibility that signals representing anticipated or implied motion could exist within neural motion areas is supported by prior work showing that neurons in areas MT can be activated by remembered motion (Lui & Pasternak, 2011), or by static visual cues that precede the presentation of motion (Schlack & Albright, 2007; but not all cues are effective; Rao, DeAngelis, & Snyder, 2012). In humans, motion areas MT/MST show activity when viewing static scenes whose configuration implies the presence of motion (Kourtzi & Kanwisher, 2000; but see Lorteije et al., 2011). The involvement of areas that encode ongoing motion may also be consistent with the finding that the barrier cue was more effective when signaling a change of direction of ongoing motion, than before motion onset when the target was still stationary, although other processes connected to differences between the control of fixation and active pursuit might also be relevant. An involvement of sensory motion systems in generating anticipatory smooth eye movements raises the interesting question of whether any anticipatory motion signals also might create distortions in the perceived path of motion (Kerzel & Gegenfurtner, 2003). 

An alternative possibility is that the signals generating anticipatory pursuit might originate from sources that are involved with decisions related to motor planning, rather than with representations of motion. Shichinohe et al. (2009), for example, found that neurons in SEF, an area that has been associated with anticipatory pursuit (Missal & Heinen, 2001, 2004; de Hemptinne et al., 2008), show activity during delay periods following the presentation of cues indicating the direction of future motion of a pursuit target. Although Shichinohe et al. (2009) did not link this result specifically to anticipatory pursuit responses, they concluded that the SEF activity they observed in response to the motion cues was not likely to have originated from sensory motion areas. They suggested that a more likely source would be an area involved with memory, such as dorsolateral prefrontal cortex. 

The role of certainty about the direction of future motion

Is there a common framework within which the differences among effects of the cues can be understood? Each cue was completely valid within the experiment; however, this does not guarantee that the information in each was encoded with the same level of subjective certainty about the direction of upcoming motion. 

The differences among the effects of the cues could be connected to differences in the level of certainty of the expectations each evoked about the direction of motion. For example, extensive prior experience viewing moving objects tells us that it is far more likely that an object will move along an unblocked path than that it will crash through a barrier. Neither the relatively brief exposure to barrier-crashing during the experiment, nor the instructions to expect the crash, may be able to override the years of past experience. By contrast, the association between the filled-bar cue we tested and the direction of motion was arbitrary. The level of certainty attached to an arbitrary cue may be reduced by the knowledge that any arbitrary association between a cue and the direction of motion is temporary and reversible. Expectations derived from the sequence of prior motions, such as in the conditions in which motion direction alternated on each trial, or remained the same within a block of trials, may be represented with relatively low levels of certainty because of knowledge that errors in keeping track of the directions due to lapses of memory or attention are possible, even likely. 

The level of certainty about the properties of the future motion is likely to be relevant to the velocity of the anticipatory smooth pursuit. The lower the certainty about the direction of future motion, the less the pursuit system should rely on anticipation, and the more it should rely on the immediate sensory motion, assuming that the pursuit system is integrating expected motion and immediate motion optimally (e.g., Bogadhi et al., 2013). Thus, cues or signals that evoke lower levels of subjective certainty should produce slower anticipatory movements. 

Clearly, these arguments entail a degree of speculation about the factors that might influence the representations of the information conveyed by cues. Nevertheless, if we accept that such representations are generated, the notion that subjective certainty is important in determining the effectiveness of cues is plausible. Many models of perception and motor control incorporate the idea that the level of certainty accompanying the encoding of a sensory signal, or the certainty with which we evaluate our own motor abilities, are critical in determining how we represent the environment or how we plan actions (Battaglia & Schrater, 2007; Trommershäuser, Maloney, & Landy, 2008; Fetsch, DeAngelis, & Angelaki, 2013). In prior research, the level of certainty was often controlled by manipulating either the noise of the sensory signals, or the validity of the cues. In the results presented here, the cues were always valid. The differences among the effects of the cues came solely from how their structural aspects affected the encoding of the information. 

Significance of anticipatory smooth eye movements

We found large differences in the anticipatory smooth eye movements evoked by different cues that disclosed the direction of future motion of a target. The differences in the effects of the cues may point to separate mechanisms. For example, the barrier cue, which promotes a clear visualization of the motion, might generate representations of expected motion within the sensory motion system. Cues that require keeping track of associations of cues with motions, or require keeping track of the past history of motions, might operate either within the sensory motion system, or elsewhere, such as the areas in frontal cortex involved in memory or decision-making that form part of the pursuit pathway. We have also argued that the differences in the effectiveness of the cues may be due at least in part to differences in the level of subjective certainty attached to the expectation of the future motion. Within such a framework, the pursuit command may be based on a combination of immediate sensory motion with a signal representing predicted motion, projected out 100 to 200 ms in the future. The prediction may be determined by a combination of the available sources of information, including symbolic cues, contextual cues, and memory for recent motions, each weighted according to the subjective level of certainty. 

Eye movements do not operate in a vacuum. Eye movements operate in the context of other ongoing perceptual activities, cognitive decisions, and motor plans. It would be surprising if the brain should devote so much effort to predicting the future solely for the benefit of reducing processing delays for one activity, eye movements. The study of anticipatory smooth eye movements, an overt indicator of predictions and expectations, may be able to shed light on the broader question of how we represent the information in cues, generate predictions of future events, and plan behaviors so as to take the best advantage of those predictions. Models of anticipatory smooth eye movements evoked by cues, and information about the neural correlates of these movements, may provide insights into what have become very timely and central questions about the role of prediction, expectations, prior knowledge, and beliefs in perception, cognition, and motor control. 

Details of methods

The methods below apply to the data shown in Figures 2 through 7. Methods for data shown in Figure 8 are in Zhao and Marquez (2013); methods for data shown in Figure 9 are in Ross and Santos (2014). 

Participants

Twenty-one subjects were tested, either paid volunteers or participants in the undergraduate psychology subject pool compensated by course credit. All were naive about the purpose of the experiments and all had normal vision (no glasses or contact lenses). Procedures were approved by the Rutgers University Institutional Review Board for the Protection of Human Subjects and are in accordance with the Declaration of Helsinki. A subset of this group was tested on the data presented in Figures 4 and 5 (N = 4 subjects), and in Figures 6 and 7 (N = 6). 

Eye movement recording

Eye movements (right eye only) were recorded by the Eyelink 1000 (SR Research, Osgoode, Canada), tower-mounted version, sampling at 1000 Hz. Chin and forehead supports were used to stabilize the head. Viewing was binocular. 

Stimuli

The stimulus was displayed in a fully lighted room on a Viewsonic G90fB 19-in. CRT monitor, 1024 × 768 resolution, refresh rate 60 Hz, viewed from a distance of 118 cm. The display area subtended 16.2° (972 arcmin) horizontally by 12.3° (738 arcmin) vertically. 

The display contained a line drawing (white on black background) of an inverted Y-shaped tube (tube arm width 68 arcmin; Figure 2A, inset). The oblique branches were at a directional angle of 40° from vertical. A disc 58 min arc in diameter, initially located near the top of the tube, moved downward for a vertical distance of 532 min arc and then traveled down either right or left oblique branch. For Figures 2, 3, 6, and 7: The vector velocity of the disc was 177 arcmin/s and the horizontal component of the velocity in the oblique branch was 114 arcmin/s. For Figures 4 and 5: The vector velocity of the disc was 263 arcmin/s and the horizontal component of the velocity in the oblique branch was 169 arcmin/s. The barrier cue (Figures 2A, 4A, and 6A) and bar cue (Figure 4B), when used, were shown at trial onset and remained visible throughout the trial. In all cases except for the blocked-direction condition (Figure 4C) and alternating direction condition (Figure 6B), the direction of motion (right or left) was chosen randomly with equal probability on each trial. In the blocked direction condition, direction remained the same for blocks of 40 trials. In the alternating direction condition, directions alternated right versus left over blocks of 10 trials. The subject was told which direction (right or left) would be tested on the first trial of each block. Experimental sessions contained either 10 trials (Figures 2, 3, 6, and 7) or 40 trials (Figures 4 and 5). 

Procedure

Before each trial the outline drawing of the tube with a fixation cross at the top was shown, along with the barrier or bar cue (if used). Subjects fixated the cross and pressed a button when ready to start the trial. The cross changed to the disc, which began to move down the tube 1 s later. For conditions where the initial position of the disc was the intersection of the oblique arms (Figure 6C), the initial location of fixation was at the disc and the disc remained stationary for the 1-s interval. When a second disc was added to act as a launcher (Figure 6D), the downward motion of the launcher was the same as that of the disc in the basic experiment. The instructions were to pay attention to the motion of the disc, and avoid jumping ahead of the moving disc and look toward the choice point in the Y-shaped tube. If this occurred, subjects were reminded to pay attention to the motion and asked not to jump ahead of the disc. 

Analysis

The onsets and offsets of saccades were determined offline by computing eye velocity during 13-ms samples, with onsets separated by 1 ms. Saccade onsets and offsets were detected using a velocity criterion that was determined and subsequently confirmed for each subject based on an exhaustive examination of analog records of eye position. Criteria (eye velocity during 13-ms intervals) ranged from 7–16°/s (420–960 arcmin/s). Determination of saccade offsets were subjected to the additional constraint that velocity had to be below the criterion for 33 ms, which was long enough to bypass the overshoots typically accompanying saccades. For the determination of smooth pursuit velocity in the time course graphs (Figures 2, 4, and 6), eye velocity was computed for successive 50-ms intervals whose onsets were separated by 2 ms. Intervals containing saccades or blinks were discarded. For determination of eye velocity:target velocity ratios (Figures 3, 5, and 7), eye velocity was calculated over the interval −50 ms to +50 ms relative to the onset of target motion. Velocity intervals containing saccades, blinks, or episodes in which the signal from the tracker was lost due to interference from the eyelid were discarded. 

Acknowledgments

E. Santos and N. Ross were supported by NSF-DGE 0549115. C. Aitkin was supported by a grant from the NJ Governor's Council on Autism Spectrum Disorder. We thank Adrianna Garcia-Torres for technical assistance with a portion of these experiments, and we thank John Wilder, Chia-Chien Wu, Elizabeth Torres, Manish Singh, and Tania Pasternak for valuable discussion and comments. 

Commercial relationships: none. 

Corresponding Author: Eileen Kowler. 

Email: [e-mail]. 

Address: Department of Psychology, Rutgers University, Piscataway, NJ, USA.