The parallel programming of landing position in saccadic eye movement sequences

Abstract

Saccadic eye movements occur in sequences, gathering new information about the visual environment to support successful task completion. Here, we examine the control of these saccadic sequences and specifically the extent to which the spatial aspects of the saccadic responses are programmed in parallel. We asked participants to saccade to a series of visual targets and, while they shifted their gaze around the display, we displaced select targets. We found that saccade landing position was deviated toward the previous location of the target suggesting that partial parallel programming of target location information was occurring. The saccade landing position was also affected by the new target location, which demonstrates that the saccade landing position was also partially updated following the shift. This pattern was present even for targets that were the subject of the next fixation. Having a greater preview about the sequence path influenced saccade accuracy with saccades being less affected by relocations when there is less preview information. The results demonstrate that landing positions from a saccade sequence are programmed in parallel and combined with more immediate visual signals.

Introduction

Saccadic eye movements move the fovea from one location of interest to the next. Each fixation is normally preceded by a saccade and followed by another saccade and fixation. This sequential sampling behavior helps us gather useful information about our environments and so guides complex actions and behaviors. This behavior has been described as the Active Vision Cycle (Findlay & Gilchrist, 2003). The majority of research on saccadic eye movements has focused on the control of single movements, or at least studied single saccades in isolation, with relatively fewer studies examining the underlying control of multiple sequential saccades. This is surprising given that no saccade can ever occur in isolation and that saccade sequences are so fundamental to our ability to perform common everyday tasks. 

One key question in this field is the extent and nature of the parallel programming of individual eye movements in a saccade sequence. 

There is now strong converging evidence from a number of studies that eye movements are programmed in parallel. These range from experiments involving everyday tasks, such as tea and sandwich preparation, driving, or reading a book to laboratory-based studies involving restricted visual environments, such as small visual targets on a computer display. Both of these types of tasks show evidence for the parallel programming of visual information for saccadic eye movements. For instance, in food preparation studies, eye movement scan paths show that objects used later in the task are accurately fixated even though they had not been previously fixated or were the subject of a long past fixation (Hayhoe, 2017; Land & Hayhoe, 2001; Rayner, 2009). The research examining this in more strictly controlled experimental visual environments, in which two or three saccade responses have been made or saccade sequences have been preplanned, show improved task performance at future target locations and effects on saccade landing positions, trajectories, and response latencies (Baldauf & Deubel, 2008; Becker & Jurgens, 1979; Bhutani, Ray & Murthy, 2012; De Vries, Hooge & Verstraten, 2014; Gersch, Kowler & Dosher, 2004; Gersch, Kowler, Schnitzer & Dosher, 2009; Godijn & Theeuwes 2002; McPeek, Skavenski & Nakayama, 2000; McSorley et al., 2019; Theeuwes, Kramer, Hahn & Irwin, 1998; Walker & McSorley, 2006). Furthermore, incidental evidence from the very short latencies of secondary saccades of a proportion of corrective saccades also points to the parallel programming of responses with very short intervals between saccades suggested to be only possible if both saccades are programmed at the same time (Amador, Schlag-Rey & Schlag, 1998; Findlay, Brown & Gilchrist, 2001; Godijn & Theeuw es 2002; Hallet, 1978; Hooge & Erkelens, 1996; McPeek et al., 2000; Mokler & Fischer, 1999; Theeuwes et al., 1998; Vivani & Swensson 1982; Weber, Dürr & Fischer, 1998). The apparent compression of visual space while participants make sequences of saccades also supports the parallel programming of saccades (Zimmermann, Morrone & Burr, 2014; Zimmerman, 2015). Evidence from interword and intraword saccade patterns in a reading task shows that their parallel programming may take place in retinocentric and oculocentric reference frameworks (Lavergne et al., 2008; Vergilino & Beauvillain, 2001; Vergilino-Perez & Findlay, 2001; 2006). 

Recently, we have developed a novel paradigm to investigate the parallel programming of multiple saccades over a far greater range of saccades within a sequence (McSorley et al., 2019; submitted). In this paradigm, saccades are made in turn to seven visual targets, either displayed simultaneously or with their onset controlled so that information about the next one, three, or five target(s) was available at any one time. We found consistent evidence across a number of experiments that the parallel programming of saccades to visual targets extended across the entire sequence. The time taken to complete the sequence reduced as more targets were available with both a decrease in number of saccades made and reduced latencies of these saccades. However, along with this speed benefit came some degradation of saccade targeting. As more information about targets was made available, the less accurate the saccades were: a speed-accuracy trade-off. 

The latencies of saccades are reduced when visual information is removed from the display prior to being fixated; this Gap Effect is taken as evidence for a role for disengagement from the currently fixated item before being able to move to the next item (Saslow, 1967). We have found the same effect in the control of the parallel programming of multiple sequential saccadic eye movements (McSorley et al., submitted). 

There are at least two interpretations that could account for parallel programming of saccades (McSorley et al., 2019; submitted). One is that the speed-accuracy trade-off is due to the impact of each isolated target competing to become the endpoint of the next saccade (McSorley et al. 2019). Accuracy worsens as more targets became available and more speeded responses are made. Another is that the speed-accuracy trade-off is due to saccades being programmed on the basis of the holistic overall shape or path of the visual targets, the Gestalt, rather than individual targets themselves (Wagemans, Elder & Kubovy, 2012; Wagemens, Feldman & Gepshtein, 2012). Here again, saccade accuracy would worsen as they are directed to the context dictated by the general path of the targets not the individual targets themselves. They would then be less likely to be corrected and the time taken to complete the target sequence would decrease. Alongside these explanations there may be an effect of visual crowding on the precise isolation of individual visual targets resulting in poorer saccade targeting and more rapid responses reflecting participant's willingness to reduce caution due to this increase target uncertainty and task difficulty (Herzog, Sayim, Chicherov & Manassi, 2015). Obviously each of these explanations does not exclude the others. 

Thus, the speed-accuracy trade-off involved in the parallel programming of multiple saccades to a sequence of visual targets may be the result of poorer isolation of individual visual targets and/or the processing of the holistic shape or Gestalt of the visual target sequence. This raises the question of what spatial aspects of the target and the motor response are being programmed in parallel across the sequence. 

We report two experiments that examine for the first time what spatial aspects of the visuo-motor coding of saccades are being programmed in parallel. We manipulated the location of selected targets across a sequence of visual targets as the next target in the sequence was being saccaded to. We reasoned that changing target locations would have no effect on those that had been programmed in parallel but would show an effect on locations that had not yet been fully programmed. The extent and manner in which the saccades were affected should reveal what the form of underlying parallel visuo-motor programming takes. In Experiment 1, participants were asked to saccade sequentially to seven visual targets. As they saccaded to the first target, one of the remaining targets was relocated a small amount away from the original location, and the current target was removed from the display. Given previous reports of parallel programming across multiple targets we would expect that saccades to targets closer to the current fixation would be less susceptible to any changes to its location. However, it is unclear if the metrical properties of the motor responses to targets further removed from the current fixation position are partially programmed and whether this partial programming would have an effect on the saccade finally made to the target. 

It seems obvious that having more information about the upcoming sequence should lead to, or at least allow, greater parallel programming of the upcoming targets in the sequence. However, it is not clear that it is the individual targets that are being programmed in parallel or whether it is the overall shape and Gestalt of the sequence that promotes a more stable representation of the sequence from which to plan a series of actions. Does having this stable framework work against making rapid changes to individual saccade programming? In order to examine the role of the Gestalt in the parallel programming of eye movements in Experiment 2, we varied the amount of prior information about the sequence while a change is made to the location of the second target. By increasing the amount of prior information participants have about the sequence we are varying the extent to which a stable representation of the sequence can be built up by the participant prior to individual saccade targeting. If the Gestalt had a role to play, we would expect to see saccades to closer targets being less affected by (more resilient to) changes made to the second target location as the amount of prior information increased. 

General methods

Observers

Ten naïve observers participated in the Experiment 1 (8 women), and a separate 10 took part in Experiment 2 (7 women). All were aged between 18 and 21 years old and all had normal, or corrected to normal eyesight. The University of Reading Ethics Board approved the study, and the study was conducted in accordance with the standards described in the 1964 Declaration of Helsinki. Participants provided written informed consent. The authors declare that there is no conflict of interest. 

Apparatus

Participants’ eye movements (left eye only) were recorded using an Eyelink II eye tracker (SR Research, Kanata, Canada), which is a head-mounted eye tracker with a 500 Hz sampling rate and a typical spatial resolution (RMS) of 0.025 deg. Participants placed their chin on a rest, which constrained any head movements and ensured the viewing distance remained at 57 cm. Before the experiment began, the eye tracker was calibrated using a nine-point grid, and then validated using a different grid. Participants were allowed to begin the experiment when there was an average difference of <0.5 degrees between the actual eye position and that predicted from the calibration and the validation. Stimuli were presented on a 21 in. color monitor that had a refresh rate of 75 Hz. 

Stimuli

The fixation stimulus was a white “+” 0.5 degrees in extent. The target stimuli were white circles (also 0.5 degrees) overlaid with a black line (0.35 degrees in length) placed in the center of each target that served to indicate (“point”) to the next target. Each target was shown on the principal or oblique angles relative to preceding target at 6 degrees horizontal and vertical center-to-center separation distance and, thus, the oblique locations being 8.5 degrees. Stimuli were shown on a mid-grey background. 

Design

Participants were instructed to saccade to seven targets in turn. The target locations were, to some extent, randomly generated but organized so the sequence naturally moved in a linear fashion (i.e., locations of the targets were shown so that they appear to progressively move away from the first target). Targets were positioned on the intersections of an unseen square lattice of potential target locations. It is important to note that the target sequence never turned back on itself. This gave participants a sense that the sequence was naturally unfolding as they made their responses. Targets were removed from the display as they were saccaded to. Participants experienced this as “picking off” the targets one-by-one. 

The key manipulation in both experiments was a change in the position of an existing target. Targets could move by 0 (no change), 0.5, 1, or 2 degrees of visual angle away from the original location with an angular component of 0, 5, 10, or 20 degrees (clockwise and counter-clockwise). These are calculated relative to the original target location and the axis of rotation was orthogonal to the imaginary axis connecting the next target to the preceding target (see Figure 1). This gives 28 relocation conditions overall (4 distance positions with 7 possible angular components). There were 12 trials for each trial type. For both experiments, this gave 336 trials overall. The changes made to the display (removal, addition, and displacement of targets) were made during saccade flight in order to take advantage of the reduction of visual sensitivity found during saccade suppression and, thus minimizes the disruption of new visual events on visual processing (Burr, Holt, Johnstone & Ross, 1982; Ross, Morrone Goldberg & Burr, 2001; Zimmermann, Morrone & Binda, 2018). The time at which targets were offset or onset was determined by a position criterion rather than a velocity one as it was found to be more stable, thus, once the eye position crossed an invisible boundary set at two degrees of visual angle from the center of the next target position (either the stepped location fixation cross or the next target) then the to-be-fixated target was extinguished and any new target was displayed. If the next target was not localized with sufficient accuracy, then the trial halts. On no occasion within the experiment did this happen; participants were successful in following instructions and their saccades were generally accurate as defined by this criterion. 

Figure 1.

 

Shows a schematic of an example trial for each experiment: experiment 1 (I) experiment 2 (II). Participants initially fixated a central cross (shown here as dashed lines but actually solid when displayed). This fixation point disappeared then immediately reappears 10 degrees to the left or right on the horizontal meridian (shown as a leftward movement here) to which a saccade was made. During this saccade the sequence is shown. In experiment 1, all seven targets were initially shown, in experiment 2, the number of targets shown during the trial depended on the prior visual information level in that trial (2, 4, or 6 – a PI level of 4 is illustrated here). In both experiments, as the saccade was executed to the next target, it was removed from the display, as illustrated here by dashed lines around the targets not currently displayed. The arrows show the axes around which targets were relocated as saccades to targets were executed. The relocation extent was 0.5, 1, or 2 degrees in extent and made relative to the preceding target. In experiment 1, the target relocated could be any of the remaining targets and always the next target in the sequence in experiment 2.

Experiment 1 and experiment 2 differed in a number of ways. In experiment 1, all seven targets were displayed at the start of the sequence and one of the targets was displaced while the participant made their initial saccade. As a result, the spatial displacement was systematically varied in its position in the forthcoming sequence. In experiment 2, the next two, four, or six targets in the sequence ahead of the current fixation were displayed until all seven were available. We describe this as the prior information (PI). In this experiment, the target displacement was always applied to the target that was two along from the current fixation. Thus, if the next target is N then the displacement is applied to target N+1 during the saccade to target N. This occurred throughout the trial so multiple sample points were taken within each trial. 

Procedure

Participants were first familiarized with the stimuli and the task and were encouraged to carry out as many practice trials as they felt was necessary to become comfortable with the task. The calibration procedure was then carried out. Each trial started with a drift correct procedure in which a small spot was displayed offset from center by 10.5 degrees of visual angle horizontally and 5.3 degrees vertically and once fixated, eye position was accepted, and automatic adjustments were made by the eye-tracking software. Once accepted, a fixation cross was shown centrally for 800 to 1200 ms after which it “stepped” (was removed from display and then reappeared) 10 degrees of visual angle horizontally to the left or right and participants saccaded to the new position. After each sequence of seven targets was fixated, the trial ended and a new drift correct procedure was initiated before commencing the next trial. 

Data analysis

The eye-tracking software includes a parser that was used to identify the start and ends of saccades using a 22 degree per second velocity and 8,000 degrees per second squared acceleration criteria. Further analysis of saccade dynamics and metrics were accomplished offline using Matlab (Mathworks Inc., Natick, MA, USA) to isolate individual saccades and in-house software analysis to calculate averages. In order to get a complete overview of saccade control in the execution of saccade sequences, no exclusion criteria for saccades were adopted. All movements were accepted as being a legitimate response to the target sequences. 

A number of measures were extracted from each saccade: (i) Saccade latency was defined as the amount of time between end points of one saccade and the initiation of the next; and (ii) Saccade accuracy was defined using a measure of landing position error: the Euclidean distance of each saccade end point from its closest target. 

For experiment 1, the condition in which no change was made to target location was used as a baseline for both saccade latencies and saccade accuracy. Much of the subsequently reported data is in terms of this baseline. The primary measures were the changes in saccade latency and saccade accuracy for saccades toward the change targets. However, we also examined changes in saccade latency and saccade accuracy for the first saccade executed after a change was made to see if there were any subtle impacts on saccade programming from the change alone regardless of where the change was. 

For experiment 2, saccade latencies and accuracy to the displaced target location were examined within each trial and comparisons were then made across the extent of target PI available to examine whether more PI about the coming sequence modulated the effect of changes made to target locations. Data in each figure is shown as the average across participants and error bars are within participant (Loftus & Masson, 1994). 

A further aspect of the data considered was the relationship between saccade latencies and saccade accuracy. It might be expected that shorter saccade latencies should result in saccades landing closer to original target locations, whereas those saccades preceded by longer latencies should allow greater reprogramming to take place and would, hence, be more accurately directed to relocated targets (Becker & Jurgens, 1979; McPeek et al., 2000). In order to examine this, we considered two analyses of the relationship between saccade latency and their landing positions: 

1. 1. The time between the target relocation and the onset of the saccade to that target.
2. 2. Time between the preceding saccade and the saccade to the relocated target (the saccade latency or intersaccadic interval).

The first analysis is commonly reported in double-step experiments and known as reprocessing time (Becker & Jurgens, 1979). This is an ideal measure in the case of two saccade responses executed in response to two targets, however, for the longer sequences used here it is less suitable. Take for example experiment 1 here in, which target positions are relocated during the first target directed saccade. The relocated target could be the second, third, and so on, up to the seventh, such that processing time for the relocated target must increase as the time between the relocation and the final saccade to the relocated target increases. Therefore, reprocessing time for the relocated target necessarily increases as the target recedes in the sequence from the first target (e.g., in the condition when the fifth target is relocated during the saccade executed to the first target, the fifth target becomes the saccade target approximately 1 second later). For this reason, it is difficult to know exactly what to expect to see in the saccade accuracy as a function of time when the time between position relocations and first saccade to the relocated target is so large. This analysis was not carried out. However, the second analysis is appropriate for both experiment 1, for the condition when the second target is relocated and for all conditions in experiment 2 and is reported. For this analysis, it might be expected that the longer the latency of the saccade response to the relocated target in position two the more accurate it would be as greater reprocessing time of the new target location may have taken place. 

Results

Experiment 1 results

Effect of target relocation on first saccade latencies and accuracy

For saccade latencies, we found no reliable effects of the target displacement on the saccade latencies of the responses relative to the baseline of the first saccade to the second target (T2; see Table 1): A 2-way ANOVA with target moved (6 levels: T2−7) and relocation extent (3 levels: 0.5, 1, or 2 degrees of visual angle) showed no main effects or interaction (all p values > 0.098). This shows that the latency of the saccade at the time the displacement was made was not affected by the displacement. 

Table 1.

 

Shows average saccade latencies to the targets in milliseconds (with SE in brackets). Baseline latencies to each target location are shown in the first column (i.e., across trials on which targets were not relocated). Saccade latencies to the second target are shown in columns 2 to 4 after the target indicated in each row was relocated. Saccades latencies to each relocated target itself are shown in columns 5 to 7. Column headings (0.5, 1, and 2) show the extent to which target relocated in degrees of visual angle. N/A, not applicable

 

In contrast, first saccade accuracy for the first saccade after target displacement did show a main effect of target moved (Fig. 2): F(5, 45) = 4.40, Mean Square Error (MSE) = 0.013, p = 0.004, η2 = 0.310, but not relocation extent: F(2, 18) = 2.46, MSE = 0.014, p = 0.114, η2 = 0.214) and there was a significant interaction, F(10, 90) = 3.47, MSE = 0.013, p = 0.001, η2 = 0.278. Further contrasts showed the target main effect was due to landing position error of the first saccade after relocation being significantly less affected by the relocation at all targets relative to that induced by a change in T2 and T3 (T2 vs. Ts 4−7, p < 0.025, T2 vs. T3, p = 0.204). This suggests that the relocation of targets other than T2 and T3 had little effect on the response to T2. Indeed, exploration of the interaction showed that the effect of relocation of each target was mixed. The relocation of all targets showed no effect of the extent of the relocation except target position five, which showed a significant difference between the smallest and medium relocation relative to the largest (0.5 vs. 1 p = 0.002, 1 vs. 2, p = 0.009). Although not a particularly clear picture overall, the pattern of results, taken within each target looking across relocation extent, do suggest that changes at T2, and perhaps T3, affected the error of the first saccade such that they were pulled toward them (the error coding is relative to the original target locations so any value greater than zero would show a pull toward the relocated position). Whereas changes made to the location of Ts 4 to 7 show little effect on saccade accuracy. Interpreted in this way, these results suggest, surprisingly, that the underlying programming of saccades made to T2 and T3 (the next targets to be saccaded to) were not completely programmed prior to the saccade made to T1 as the landing positions were affected by relocation of the T2 and T3 during the T1 response. 

Effect of relocation on saccade latencies and accuracy to the relocated target

The relocation of the target had no systematic reliable effect on the saccade latency to that target when compared to when the target was not relocated (see Table 1 and 2Figure 3). A 2-way ANOVA with target moved (T2−T7) and relocation extent (3 levels: 0.5, 1, or 1 degree of visual angle) showed no main effects but did show an interaction, which contrasts revealed was driven by the relocation of a single target by one extent only (T3 relocated by 2 degrees p = 0.005; relocations of all other target showed p values > 0.113). 

Figure 2.

 

Shows the effect of target relocation extent (in degrees of visual angle) on the first saccade accuracy (landing position error in degrees of visual angle) from the original T2 location after each target was relocated. Error bars are within participant (Loftus & Masson, 1994).

Figure 3.

 

Shows the effect relocating the target on the first saccade latency (left-hand graph; ms = milliseconds) and accuracy (right-hand graph; landing position error in degrees of visual angle) to the relocated target relative to responses made to that target when it was not relocated as a function of the size of the target relocation in degrees of visual angle. Error bars are within participant (Loftus & Masson, 1994).

First, saccade accuracy to the relocated target relative to when no change is made to the target location is shown in Figure 3 (right-hand graph). Note that accuracy is coded from the original target location, so a positive number represents a saccade landing closer to the new relocated target location. It can be seen that there is a very clear relationship between target relocation extent and saccade landing position with the greater extent of target relocation producing greater effects on saccade accuracy. A 2-way ANOVA with target moved (T2−T7) and relocation extent (3 levels: 0.5, 1, or 2 degrees of visual angle) showed a significant main effect of relocation extent: F(2, 29) = 42.0, MSE = 0.0101, p < 0.0001, η2 = 0.207, but only a trend for target moved: F(5, 45) = 2.349, MSE = 0.013, p = 0.056, η2 = 0.207, with no significant interaction (p = 0.128). Contrasts comparing across relocation extent show that both 0.5.versus 1 degree and 1 versus 2 degrees are significant p values = 0.001. 

The relationship between saccade latency and accuracy to the relocated target

In order to examine the extent of the relationship between saccade latency and saccade accuracy (i.e., whether targets that had been programmed to a greater degree are less sensitive to changes in target positions), the results for saccade responses to the T2 condition only have been isolated. The following steps were taken. Each participant was examined separately. Results are collapsed across position relocation extent (0.5, 1, and 2 degrees) to maintain a sensible number of trials that can be subjected to a time bin analysis (the number of trial drops to 12 per position shift so splitting this number of trials into separate time bins for each participant seems unreasonable). To derive the dependence of saccade accuracy on latency trials on which T2 was relocated was split into three equally sized time bins into terciles in a standard vincentizing procedure (Vincent, 1912): the latency distribution for each participant was isolated, split into three, and the average saccade accuracy determined for each third. Note, the exact time bins used were dictated by the idiosyncratic saccade latencies of each participant (so those with shorter saccade latencies had time bins that were to the quicker side of saccade latencies, whereas those with longer latency responses had time bins that were relatively centered at the longer side of latencies) but averages were taken across these time bins so each tercile collapses across quicker and slower responding participants. We also did the same for trials on which no relocation in position occurred to get a baseline measure of the changes in accuracy across saccade latency. For each participant, the difference between the trials on which target position was moved versus those where no relocation occurred was subtracted such that larger scores indicated a greater influence of the new target position. These data were subjected to a 1-way repeated measures ANOVA with time bin as a factor with three levels (short, medium, and long). We would expect to see an increase in the difference scores as latency increased, however, no such difference was found, F(2,9) = 1.07, MSE = 0.011, p = 0.365 (see Figure 4). 

Figure 4.

 

Shows average saccade accuracy (in degrees) as a function of preceding saccade latency when grouped into terciles (i.e., vincentized) across the latency distributions as determined for each individual participant prior to being averaged. Error bars are within participant (Loftus & Masson, 1994).

Conclusion

Overall, we found little evidence that saccade latencies were affected in any meaningful way but saccade accuracy was significantly affected. The results show that the saccade landing position was changed by target relocation, which suggests that the relocated position was feeding into the programming of the response. Specifically, saccade accuracy was affected by target relocation and this effect was modulated by the size of the relocation. We find no reliable evidence that this effect was modulated by how far ahead in the sequence of saccades the relation occurred. This suggests that the programming of saccade landing position was occurring for all seven targets in the sequence. Note however that there is some tendency for the displacement to follow the order from T2 to T7. This pattern of results suggests that new target position information can affect saccade accuracy across all target locations, even those targets that are the subject of the next fixation. Coupled with previous results showing evidence for parallel programming of saccades, this suggests that this programming is still sensitive to changes made to future visual targets in relatively long sequences of saccades. 

Experiment 2 results

Experiment 1 shows that changes made to the locations of future targets in a sequence affect the saccade programs to those locations. Surprisingly this even included the next target in the sequence. In order to examine whether there was a role to play for the overall shape or Gestalt of the target sequence in the parallel programming of eye movements, here, the amount of prior information about the sequence was varied while a change was made to the location of the second target. The more information participants have about the target sequence, the greater is the extent to which a stable representation of the sequence can be built up prior to individual saccade targeting. If the Gestalt had a role to play, we would expect to see saccades to closer targets being less affected by (more resilient to) changes made to the second target (T2) location as the amount of prior information increased. 

Effect of increasing prior information and target relocation on first saccade latencies and accuracy

Figure 5 shows effect of target location on saccade latency (left-hand graph) and saccade accuracy (landing position distance from original target location in degrees of visual angle; right-hand graph) as a function of the extent of target relocation of T2 and amount of PI about the target sequence. Data are shown as subtraction from a baseline condition in which there was no target relocation within the appropriate level of PI (e.g., PI 2 relocation of 1 degree subtracted from PI 2 no relocation). The baseline latencies for each PI level were: PI 2: 252 ms (3.9); PI 4: 251 ms (3.1); and PI 6: 241 ms (2.1). Baseline accuracy was PI 2: 1.30 degrees (0.02); PI 4: 1.14 degrees (0.03); and PI 6: 1.39 (0.06) degrees of visual angle. 

Figure 5.

 

Shows the effect of target relocation and Prior Information (PI) on saccade latency (ms = milliseconds) and saccade accuracy (landing position error in degrees of visual angle, degrees). Error bars are within participant (Loftus & Masson, 1994).

Neither PI level nor relocation extent had an effect on saccade latencies: A 2-way ANOVA with PI (3 levels: the next 2, 4, or 6 targets visible at any one time) and relocation extent (3 levels of 0.5, 1, or 2 degrees of visual angle) showed no main effects (PI: F(2, 18) < 1; relocation extent: F(2, 18) = 2.48, MSE = 146.2, p = 0.112, η2 = 0.216) or interaction (F<1). 

On the other hand, saccade accuracy does show a main effect of both PI, F(2, 18) = 26.0, MSE = 0.023, p = < 0.001, η2 = 0.743, and relocation extent, F(2, 18) = 11.0, MSE = 0.001, p = 0.001, η2 = 0.55, and an interaction between them, F(4, 36) = 9.41, MSE = 0.018, p < 0.001, η2 = 0.511. Contrasts show different patterns of the effect of relocation extent for each level of PI. Both PI 2 and 4 are affected by the largest change in target position whereas PI 6 affected more by the medium change (PI 2: 0.5 vs. 1, p = 0.997, 0.5 vs. 2 and 1 vs. 2 p values < 0.003; PI 4: 0.5 vs. 1, p = 0.033, 1 vs. 2 p = 0.043; PI 6: 0.5 vs. 1, p = 0.014, 1 vs. 2, p = 0.014). To take these in turn: when PI is 2, saccade accuracy is little affected by smaller changes to the target location and are directed toward the original target location only until larger changes in the extent of target relocation are made. Whereas when PI is 4 and 6, saccade accuracy is affected across all relocation extents, albeit to different degrees. 

The relationship between saccade latency and accuracy to the relocated target

In order to further examine the processing of the new target location as a function of time, the relationship between saccade latency and saccade accuracy was determined using the same analysis approach performed for the T2 condition of experiment 1. Here, the position of T2 was varied for different path lengths. As the number of trials on which T2 was varied has now increased, this allows us to examine the effect of changes in position extent on saccade latency across latency. To this end, the saccade accuracy for each participant was examined when collapsed across path extent (the path lengths of 2, 4, and 6) but with position relocation extent kept separate (0.5, 1, and 2 degrees). Separating into both path length and position shift would reduce the number of trials available for sensible analysis too much. Thus, saccade accuracy was derived for each extent of position relocation regardless of the path extent in the same vincentizing procedure used for experiment 1 across three equally sized time bins that were participant dependent (the latency distribution for each participant was isolated, split into terciles, and the average saccade accuracy determined for each third). Again, this was compared with the baseline change in saccade accuracy across the vincentized terciles of saccade latency for each participant (see Figure 6). The resulting subtraction of saccade accuracy (with a greater value indicating a greater influence of the new target position) was subjected to an ANOVA with relocation extent (0.5, 1, and 2 degrees) and time bin (short, medium, and long) as factors. Here, there were main effects and an interaction (relocation extent, F(2, 18) = 7.761, MSE = 0.015, p = 0.004, η2 = 0.463; time bin, F(2, 18) = 9.024, MSE = 0.044, p = 0.002, η2 = 0.501, interaction, F(4, 36) = 11.634, MSE = 0.015, p < 0.001, η2 = 0.564). Contrasts show a slightly confusing pattern but one in which saccades are increasingly directed toward the new target position as latency increases but then become more driven by the old target position again as latency increases for the longest extent (relocation extent 0.5 degrees: time bin 1 vs. 2, p = 0.006; 1 vs. 3, p = 0.873; 2 vs. 3, p = 0.110; relocation extent 1 degree: time bin 1 vs. 2, p = 0.009; 1 vs. 3, p = 0.007; 2 vs. 3, p = 0.803; relocation extent 2 degrees: time bin 1 vs. 2, p = 0.04; 1 vs. 3, p = 0.037; 2 vs. 3, p = 0.011). 

Figure 6.

 

Shows average saccade accuracy (in degrees) as a function of preceding saccade latency when grouped into terciles (i.e., vincentized) across the latency distributions as determined for each individual participant prior to being averaged. Error bars are within participant (Loftus & Masson, 1994).

Conclusion

Overall, these results suggest, much like experiment 1, that relatively late changes made to the location of a visual target in a sequence can influence a saccade directed to that target across all levels of prior information about the visual target sequence. Thus, while not detracting from evidence suggesting parallel programming of multiple saccades, these results suggest that any parallel programming of saccade landing position accommodates and allows new information about the next target location to be integrated with existing saccade programming. 

Interestingly, however, the level of PI was also found to change the degree to which target relocation influences saccade accuracy. Saccades made to the next target in the context of a PI of 2 shows saccades directed to the original target location for 0.5 and 1 degree shifts with only 2-degree relocations showing an impact. This suggests that saccades made on the basis of original target locations are programmed in parallel to a greater degree (or at least are more immune to changes in target location) than at greater PI levels. This may reflect an increasing role for the overall shape or Gestalt of the sequence path as PI increases, which leads to a reduction in the extent to which the parallel programming of individual targets are preprogrammed. Equally, it may be the case as PI increases there is an increase in crowding of the individual targets, which may lead to a reduction in the ease at which visual targets in a sequence can be preprogrammed to allow for accurate guiding of saccades. 

General discussion

In order to examine the extent to which the spatial aspects of multiple saccadic responses to a sequence of visual targets are programmed in parallel, we manipulated the locations of the targets as a saccade was being generated to them while also increasing the amount of information available about the sequence. We reasoned that visual targets closer to the current fixation would be preprogrammed to a greater depth; be less susceptible (if at all) to small, medium, or large changes in target locations and would be directed to the original target locations. Those programs coding targets further removed would be more malleable allowing reprogramming to take place (e.g., the targets are likely to be programmed to be less precise and there is more time to adapt the final program). In both experiments, we found that saccade latencies were not affected in any meaningful way by these manipulations but the saccade landing position accuracy was. Saccade accuracy was affected by the relocation of the target showing that the new target position was factored into the programming of the saccadic response executed to all targets, even those that are the subject of the next fixation. We also found that having more information about the sequence path influenced saccade accuracy with saccades to original target locations being less affected by relocations when there is less information about the sequence path. This could suggest an increased role for the parallel programming of the overall shape or Gestalt or potentially an increase in crowding of the individual targets which may lead to a reduction in the ease at which visual targets in a sequence can be preprogrammed to allow for accurate guiding of saccades. Although parallel programming may occur across multiple target locations, as shown by the many studies discussed in the Introduction, the spatial aspects of the underling visuo-motor programming of the saccadic response are still sensitive to changes made to future target locations. Therefore, changes made to the location of a visual target in a sequence can influence a saccade directed to that target across all levels of prior information about the visual target sequence. 

The results also showed some support to suggest a relationship between the timing of the preceding saccade and the saccade to the relocated target (the saccade latency or intersaccade interval) but not a wholly convincing one. Some evidence for a relationship was found with experiment 2 (for the smaller target relocation steps) showing saccades being directed more toward the new target location as saccade latency increased. This suggests that a shorter latency period resulted in saccades being directed toward the original target, whereas an increased reprocessing period allowed saccades to be more driven by the relocated position. However, this was not found for experiment 1 (T2 shift condition only) or the largest target relocation in experiment 2 where no relationship between saccade latency and accuracy was found. 

This lack of consistent pattern might be due to the number of different processes that take place during the on-going execution of sequences of saccades and the difficulty in determining when in a sequence future saccades are being programmed in parallel. For instance, there are a number of reasons why saccade latencies might be longer, which would not result in greater preprogramming and result in changes of saccade accuracy. Saccade latencies may be longer because of fluctuations in internal noise across trials, lower activity baselines across trials, external noise in the display from more difficult saccade competition, or slower accumulation of saccade identification. 

Furthermore, when saccades are executed as part of a longer sequence it is difficult to isolate exactly what to hypothesize on the basis of the latencies of saccades. It might seem sensible to suggest that the longer the saccade latency the more opportunity there is a for parallel programming, but the question becomes one of exactly how far back in a sequence are we suggesting parallel programming takes place? Is it the “just-to-be-executed” saccade, the preceding one, the one before that, or that one before that etc.? We could keep going back in the chain on the assumption that eventually the parallel programming of a future saccade target is not being programmed but it is unclear exactly when in that preceding set of saccades the parallel programming of the future saccade started. Therefore, although on the face of it we might expect to find longer latency saccades should have an effect on saccade accuracy, it is difficult to know exactly which saccades latencies to examine as they would each form part of larger ongoing set of multisaccade sequencing. 

Overall, the results suggest that several mechanisms may be involved in the parallel programming of saccades, which reflect the visual, attentional, and motor processing involved in executing sequential serial responses to multiple targets. For example, each isolated target will compete to become the end point of the next saccade, which may explain why accuracy worsens as more targets became available and more speeded responses would be made (McSorley et al., 2019). Other mechanisms already alluded to and likely to play a role in the parallel programming of multiple saccades are perceptual organization and visual crowding. These will now be discussed in more depth. 

It is a fundamental observation of perception and perceptual experiences that sensations are organized through hierarchical structures given by the perceptual relations between parts and wholes. It is not the individual components of a visual environment that form the basis for these experiences but the organization of them into meaningful conglomerations. This organization reflects the “goodness” of the overall configuration (through such processes as grouping and segregation) is often referred to as a Gestalt from which the experience of the whole is greater than, or at least in some meaningful way, different from that that would be suggested by consideration of the individual components (Wertheimer, 1923; Wagemans, 2018). These perceptual grouping processes have been shown to occur in parallel without the need for attention (e.g., Gilchrist, Humphreys & Riddoch, 1997). Identification of fundamental ways (commonly referred to as laws or factors) in which visual experiences may be organized could be at play in the stimuli used in our experiments. The visual paths used in our experiments consist of small similar looking spots, which appear in isolation (i.e., a noncluttered environment). The manner of display encourages a visual experience not just based on the spots themselves but also on the shape formed by them. In terms of some of the common factors involved in perceptual organization, the visual experience of the participant could be based on grouping of the individual targets by proximity and similarity. Indeed, participants commonly report this experience and see the targets as forming a path or “snake” or target. The task itself of course works against this as the participants are required to saccade to each target in turn, thus accentuating the components. However, it is likely to be the case that the paths formed by the individual targets will be automatically grouped to give a perceptual experience that will be determined, to some extent, by the overall shape of the path itself. It could then be the case that this grouping leads to poorer control of the landing position of saccades and inaccurate targeting of the individual components of the paths. Studies of the programming of saccades to a single target in the presence of an additional distractor have demonstrated that the saccade is directed toward the center of mass of the two items, this global effect (e.g., Findlay, 1982; McSorley, Haggard & Walker, 2009) suggest that even the metrics of a saccade to a single target can be influenced by neighboring items. The global effect has also been shown to be modulated by Gestalt grouping principles, which influence the landing position over time (Findlay & Gilchrist, 1997). Indeed, this sits well with findings showing that processing of a visual environment unfolds by first extracting its overall structure prior to processing the constituent parts (Navon, 1977; Navon, 2003) perhaps with the principles of perceptual organization described playing a role (Kimchi, 2015). 

Another factor that may well influence the ability to effectively individuate visual targets with accurate eye movements is visual crowding. Visual crowding is the well-established finding that the ability to identify visual targets is generally worse when surrounded by other stimuli (Bouma, 1970) and depends on the proximity and similarity relationship between the target and surround (among other things). It has also been suggested to involve grouping processes (Harrison, Mattingly & Remington, 2013; Herzog et al., 2015) and also be due to increases in the uncertainty of positional information (Greenwood, Szinte, Sayim & Cavanagh, 2017). Links to the control of eye movements has been reported with reductions in visual crowding just prior to saccade onset (Greenwood, Bex & Dakin, 2009). Reports have also been made of strong correlations between visual crowding performance, saccade control and other spatial localization tasks suggesting a common pattern of processing. However, recent evidence of idiosyncratic patterns within these suggest that saccade control and visual crowding are linked through a common lower level spatial representation rather than being closely linked (Greenwood et al., 2017). This could then provide an explanation that sits with the perceptual organization of the target path into a larger Gestalt on the basis of pooling of lower level location information of individual targets, which, in turn, increases the positional uncertainty of individual targets and results in an increase in saccade landing position error found here (Yildirim, Meyer & Cornelissen, 2015). 

It has been suggested (McSorley et al., 2019; submitted) that these types of explanations could take place within the context of a general three-stage framework for understanding eye movement control in which bottom-up processing of visual information is intertwined with higher level task priorities and previous experience to produce a final motor output. The visual saliency stage involves bottom-up sensory encoding of stimuli with the goal to compute a saliency map (Itti & Koch, 2000). A second, intermediate stage combines that visual saliency information with top-down goal demands, selection history, and experience to produce a common priority map of movement goals (Awh, Belopolsky & Theeuwes, 2012; Fecteau & Munoz, 2006), which then feeds down into a final motor stage on which motor representations are generated in order to produce serial saccadic eye movements. This final motor output to the saccade generator would have to be the result of a dynamic and changing set of computations of both the visual saliency of the stimuli as the target sequence was revealed throughout the trial, and the priority map as the top-down strategy to follow the shape or Gestalt of the shape was also updated (Awh et al., 2012; Fecteau & Munoz, 2006; Godijn & Theeuwes, 2002; McPeek et al., 2000). This would have to incorporate feed-forward and feedback processes involved in perceptual organization and grouping in which the overall holistic Gestalt of the path is computed rapidly without positional precise information of the individual targets themselves. Evidence for rapid processing, which captures large scale aspects of the visual environment (such as global shape or the gist of a scene), comes from a number of sources (van Rullen & Thorpe, 2001). This has been suggested to then form the basis and guide for recurrent and refined visual processing to provide further scrutiny (Hochstein & Ahissar, 2002), the guidance of attention (Macknik & Martinez-Conde, 2009), or as a way to minimize prediction error (Rao & Ballard, 1999). 

Overall, programming saccades to a series of visual targets is sensitive to new positional information induced by location displacements even when the target is the subject of the next saccade response. Having more information about the sequence path increased this sensitivity, which we have interpreted as reflecting an increased role for perceptual organization and crowding across the sequence path in providing the basis for guidance of saccades rather than the individual targets themselves. Thus, there may be an increased role for the parallel programming of the overall shape or Gestalt and potentially an increase in crowding of the individual targets, which leads to a reduction in the ease with which visual targets in a sequence can be preprogrammed to allow for the accurate guiding of saccades. 

Acknowledgments

Commercial relationships: none. 

Corresponding author: Eugene McSorley. 

Email: [e-mail]. 

Address: School of Psychology & Clinical Language Sciences, University of Reading, Reading, UK.