Eye movements in a sequential scanning task: Evidence for distributed processing

Abstract

Current models of eye movement control are derived from theories assuming serial processing of single items or from theories based on parallel processing of multiple items at a time. This issue has persisted because most investigated paradigms generated data compatible with both serial and parallel models. Here, we study eye movements in a sequential scanning task, where stimulus n indicates the position of the next stimulus n + 1. We investigate whether eye movements are controlled by sequential attention shifts when the task requires serial order of processing. Our measures of distributed processing in the form of parafoveal-on-foveal effects, long-range modulations of target selection, and skipping saccades provide evidence against models strictly based on serial attention shifts. We conclude that our results lend support to parallel processing as a strategy for eye movement control.

Introduction

Whether items are processed serially or in parallel has led to enduring controversies in psychology (Logan, 2002). While this distinction might be subtle in some cases, the consequences for theories of movement planning are fundamental. For example, development of theories on eye movement control is strongly influenced by the two diverging model categories of serial attention shifts and parallel processing of words (Reichle, Liversedge, Pollatsek, & Rayner, 2009). Even though an empirical distinction of these model classes is difficult, differences are theoretically important because of the models' predictions on limitations of cognitive capacities (Townsend, 1990). Here, we analyzed eye movement control in a sequential task where each stimulus informs about the location of the next stimulus, a paradigm developed to maximize the chances to observe serial control. We focus on effects of distributed processing by analyzing (i) modulations of fixation durations and target selection by upcoming stimulus elements and (ii) skipping saccades to formulate tests of sequential attention shifts as the basis of eye movement control. 

Eye movements probably represent the most sensitive behavioral measure to investigate ongoing cognitive processing (Rayner, 2009). Because of acuity limitations of the visual system, our eyes continuously scan the environment to foveate areas of interest. As a consequence, visual perception is an active process critically based on eye movements (Findlay & Gilchrist, 2003). Since attention and eye movements are closely coupled, eye movement measures can be used to infer attentional processes (Deubel & Schneider, 1996; Kowler, Anderson, Dosher, & Blaser, 1995). A crucial observation is that attention can be divided to multiple unconnected spatial locations at least in some of the tasks requiring sequences of saccades (Baldauf & Deubel, 2008; Gersch, Kowler, Schnitzer, & Dosher, 2009; Godijn & Theeuwes, 2003). Furthermore, Bichot, Rossi, and Desimone (2005) showed that features of stimuli are processed in parallel while eye movements add a serial component by moving selected stimuli for closer inspection onto the fovea, that is, the part of the retina with highest visual acuity. 

In recent years, considerable progress has been made to understand eye movements in complex tasks that require long sequences of fixations. Integration of perceptual, cognitive, and oculomotor levels has led to a number of models on eye movement control during reading (e.g., Engbert, Nuthmann, Richter, & Kliegl, 2005; Reichle, Pollatsek, Fisher, & Rayner, 1998), visual search (e.g., Najemnik & Geisler, 2005), and scene perception (e.g., Itti & Koch, 2001; Nuthmann, Smith, Engbert, & Henderson, 2010). However, this progress on computational models has not resolved the debate on serial versus parallel modeling frameworks (Engbert & Kliegl, 2011). 

SAS models and E-Z Reader

Originally developed as a model for eye movements in reading, the E-Z Reader model (Reichle et al., 1998; Reichle, Warren, & McConnell, 2009) has developed into an important heuristic tool for the interpretation of eye movement tasks (Reichle, Pollatsek, & Rayner, 2012). The model represents the class of sequential attention shift (SAS) models and postulates that allocation of attention is restricted to serial processing of objects/words after a preattentive parallel processing stage. Even though SAS models were first developed for eye movements during reading, the class of models has been extended to a number of other tasks. Thus far, SAS mechanisms have been proposed for scene perception (Henderson, 1992; Rayner & Pollatsek, 1992), visual search (Salvucci, 2001; Williams & Pollatsek, 2007), driving, arithmetic (both by Salvucci, 2001), and most recently in a number of search-like scanning tasks (Reichle et al., 2012). 

Across tasks, SAS models can be defined by two principles related to attention allocation and saccade programming. First, processing is limited to a single stimulus, and after completion of a certain processing stage, attention shifts from the currently processed stimulus to the next. Within a task, attention shifts are always triggered by the same processing event as lexical access of the attended word during reading (Reichle et al., 1998) or deciding whether a target is present within the fixated area during visual search (Becker & Williams, 2011; Rayner, 1995). Second, saccade programs are either initiated in synchrony with an attention shift toward the next stimulus (Engbert & Kliegl, 2001; Heinzle, Hepp, & Martin, 2010; Reichle et al., 2012; Salvucci, 2001) or are programmed toward the next stimulus while processing of the current stimulus is in a final stage (Reichle et al., 1998). 

Recently, Reichle et al. (2012) used the E-Z Reader model to investigate the eye–mind link in 1-D scanning tasks (target word search, z-string reading, and Landolt C search) and emphasized the distinctive role of eye movement control in reading. According to their simulations, saccades and attention shifts are programmed in synchrony during scanning tasks, while initiation of a saccade program toward the next word preceded attention shifts during reading. Furthermore, the authors concluded that the SAS mechanism seems flexible enough to guide eye movements in tasks other than reading. 

Since eye movements are closely related to the serial progression of attention, a number of hypotheses can be derived for SAS models across tasks. First, due to the attentional shift operation, properties of upcoming stimuli do not influence fixation durations. In contrast, parafoveal-on-foveal (PoF) effects, defined as the modulation of fixation durations by the next word/stimulus, have been reported during reading (e.g., Kliegl, Nuthmann, & Engbert, 2006) and visual search (Trukenbrod & Engbert, submitted for publication; Williams & Pollatsek, 2007; but see Rayner, Pollatsek, Drieghe, Slattery, & Reichle, 2007; Kliegl, 2007, for a discussion of PoF effects). Since PoF effects are generally stronger when fixations are close to the next word/stimulus, saccadic undershoots have been suggested as the main cause for PoF effects (Drieghe, 2008; Williams & Pollatsek, 2007). From this perspective, PoF effects arise from mislocated fixations that were intended to fixate the next word/stimulus. Here, we suggest to examine refixation behavior in order to minimize the role of mislocated fixations when analyzing PoF effects. Both the decision to refixate and the fixation duration prior to a refixation are determined before attention moves away from the fixated stimulus. Interestingly, this assumption is also true for the decision to move to the next stimulus. Thus, even if a refixation results from a saccadic undershoot in SAS models, the decision was based on the refixated item. 

Second, according to SAS models, saccades will be directed toward attended or soon-to-be-attended stimuli. Hence, stimulus elements beyond the next saccade target have not yet been in the focus of attention and cannot influence target selection or fixation durations. The existence of long-range modulations, that is, the farthest stimulus that affects eye movements is significantly different from the next saccade target, contradicts sequential attention shifts as the basis of eye movement control. 

Third, SAS models predict increased fixation durations before skipping saccades. Skipping costs inevitably arise in SAS models due to the cancellation of a saccade program to stimulus n + 1 and the initiation of a new saccade program to stimulus n + 2. Saccade cancellation and initiation are time-consuming and induce prolonged fixation durations prior to skippings. Hence, the observation of skipping benefits, that is, reduced fixation durations before skippings, is incompatible with the SAS framework. Both skipping costs (Pollatsek, Rayner, & Balota, 1986; Pynte, Kennedy, & Ducrot, 2004; Rayner, Ashby, Pollatsek, & Reichle, 2004) and skipping benefits have been reported in reading experiments (Drieghe, Brysbaert, Desmet, & De Baecke, 2004; Radach & Heller, 2000). Following a corpus analysis, Kliegl and Engbert (2005) suggested word length (or word frequency) as the mediating factor, since skipping costs arose before skipping of long (or low-frequency) words, while skipping benefits arose before skippings of short (or high-frequency) words. 

The rationale behind the experimental paradigm studied here was to check the compatibility of serial processing as the basis of eye movement control in a serial task. Our approach is based on a sequential scanning task in which each stimulus informs about the position of the next task-relevant stimulus (Trukenbrod & Engbert, 2007; see also Greene & Rayner, 2001; Hooge & Erkelens, 1998). Participants were instructed to identify a path embedded into a display of Landolt Cs (Figure 1). Gaps of Landolt Cs pointed toward the next stimulus in a sequence. A trial started with a highlighted stimulus. In the example, the gap of the first stimulus is on the right side and points toward the next stimulus to the right. The next two stimulus elements also point rightward. A sequence ends on a target symbol consisting of a stimulus with four gaps. According to SAS models, participants need to recognize the orientation of the gap in the currently fixated Landolt C before attention can be shifted toward the next stimulus. 

Figure 1

 

Example for the display in the sequential search task. Participants were instructed to follow a path of Landolt Cs. The gap of each stimulus element indicates the movement direction to the next symbol. The sequence started on a bold symbol, the target consisted of a ring with four gaps. A typical eye movement sequence is plotted by the line trace.

Methods

Participants

Thirty students of the University of Potsdam participated in an experiment consisting of two blocks tested in separate sessions. Data from one block are reported in this study. In the other block, the same type of stimulus material was used with a different instruction. Sessions lasted about 1 h and blocks were counterbalanced across participants. All subjects reported normal or corrected-to-normal vision and received study credits or were paid 10€ for participation. 

Procedure and materials

Participants performed 50 trials consisting of sequences containing 52 to 55 symbols. Each sequence was embedded into a square display of 18 × 18 aligned symbols (Figure 1). Stimulus elements were black Landolt Cs (size: 0.86° when presented centrally, line width: 0.18°) with a gap (size: 0.23°) in one of four cardinal positions (0°, 90°, 180°, 270°) presented on a light gray background. Stimulus elements were placed on an invisible grid with a distance of 1.73° between stimulus centers resulting in a quadratic arrangement of 30.21° width. The first stimulus element in a sequence was presented with a bold line (0.27°). Gaps of Landolt Cs pointed to the position of the next stimulus. Participants scanned each sequence until they found a target stimulus, which was a ring with four small gaps (size: 0.05°). 

Eye movement recording and data preprocessing

We recorded eye movements using the EYELINK-II system (SR Research, Osgoode, ON, Canada) with a sampling rate of 500 Hz and an instrumental spatial resolution of less than 0.01°. Participants were seated 50 cm in front of a 21-in. Iiyama Vision Master Pro 514 CRT monitor (1024 × 768 resolution, refresh rate of 100 Hz) with their head on a chin rest to reduce head movements. Stimulus presentation and response collection were controlled by an Apple G3 computer and implemented in MATLAB (The MathWorks, Natick, MA, USA) using the Psychophysics (Brainard, 1997; Pelli, 1997) and Eyelink (Cornelissen, Peters, & Palmer, 2002) toolboxes. 

Saccades were detected using a velocity-based algorithm (Engbert & Mergenthaler, 2006) with a relative velocity threshold of 5 standard deviations, a minimum duration of 6 data samples, and minimum amplitude of 1/3 symbol size. For each fixation, we calculated a mean position averaged across both eyes and assigned it to the closest symbol. Poorly calibrated trials, fixations on the first and last symbols, as well as fixations/saccades containing blinks were excluded from further analyses. All results are based on first-pass scanning of the sequence. Eye movements after saccadic errors and regressions were excluded from further analysis until participants inspected a previously unexplored part of the sequence. Overall, 48,419 fixations remained for further analyses. 

Eye movement classification

Examples of different saccade types are plotted in Figure 2. 

Figure 2

 

Illustration of different saccade types. Stimulus elements on the path are plotted in black, and gray stimulus elements represent distractors. Average eye position is drawn with a red line where dashed parts represent saccades and solid lines represent fixations, respectively. (a) Forward saccades. (b) Skipping saccade. (c) Refixation and regression. (d) Erroneous saccades. Red arrows indicate movement direction of eye traces in each example.

Symbols belonging to the path are plotted in black and distractors in gray. Single fixations in Figure 2a are connected by forward saccades from the currently fixated stimulus n to the next stimulus n + 1. Figure 2b displays an immediate saccade from stimulus n to stimulus n + 2, that is, a skipping. A refixation followed by a regression is depicted in Figure 2c. After fixating the stimulus in the lower left, a second saccade changes the fixation position within the stimulus n. Subsequently, the eyes move back to a previously inspected stimulus n − 1. Finally, Figure 2d shows saccades that move the eyes away from the sequence. In this example, two distractors are fixated in close succession. 

Results

A representative eye trajectory is displayed in Figure 1. Participants scanned the embedded sequences of Landolt Cs. Fixations were typically located on stimuli belonging to the path. First-pass analysis revealed that most saccades were directed toward the next symbol n + 1 (forward saccades: 62%) or changed the fixation position within a fixated stimulus n (refixations: 19%). Participants occasionally skipped over the next stimulus n + 1 and moved immediately to a more distant stimulus n + 2 (skippings: 9%) or moved backward to symbol n − 1 (regressions: 3%). A small fraction of fixations was placed on distractors outside of the required sequence (errors: 5%). All others cases were negligible. 

Fixation durations

In a first step, we analyzed the effect of upcoming stimuli on fixation durations. Distance of the next direction change was determined within straight segments of Landolt Cs and was computed as the remaining number of stimuli pointing into the same direction. Fixation durations before refixations, forward saccades, and skippings are plotted in Figure 3. 

Figure 3

 

Fixation durations. The influence of the next direction change on fixation durations is plotted as a function of the distance to the next stimulus pointing into a direction different from the current movement direction. Fixation durations of refixations, forward saccades, and skippings are shown for saccades executed from the first stimulus (green lines), second to fourth stimulus (red dashed lines), and averaged across saccades executed from second to fourth stimulus (red solid lines).

Durations of fixations on the first stimulus of a segment (green lines) were longer than fixations on later stimulus elements (red lines). Since fixation durations were very similar after fixation of the first stimulus in a segment, we calculated average fixation durations (solid red lines) from the second to fourth stimulus (dashed red lines). In line with a possible parafoveal-on-foveal effect, we observed longer fixation durations before all forward saccades and before refixations on the second to fourth stimulus, when the next symbol n +1 pointed into a new direction. As argued earlier, according to SAS models, fixation durations before refixations are least affected by mislocated fixations. Hence, the observed modulations before refixations can be interpreted as strong evidence against eye guidance by sequential attention shifts in this type of task. More distant direction changes, however, had no additional effect on fixation durations. 

We statistically validated this observation by computing a linear mixed model for log fixation durations using the glmer function of the lme4 package (Bates & Maechler, 2010) implemented in the R system for statistical computing (version 2.12.0; R Development Core Team, 2010) under the GNU General Public License (version 2, June 1991). Helmert contrasts were used to estimate the influence of distance on fixation durations (Table 1). In summary, we observed substantial support for parafoveal-on-foveal effects, that is, a reliable influence of the next stimulus n + 1 on the current fixation duration before refixations on the second to fourth stimulus and before all forward saccades (highlighted in bold font, all t ≥ 3). Interestingly, the control of fixation durations was primarily limited to stimulus elements close to fixation, since our analyses revealed no or only weak evidence for reliable long-range modulations of fixation durations (all t < 3). ¹  

Table 1

 

Helmert contrasts testing long-range modulations. The table shows estimated coefficient, standard error, and t-value of the specified contrasts computed with a general linear mixed model for the log fixation durations of refixations, forward saccades, and skippings. Except for the intercept, contrasts compare fixation durations on the nth stimulus before a change with fixation durations on stimulus elements further away from the change (>n). Reliable differences (t > 3) are highlighted with bold font.

Saccade type	First			Second+
	Estimate	SE	t	Estimate	SE	t
Refixations
(Intercept)	5.4262	0.026	208.39	5.3164	0.027	194.01
1 vs. 2–7	0.0005	0.002	0.25	0.0119	0.003	4.29
2 vs. 3–7	0.0013	0.003	0.50	0.0045	0.004	1.07
3 vs. 4–7	0.0010	0.004	0.29	0.0025	0.007	0.38
4 vs. 5–7	0.0026	0.006	0.42	0.0097	0.011	0.88
5 vs. 6–7	0.0050	0.011	0.45	0.0273	0.020	1.33
6 vs. 7	0.0485	0.022	2.17	−0.0488	0.045	−1.09
Forward saccades
(Intercept)	5.5138	0.021	262.22	5.3845	0.021	250.86
1 vs. 2–7	0.0083	0.001	6.05	0.0138	0.001	14.50
2 vs. 3–7	0.0028	0.002	1.54	0.0022	0.001	1.60
3 vs. 4–7	−0.0014	0.003	−0.55	0.0009	0.002	0.44
4 vs. 5–7	0.0061	0.004	1.42	−0.0052	0.003	−1.49
5 vs. 6–7	0.0075	0.008	0.97	0.0014	0.006	0.22
6 vs. 7	−0.0077	0.016	−0.49	−0.0116	0.014	−0.82
Skippings
(Intercept)	5.3069	0.032	166.83	5.2645	0.027	198.33
2 vs. 3–7	0.0077	0.007	1.10	0.0004	0.002	0.17
3 vs. 4–7	0.0150	0.008	1.90	0.0023	0.003	0.85
4 vs. 5–7	0.0321	0.013	2.54	0.0036	0.004	0.87
5 vs. 6–7	0.0596	0.022	2.70	0.0048	0.007	0.67
6 vs. 7	0.0355	0.046	0.77	−0.0073	0.015	−0.48

Target selection

Due to the architecture of SAS models, upcoming objects are expected to influence target selection only if attended. Hence, modulations are restricted to stimuli in close proximity to the next saccade target. In order to test this hypothesis, we analyzed the effect of distance of the next direction change on the proportion of refixations, forward saccades, and skippings analogous to the previous analyses (Figure 4). 

Figure 4

 

Target selection. The influence of the next direction change on target selection is plotted as a function of the distance to the next stimulus pointing into a direction different from the current movement direction. Probabilities of refixations, forward saccades, and skippings are shown for saccades executed from the first stimulus (green lines), second to fourth stimulus (red dashed lines), and averaged across saccades executed from second to fourth stimulus (red solid lines).

While distance had almost no effect on the proportion of refixations, forward saccades, and skippings executed from the first stimulus in a sequence (green line), distance had a major impact on saccades from the second to fourth stimulus (red line). For statistical inference, we computed a logistic linear mixed model using the glmer function of the lme4 package (Bates & Maechler, 2010) implemented in the R system for statistical computing (version 2.12.0; R Development Core Team, 2010). In line with SAS models, target selection on the first stimulus was rather locally determined. Only stimulus elements close to the saccade target modulated target selection. However, contrary to predictions of SAS models, Helmert contrasts confirmed reliable long-range modulations of refixations, forward saccades, and skippings extending up to at least five stimulus elements for saccades executed from the second to fourth stimulus in a sequence (Table 2). ² Hence, distant stimulus elements have a strong influence on local eye movement behavior. 

Table 2

 

Helmert contrasts testing long-range modulations. The table shows estimated coefficient, standard error, z-value, and p-value of the specified contrasts computed with a logistic linear mixed model for the likelihood of refixations, forward saccades, and skippings. Except for the intercept, contrasts compare saccade probabilities on the nth stimulus before a change with saccade probabilities on stimulus elements further away from the change (>n). Reliable differences (p < 0.01) are highlighted with bold font.

	First				Second+
	Estimate	SE	z	p	Estimate	SE	z	p
Refixations
(Intercept)	−1.082	0.064	−16.95	<0.001	−2.406	0.084	−28.76	<0.001
1 vs. 2–7	0.016	0.006	2.77	0.006	0.155	0.008	19.92	<0.001
2 vs. 3–7	0.008	0.008	0.98	0.325	0.076	0.012	6.46	<0.001
3 vs. 4–7	0.021	0.011	2.01	0.044	0.072	0.018	3.93	<0.001
4 vs. 5–7	−0.008	0.018	−0.45	0.649	0.054	0.031	1.75	0.080
5 vs. 6–7	−0.020	0.032	−0.62	0.534	−0.005	0.057	−0.09	0.925
6 vs. 7	0.019	0.065	0.30	0.767	0.081	0.124	0.65	0.516
Forward saccades
(Intercept)	0.188	0.078	2.40	0.016	0.450	0.052	8.60	<0.001
1 vs. 2–7	−0.004	0.005	−0.86	0.388	0.068	0.005	15.00	<0.001
2 vs. 3–7	0.003	0.007	0.37	0.710	0.126	0.007	18.84	<0.001
3 vs. 4–7	−0.005	0.009	−0.49	0.622	0.058	0.010	5.90	<0.001
4 vs. 5–7	−0.010	0.016	−0.62	0.534	0.074	0.016	4.69	<0.001
5 vs. 6–7	0.010	0.029	0.35	0.727	0.094	0.028	3.38	0.001
6 vs. 7	0.043	0.058	0.75	0.453	0.093	0.060	1.55	0.122
Skippings
(Intercept)	−2.949	0.136	−21.76	<0.001	−1.219	0.084	−14.48	<0.001
2 vs. 3–7	−0.139	0.017	−8.30	<0.001	−0.176	0.008	−20.89	<0.001
3 vs. 4–7	−0.039	0.019	−2.02	0.044	−0.043	0.011	−3.87	<0.001
4 vs. 5–7	−0.015	0.031	−0.48	0.635	−0.058	0.018	−3.24	0.001
5 vs. 6–7	0.054	0.054	1.00	0.318	−0.045	0.031	−1.45	0.147
6 vs. 7	−0.221	0.113	−1.95	0.051	−0.103	0.066	−1.54	0.123

Skipping benefits

Finally, we compared mean fixation durations before forward saccades and skippings averaged across participants. Instead of the skipping costs predicted by SAS models, we observed skipping benefits, that is, a reduction of fixation durations before saccades that induce skippings of stimulus elements (Table 3). Fixation durations decreased by 37 ms, t(29) = 14.29, p < 0.001. Since fixation durations might interact with the sequence of fixations, we restricted the analysis to a subset of fixations in a second analysis (see also Kliegl & Engbert, 2005). First, to ensure local similarity, we selected single fixations on a stimulus element. Second, to increase similarity of compared fixation sequences, we restricted our analysis to fixations that were preceded and followed by single fixations. Thus, we analyzed the central fixation duration in a triplet of single fixations. Skipping benefits persisted even for these selection criteria, t(29) = 9.13, p < 0.001 (Table 3). 

Table 3

 

Mean fixation duration (M) before executing a saccade to the next symbol (forward) or before skipping the next symbol (skipping); Δ denotes the observed skipping costs/benefits. Skipping costs/benefits were computed for the entire set (Prior FD) and a subset with three successive single fixations (Single FD).

Saccade type	Prior FD			Single FD
	M	SE a	N	M	SE a	N
Forward	254	5.1	30,119b	230	51.2	3500b
Skipping	217	5.2	4060b	199	5.5	560b
Δ	−37***	2.6	30c	−31***	3.4	30c

 

***p < 0.001, two-tailed t-test for paired samples.

 

^aStandard errors (SEs) across subjects.

 

^bNumber of observed events.

 

^cNumber of participants with both forward saccades and skippings.

In order to see whether skipping benefits are robust across different movement directions, we further divided the triplets according to the movement direction of the upcoming saccade (Table 4). Numerically, we observed skipping benefits for all saccade directions. However, for upward movements, effects were not statistically reliable, t(23) = 1.35, p = 0.19 (in all other cases, p < 0.001). To test whether skipping benefits were compensated by additional processing time during the previous and next fixations, we inspected the corresponding fixation durations (Table 4) and observed no compensatory prolongation of adjacent fixations (all p > 0.10). 

Table 4

 

Mean fixation duration (M) before executing a saccade to the next symbol (forward) or before skipping the next symbol (skipping); Δ denotes the observed skipping costs/benefits. Results are based on a subset with three subsequent single fixations on symbols n − 1, n, and n + 1 and were divided into cases according to movement direction of saccades.

Fixation		Horizontal movement
		Leftward			Rightward
		M	SE a	N	M	SE a	N
Single FD n − 1	Forward	257	6.7	759b	243	6.9	721b
	Skipping	223	11.7	109b	243	12.3	210b
	Δ	−34*	12.3	29c	0	11.4	30c
Single FD n	Forward	221	7.8	759b	204	5.5	721b
	Skipping	182	7.5	109b	178	6.0	210b
	Δ	−39***	6.6	29c	−26***	4.9	30c
Single FD n + 1	Forward	235	7.5	759b	222	7.1	721b
	Skipping	248	14.5	109b	218	8.8	210b
	Δ	13	13.3	29c	−4	8.9	30c
		Vertical movement
		Upward			Downward
		M	SE a	N	M	SE a	N
Single FD n − 1	Forward	247	7.0	1073b	277	9.2	947b
	Skipping	236	10.6	85b	257	12.5	156b
	Δ	−11	10.6	24c	−20	11.9	29c
Single FD n	Forward	226	6.5	1073b	268	9.8	947b
	Skipping	211	11.9	85b	224	7.8	156b
	Δ	−15	10.8	24c	−44***	6.1	29c
Single FD n +1	Forward	235	7.5	1073b	273	8.2	947b
	Skipping	248	14.5	85b	257	16.3	156b
	Δ	13	13.3	24c	−16	14.0	29c

 

*p < 0.05; ***p < 0.001, two-tailed t-test for paired samples.

 

^aStandard errors (SEs) across subjects.

 

^bNumber of observed events.

 

^cNumber of participants with both forward saccades and skippings.

Finally, in order to control for a possible confounding effect of priming or by statistics of the paths, we split our analysis of skipping benefits for different positions within straight segments of the path (Figure 5). 

Figure 5

 

Skipping benefits. Fixation durations before forward saccades (red lines) and skippings (green lines) for saccades executed from the first to fourth stimulus in straight segments (panels from left to right). Fixation cases were subdivided into groups according to the next direction change.

Analogous to the previous analyses on long-range modulations, fixation durations are plotted relative to the next direction change for fixations on the first to fourth stimulus in a segment (panels a–d). A linear mixed effect model was estimated for log fixation durations using the glmer function of the lme4 package (Bates & Maechler, 2010). Our analysis revealed a main effect of skipping benefits, longer fixation durations on the first stimulus compared to later stimuli, and a linear trend of distance with longer fixations toward the end of a sequence (Table 5). The only reliable interaction revealed that skipping benefits were smaller on the first compared to the second stimulus. All other interactions were statistically not reliable (all t < 2.00). In summary, skipping benefits were highly reliable and persisted over different movement directions as well as across different positions within the path. 

Table 5

 

Fixed effects of the linear mixed model. Third-order interactions were removed from the model (all t < 2). The table shows estimated coefficient, standard error, and t-value when comparing log fixation durations before forward saccades and skippings. Reliable effects are highlighted in bold font (t > 2).

	Estimate	SE	t
(Intercept)	5.391	0.020	269.08
SKB (Skipping benefits)	−0.104	0.008	−12.75
1 vs. 2	−0.063	0.011	−5.63
1 vs. 3	−0.151	0.013	−11.42
1 vs. 4	−0.132	0.015	−8.65
DTE (Distance to end)	−0.006	0.003	−2.10
DTE × (1 vs. 2)	0.002	0.006	0.33
DTE × (1 vs. 3)	−0.010	0.008	−1.20
DTE × (1 vs. 4)	0.005	0.010	0.49
(1 vs. 2) × SKB	−0.052	0.023	−2.28
(1 vs. 3) × SKB	0.007	0.027	0.27
(1 vs. 4) × SKB	0.061	0.031	1.96
DTE × SKB	0.002	0.005	0.45

Discussion

We were interested in the question of whether eye movements are under control of serial processing of objects in a sequential task. We investigated three critical predictions of sequential attention shift (SAS) models: (i) the absence of parafoveal-on-foveal effects, (ii) the absence of long-range modulations of upcoming stimulus elements, and (iii) the existence of skipping costs. All predictions were refuted for this type of scanning task. First, fixation durations on stimulus n were prolonged by direction changes on the next stimulus n + 1. Most critically, the prolongation was also observed before refixations, excluding a possible role of mislocated fixations. Second, proportion of refixations, forward saccades, and skippings were modulated by distant stimuli. Since these stimuli were far away (up to about five stimuli) from the next saccade target (the restricted locus of attention in SAS models), our findings are incompatible with the SAS framework. Third, instead of prolonged fixation durations before skipping saccades, we observed shortened fixation durations. These skipping benefits were stable across movement directions and persisted when analyses were restricted to selected sequences of fixations. In addition, triplets of single fixations revealed no temporal compensation on preceding or subsequent fixations. Thus, even if skipping benefits resulted from a speed–accuracy trade-off, ³ missing compensatory effects on neighboring stimuli indicate that our data are incompatible with the SAS framework. 

Distributed processing

In SAS models, attention is directed toward single objects. In our analyses, we assumed that individual Landolt Cs represent the unit of an object. However, objects can emerge from multiple stimuli like words that are composed of multiple letters. In a similar way, segments of several Landolt Cs, which point into the same direction, might form larger objects. Consequently, one could argue that long-range modulations of saccadic behavior might still be compatible with SAS models if attention is allocated to objects composed of a varying number of elementary Landolt Cs. While this assumption seems psychologically plausible, we argue that overall eye movement patterns contradict such an interpretation. Attention allocation to objects formed from multiple stimuli should be visible in saccadic landing positions. Nevertheless, forward saccades aimed at the center of the next Landolt C (see Trukenbrod & Engbert, 2007 for detailed analyses of landing-site distributions). 

In saccadic sequences, attention is generally allocated to the location of the next saccade target (e.g., Gersch, Kowler, & Dosher, 2004), while attention allocation to subsequent saccade targets has only been reported infrequently (Baldauf & Deubel, 2008; Godijn & Theeuwes, 2003). In some tasks, this dissociation can be explained by two separate mechanisms (Gersch et al., 2009). While saccade programming facilitates processing at the endpoint of the next saccade, visual layout of the stimulus material boosts processing at subsequent saccade targets. In our experiment, grouping of stimulus elements might indeed support parallel processing of multiple Landolt Cs, which in turn affects eye guidance. Nonetheless, our results demonstrate that distributed processing influences eye movements in a sequential task and contradict eye guidance that is exclusively based on SAS principles. Interestingly, the current version of E-Z Reader (Reichle, Warren et al., 2009) assumes an early stage of parallel processing. This stage has been suggested to modulate target selection, but until now the exact interplay has not been specified. Whether an SAS model with a stronger emphasis on parallel processing is able to account for the observed effects remains open to question. 

Impact on eye guidance

Two more aspects of our results are worth highlighting. First, immediately after a direction change, target selection is solely influenced by stimulus elements close to fixation. Only on subsequent fixations, processing expands to more distant stimulus elements. Obviously, attention allocation is dynamic, where the reallocation of attention is not accomplished immediately, but needs some time to rearrange. In line with this interpretation, Golomb, Pulido, Albrecht, Chun, and Mazer (2010) reported a slow decline of attention at previously attended retinotopic locations. In our task, attention may be allocated to several stimulus elements in straight segments. Immediately after a saccade, attention will be distributed across the retinotopic locations attended prior to the saccade. If the saccade moves the eyes within a straight segment, processing may continue without interruption since the path remains mostly within the attended area. However, after a saccade toward the first stimulus of a segment, most stimulus elements within the attended area are distractors and attention needs time to be reallocated toward the new segment. One prediction of this behavior is the processing of irrelevant symbols. Interestingly, the number of saccadic errors at the end of straight segments is the most frequent error observed in our task. 

Second, even though target selection was affected by distant stimulus elements, fixation durations were primarily controlled by stimulus elements close to fixation. Even though a separation between target selection and control of fixation durations has been suggested previously (Findlay & Walker, 1999), the clear distinction suggested by results from our task is important for eye movement models in general. In particular, it will be interesting to investigate how compatible our results are with models assuming that both decisions of when and where to move the eyes are linked to the same processing event during scanning tasks (Reichle et al., 2012). Hence, the paradigm may be seen as an interesting benchmark for existing models of eye movement control. 

A model of eye movement control

Our interpretation of the results is based on the SWIFT model (Engbert et al., 2005) and can be summarized by three principles. First, visual input is processed in parallel with faster processing close to fixation. The input progresses through a number of different stages, in which features, objects, and higher level information is extracted over time. Second, an autonomous timer initiates saccades after random time intervals that may be prolonged by ongoing processing close to fixation. Thus, the temporal decision to initiate a new saccade is not fully determined by a specific processing event (Nuthmann et al., 2010; Trukenbrod & Engbert, submitted for publication). Third, eye movement targets are computed from a temporally evolving activation field. Since processing of the visual input advances steadily, various factors continuously shape the activation field and respective target-selection probabilities. Interestingly, model simulations with SWIFT demonstrate that these principles generate skipping benefits during reading (Engbert & Kliegl, 2011) and scanning tasks (Trukenbrod & Engbert, submitted for publication). The compatibility with long-range interactions and parafoveal-on-foveal effects, however, still need to be demonstrated. 

Conclusions

Model generalizability is a key concept in model evaluation, analysis, and comparison (Pitt, Myung, & Zhang, 2002). Using a sequential search task, we present experimental results incompatible with the SAS mechanism in a non-reading task. Thus, while the SAS framework might represent a useful approximation of the cognitive processing underlying certain tasks, such an architecture could turn out to be a special case of a more general cognitive architecture underlying visuomotor behavior. Candidates for this more general framework seem to be based on processing by parallel graded attention. 

Acknowledgments

This research was supported by Deutsche Forschungsgemeinschaft (DFG, Grant EN471/1-2 to R.E.). We thank Daniel Schad for his advice on statistical analyses. 

Commercial relationships: none. 

Corresponding author: Hans A. Trukenbrod. 

Email: [e-mail]. 

Address: Universität Potsdam, Exzellenzbereich Kognitionswissenschaften, Karl-Liebknecht-Str. 24-25, 14476 Potsdam, Germany. 

Footnotes

Footnotes

1  We are aware that t > 2 is commonly interpreted as a reliable influence. Due to the problem of multiple comparisons, however, we chose to interpret values of t < 3 as weak or no evidence.

Footnotes

2  Due to multiple comparisons, we chose to interpret p > 0.01 as only weak or no evidence.

Footnotes

3  Findlay (1995) reported a speed–accuracy trade-off in an eye movement task. Since skippings are executed after short fixation durations, saccade targets are less precise and might aim at an intermediate position of multiple objects.