Timing of saccadic eye movements during visual search for multiple targets

Abstract

Abstract

Abstract:

Abstract  Visual search requires sequences of saccades. Many studies have focused on spatial aspects of saccadic decisions, while relatively few (e.g., Hooge & Erkelens, 1999) consider timing. We studied saccadic timing during search for targets (thin circles containing tilted lines) located among nontargets (thicker circles). Tasks required either (a) estimating the mean tilt of the lines, or (b) looking at targets without a concurrent psychophysical task. The visual similarity of targets and nontargets affected both the probability of hitting a target and the saccade rate in both tasks. Saccadic timing also depended on immediate conditions, specifically, (a) the type of currently fixated location (dwell time was longer on targets than nontargets), (b) the type of goal (dwell time was shorter prior to saccades that hit targets), and (c) the ordinal position of the saccade in the sequence. The results show that timing decisions take into account the difficulty of finding targets, as well as the cost of delays. Timing strategies may be a compromise between the attempt to find and locate targets, or other suitable landing locations, using eccentric vision (at the cost of increased dwell times) versus a strategy of exploring less selectively at a rapid rate.

Introduction

The effective performance of visual tasks requires saccadic eye movements to direct the line of sight to sequences of selected locations. Much has been learned in recent years about how we choose both where to aim saccades (e.g., Ballard, Hayhoe, & Pelz, 1995; Eckstein, 2011; Epelboim et al., 1995; Itti & Koch, 2001; Johansson, Westling, Backstrom, & Flanagan, 2001; Kowler, 2011; Land & Hayhoe, 2001; Legge, Klitz, & Tjan, 1997; Malcolm & Henderson, 2010; Melcher & Kowler, 2001; Najemnik & Geisler, 2005; Pelz & Canosa, 2001; Ross & Kowler, 2013; Rothkopf, Ballard, & Hayhoe, 2007; Tatler, Hayhoe, Land, & Ballard, 2011; Torralba, Oliva, Castelhano, & Henderson, 2006; Verghese, 2012), and how we control saccadic timing (e.g., Gold & Shadlen, 2007; Henderson & Smith, 2009; Hooge, Vlaskamp, & Over, 2007; Ludwig, 2009; Nuthmann, Smith, Engbert, & Henderson, 2010; Palmer, Huk, & Shadlen, 2005; Trukenbrod & Engbert, 2012; Yang & McConkie, 2001). The present study investigates the role of timing in the selection of targets and the planning of saccades during visual search. 

Any planned movement can benefit from taking more time to select the goal or to plan the movement trajectory (Rosenbaum, 2009). In the case of the planning of saccades, more time can be beneficial in several ways, such as by allowing time to use cues that signal useful places to look (Araujo, Kowler, & Pavel, 2001; Hooge & Erkelens, 1999), or by allowing time to filter out interference from distracting or extraneous visual details (Cohen, Schnitzer, Gersch, Singh, & Kowler, 2007). 

Despite the benefits of taking more time, there is little evidence supporting preferences to delay saccades in an attempt to locate targets, or to improve saccadic accuracy. Wu, Kwon, and Kowler (2010), for example, found that when a saccadic scanning task was made more difficult by decreasing the size of targets or increasing target eccentricity, the preferred strategy was to use secondary saccades to correct the resulting saccadic landing errors, rather than to prolong saccadic latencies. The reliance on secondary saccades also applies to saccades made to targets surrounded by clutter. Coëffé and O'Regan (1987) were the first to observe that saccades made to a target in the midst of nearby distractors could reach the target faster by aiming at the entire configuration (the so-called “center of gravity” saccades), and then using secondary saccades to correct the landing errors, rather than by delaying the primary saccade long enough to reach the target with a single movement (see also Cohen et al., 2007; Kowler & Pavel, 2013; Stritzke, Trommershäuser, & Gegenfurtner, 2009). 

Coëffé and O'Regan's (1987) results call attention to a tradeoff that operates during saccadic tasks: Taking time to plan saccades carefully may improve the probability that a given saccade hits a chosen target, but at the cost of lowering the overall rate of saccades, and, consequently, the rate of fixation of relevant objects. On the other hand, reducing the time devoted to planning may increase the overall rate of saccades, but risks having too much time invested in making saccades to useless locations. 

Confronted with the choice between slow, but careful, saccadic planning on the one hand, and faster, but less careful, planning on the other, higher saccade rates are often preferred, even at the expense of the selection of useful targets. Araujo et al. (2001), for example, found that visual cues indicating the likely location of the target in a two-location search task were usually ignored, probably because taking the cues into account required an increase in saccadic latency of about 50 ms. Search performance suffered as a result. Hooge and Erkelens (1998) found comparable preferences to ignore cues in search tasks that required sequences of saccades. These results showed a reluctance to delay saccades even when the delays would have been helpful. 

Hooge and Erkelens (1999) found a reluctance to delay saccades in a more complex search task. The target was a single letter O that appeared in an array of Cs. Some of the distractor Cs were drawn with the same thin outline as the target O, while others were drawn with thicker lines of various widths. Fixation pause durations were longer on the thin Cs, which were identical to the targets except for the small gap, than on the thicker Cs. Nevertheless, the line width of the thicker Cs had no effect on pause duration (see their Figure 5). Hooge and Erkelens (1999) concluded that the durations of fixation pauses were determined only by the “foveal task,” in this case, the attempt to detect the small gap in the C, rather than by an attempt to use eccentric information to improve the choice of saccadic landing location. Their conclusions were further supported by their finding that increases in pause duration, induced by decreases in the size of the gap in the Cs, also resulted in a larger proportion of saccades landing on potential targets. This suggested that increases in pause duration are helpful in improving the selection of landing locations during search, but pause durations are not prolonged solely for that reason. 

By contrast to preferences to avoid prolonging saccadic latency or fixation durations during search, studies of two-choice saccadic reaction time give a different picture of saccadic decisions. These studies show that saccades are delayed as information relevant to the choice of landing location is acquired (Beintema, van Loon, & van den Berg, 2005; Carpenter & Williams, 1995; Gold & Shadlen, 2007; Ludwig, 2009; Palmer et al., 2005). The results were accounted for by models in which visual information relevant to choosing the target location continued to accumulate until it reached a criterion level. In the event the available information is not adequate to find a suitable target, information accumulation models can be modified to take into account the passage of time, either by means of deadlines, or by incorporating time in whole or part into the decision variable (Churchland, Kiani, & Shadlen, 2008; Cisek, Puskas, & El-Murr, 2009; Hooge & Erkelens, 1996; Ludwig, Gilchrist, McSorley, & Baddeley, 2005; Ludwig, 2009; Nuthmann et al., 2010). 

The work reviewed above illustrates different views about the timing of saccades during search. Some of the prior work favored the view that timing strategies emphasize achieving high saccade rates at the cost of selecting the best landing position. Other work, using two-choice saccadic reaction time, supported accumulator models in which saccades are delayed in order to acquire information relevant to selecting the appropriate goal location. These differing views, as well as the need for a better understanding of saccadic timing strategies in visual tasks, led to the present study, which investigates saccadic timing during visual search for multiple targets. 

Outline of this study and preview

The present study examines the management of saccadic timing during visual search for multiple targets. Multi-target search was studied in order to elicit saccadic sequences that would continue throughout a trial, rather than terminating abruptly when a single target is found. 

For purposes of generality, two different tasks were used, each with identical displays. The statistical estimation task required estimation of the mean property of a small visual feature (an oriented line) located within each target. The look-only task had no explicit psychophysical component. Instead, subjects were asked to fixate as many targets per trial as possible. The discriminability of targets from distractors was varied using the method of Hooge and Erkelens (1999), namely, by varying the width of the lines used to draw the target and distractor stimuli. 

A major goal was to determine whether strategies of saccadic timing favor high saccade rates at the cost of accurate target selection, or whether saccades are delayed as part of an attempt to improve the accuracy of target selection. To address these goals, we examined saccadic timing as a function of the difficulty of distinguishing targets from distractors, and as a function of local factors, namely, the nature of both the fixated location and the saccadic goal (target or nontarget), and the ordinal position of the saccade in the sequence. The results showed that all of these factors affected saccadic timing, suggesting that saccades are not simply planned to scan displays as rapidly as possible, but rather that timing is strategically adjusted according to momentary task demands during search. 

Methods

Subjects

Five subjects were tested, all undergraduates at Rutgers University. All had normal vision without glasses or contact lenses, and were naive as to the experimental design and hypotheses. Experiments were approved by the Institutional Review Board for the Protection of Human Subjects of Rutgers University. 

Eye movement recording

Movements of the right eye were recorded by an Eyelink 1000 (SR Research, Osgoode, Canada) tracker (tower mount) with head held by a chin and forehead rest. Viewing was monocular with the left eye's view occluded by a patch. 

Stimulus display

The stimuli were displayed on Viewsonic G90fb 19-in. CRT monitor (Viewsonic, Walnut, CA) at a viewing distance of 118 cm. At this distance the resolution of the display was 0.73 minarc/pixel. Displays contained 25 black unfilled circles (circle diameter = 30 minarc) on a white background (luminance = 168 cd/m²), arranged as shown in Figure 1. Six circles were defined as target circles drawn with a line whose width was set to 4 pixels (∼2.9 minarc). The other 19 circles were defined as distractor circles with circle outline width set to one of three values (5, 6, or 7 pixels; equivalent to 3.7 minarc, 4.4 minarc, and 5.1 minarc, respectively). Thus, there were three levels of target/distractor similarity. Circle width will be referred to in pixels throughout the paper. 

Figure 1

 

A sample trial of statistical estimation task illustrating the stimulus displays and the procedure. Displays for the look-only task were identical except that there were no response or feedback frames.

The locations of the six target circles were chosen randomly in each trial. The closest distance between any two adjacent circles was 150 minarc along horizontal or oblique directions, and this distance defines the unit “separation” between adjacent circles. 

Targets contained a thin (1-pixel wide) line (length = 0.1°, luminance = 33 cd/m²) tilted about the vertical meridian. The angle of line tilt was sampled from a Gaussian distribution with mean set to one of four values (±30°; ±10°), and SD set to 20°. The angle would be resampled if it was not within the range ±90° in order to avoid confusion of tilt direction (e.g., tilt −100° is visually identical to tilt 80°). Distractors contained a square dot (2 pixels on a side). 

The width of the distractor circle (5, 6, or 7 pixels) and the stimulus display duration (1 or 2 s) remained constant for each 50 trial experimental session. The number of targets (six targets) and distractor circles (19 distractors), the distributions of the angles of line tilt, and the trial durations were all chosen on the basis of pilot experiments so that performance on the estimation task would steadily improve with the number of target circles used during visual search (as will be shown below). 

Procedure

Two tasks were tested: 

(a) Statistical estimation

A central fixation cross was displayed before each trial. Subjects fixated the cross and pressed a button to start the trial when ready. The stimulus display appeared 500 ms later for a duration of either 1 or 2 s. Subjects were instructed to view as many targets as possible to estimate the mean tilt of the population from which the samples of tilted lines were drawn. After the display of circles disappeared, a response frame was shown. The response frame contained four circles, each containing a line whose tilt was set to one of the four possible population means. Subjects chose one by pressing a button. Then, a display containing the correct answer was shown. The stimulus display and experimental procedure are shown in Figure 1. 

(b) Look-only

The display was the same except that all lines within the targets had the same tilt angle, equal to one of the four population means. No psychophysical reports were taken. Subjects were instructed to look at as many target circles as possible and to avoid looking at any distractor circles. 

Numbers of trials tested and excluded

Across both tasks, subjects were tested in a total of 30 to 45 experimental sessions, with sessions containing 50 trials. Some trials were discarded due to loss of tracker lock during the trial (loss of lock is usually due to blinks or momentary partial occlusion of the pupil by the eyelid). The proportions of discarded trials for the five subjects were <1%, 12%, 11%, 4%, and 6%. 

Preliminary psychophysical testing

The following preliminary psychophysical experiments were conducted using the same stimulus display in order to verify the choice of stimulus parameters: 

(a) The ability to discriminate the tilt of the line inside the target circle

The size and the contrast of the tilted line within targets were each chosen so that fixation at or very near the line would be required in order to discriminate the tilt. To confirm that fixation was required, a preliminary test was done. Subjects fixated a cross that was located at different eccentricities from a probe circle containing a tilted line. The eccentricity of the probe was randomly set to specified fractions of a separation (150 minarc) of two adjacent circles (0.2, 0.4, 0.6, 0.8, or 1). Line tilt was randomly sampled from the range of −90° to +90°, and the stimulus duration was 500 ms. Figure 2a shows that the accuracy of identifying the tilt (right/left) was over 80% correct only for eccentricities of less than half of the separation (75 minarc) of adjacent circles. When subjects fixated on the adjacent circle (eccentricity = 150 minarc), the accuracy was about chance level (50%). Similar results were found when the discriminability of the line tilts was easy (±45°). This shows that eccentricity should be no more than half the separation of adjacent circles to identify orientation accurately. 

Figure 2

 

Preliminary psychophysical results. (a) Proportion of correct discriminations of line tilt as a function of target eccentricity. (b) d′ for distinguishing targets from distractors as a function of target eccentricity in the discrimination task. One “separation” equals the distance between any two adjacent circles = 150′.

(b) The ability to distinguish target and distractor circles in eccentric vision

The widths of distractor circles were chosen to sample a range of difficulties of distinguishing eccentric targets from distractors. The range of difficulties resulting from the choice of distractor widths was confirmed in a preliminary psychophysical experiment. Before the trial, a fixation cross and a probe (a square dot, 10 pixels = 7.3 minarc on a side) were shown. The locations of fixation cross and probe were randomly selected from the locations of 25 circles so that the distance between the probe and the fixation cross was equal to one of three values: one, two, or three circle separations (where a separation = 150 minarc). Subjects were instructed to fixate the cross through entire trial. After a button press, 12 target circles and 13 distractor circles appeared for 1 s and subjects were asked to identify whether the circle at the probed location was a target (thin outline) or a distractor (thick outline). The eccentricity of the randomly selected probed location remained the same for a block of 50 trials. Figure 2b shows that discriminability increased with increasing distractor width and fell as eccentricity increased, but d′ values never dropped below one. Thus, targets and distractors were discriminable at a level above chance even when target eccentricity was large, and discriminability depended on distractor width at all eccentricities. 

Analyses

Saccade detection

The onsets and offsets of saccades were determined by computing eye velocity during successive 13-ms samples, with onsets separated by 1 ms. Saccade onset and offsets were detecting using a velocity criterion. The criterion (eye velocity during 13-ms interval of 12°/s) was confirmed for each subject based on an examination of analog records of eye position. Determination of saccade offsets were subjected to the additional constraint that velocity had to be below the criterion for 33 ms, which was long enough to bypass the overshoots typically accompanying saccades. 

Primary versus secondary saccades

All analyses of saccade characteristics will describe characteristics of the primary saccades. A primary saccade was the first saccade to leave from the currently fixated circle and land on another circle, with the location of the landing circle determined by a nearest neighbor criterion. A secondary saccade was any saccade following the initial primary saccade that landed on the same circle as the preceding primary saccade using the same nearest neighbor criterion. Figure 3 shows primary saccades and secondary saccades in a sample eye trace plotted as x-y position (left) and the same trace as eye position over time (right). 

Figure 3

 

Sample eye trace for a single trial shown as eye positions (left) and horizontal (top) and vertical (bottom) eye positions over time (right). Saccades are numbered in both graphs. The number on both graphs represents the order of saccades in the search sequence. The eye trace shows an example of primary and secondary saccades. The shaded area shows the dwell time between successive primary saccades.

Saccade rate

Saccade rate refers to the number of primary saccades per second. In all presentations of saccade rate, the infrequent instances of revisits to a location (<6% in 2 s, <1% in 1 s) were not included. Thus, saccade rate represents the number of unique circles fixated per second, regardless of whether the circle was a target or distractor. 

Dwell time (Figure 3b)

There are two measures that may be used to represent the time devoted to planning the primary saccades. One is the latency of the primary saccade, defined as the time between the offset of the saccade preceding a primary saccade (whether the preceding saccade is primary or secondary) and the onset of the primary saccade. The other measure is dwell time, defined as the time between two consecutive primary saccades, minus the flight time of any intervening secondary saccades. Using the latency of the primary saccade as the index of saccadic planning time assumes the planning of a primary saccade starts only after the prior saccade (primary or secondary) is completed. Wu et al. (2010), in a study of saccades made to look at a series of stationary targets, found that the latency of the primary saccade became significantly shortly when the prior saccade was a secondary saccade. This implied that initiating the planning of the primary saccade to a new target did not wait for the conclusion of any preceding secondary saccade (which typically corrected landing errors with respect to the current target), but rather could begin as soon as the prior primary saccade was concluded. For this reason, dwell time, the time between two consecutive primary saccades minus the flight time of any intervening secondary saccades, as noted above, is a more appropriate indicator of the time devoted to planning primary saccades than either the time between a primary saccade and either the preceding or the following secondary saccade. Thus, dwell time will be used as the main index of saccadic timing. 

Hit rate

The proportion of primary saccades landing at or near targets, was determined by a nearest neighbor criterion (see “Test of the nearest neighbor computation” below). A nearest neighbor criterion seemed appropriate given that the line of sight needed to be within at least a distance equal to half the separation of adjacent circles in order for the orientation of the target line to be determined with accuracy better than chance (Figure 2a). Hit rate/trial was calculated as the number of primary saccades landing nearer to a target than to any distractor divided by the total number of different circles fixated by primary saccades in each trial. 

Hit rate represents observed performance, not intentions (Viviani, 1990). We are not making the assumption that any given saccade was aimed specifically to either a target or a distractor circle (nontarget). Saccades could have been aimed to clusters of circles (e.g., Coëffé & O'Regan, 1987), or to regions within the display (e.g., upper-left quadrant), with the goal of using follow-up saccades to reach a target (Wu et al., 2010). Nevertheless, the variation of hit rate as a function of the discriminability of targets and distractors is a good indicator of successful performance of tasks in which accurate fixation of targets is required. The analyses will thus focus on determining how the measured hit rate varied with the parameters of the stimuli and tasks. 

Statistical analyses

The results presented (hit rates, saccade rates, and dwell times) show the performance that was first averaged over saccades in a given trial, then averaged over trials for each subject, and finally averaged over subjects. Saccadic performance was first averaged over saccades within a trial because the individual saccades within the sequence of a given trial may not be independent of each other. The results averaged across subjects were consistent with the results from individual subjects. 

A repeated measures ANOVA was used to test the statistical reliability of the results. In the ANOVAs, subjects were treated as a random effect and the other factors as fixed effects (Myers, 1966/1979). 

Test of the nearest neighbor computation

To test the accuracy of the nearest neighbor computation, given the inevitable noise in the eye tracker output and the saccadic system itself, a verification test was run in which 12 target circles were randomly displayed, with the remaining 13 locations left blank. Subjects were instructed to perform an easy task: Look at each target circle in a specified simple order, namely, one row at a time, scanning from left to right. They were also told to take enough time to look at each displayed circle accurately. Thus, under these conditions, performance should be nearly perfect—that is, each saccade should land at a circle in the specified sequence. Measured accuracy would be limited primarily by tracker noise or by variability within the saccadic system (Kowler & Blaser, 1995), not by the efficiency of selection, since no selection was required. Each saccade was classified using the nearest neighbor criterion above. The results showed that the vast majority of saccades (for the five subjects: 98%, 95%, 90%, 90%, and 95%) were classified as following the prescribed path. 

Motivation to find and fixate multiple targets

Motivation to find and fixate multiple targets in the look-only task was provided by instruction. Motivation in the statistical estimation task was provided by the understanding that performance would improve as a function of the number of targets fixated. Examination of the psychophysical performance in the statistical estimation task verified that the task was adequate to motivate the search. Performance reached asymptotic levels after fixation of three to four targets, with performance similar to that of an ideal observer limited only by the variability in the sampled orientation of the lines within the targets (see Supplemental Figure S1). Asymptotic performance was similar to that found in a control condition (50 trials) in which all six targets (no distractors) containing tilted lines were presented at the same time (1 s) to central vision (91 minarc × 63 minarc). This shows that memory loss for previously fixated lines was not a major contributor, and thus the present task, unlike many others (Ballard et al., 1995; Epelboim & Suppes, 2001; Kibbe & Kowler, 2011), would not benefit from frequent revisits of previously viewed targets. 

Results

The description of the results begins with hit rate and saccade rate as a function of the difficulty of distinguishing targets from distractors. The main objectives were (a) to determine whether saccades were made at a slower rate when targets were less discriminable from distractors, or whether saccades were made at a uniformly brisk rate regardless of discriminability, and (b) which measure, hit rate or saccade rate, was a better predictor of the proportion of saccades reaching targets. (Hit rate represents the proportion of saccades landing at or near targets, and is thus a measure of search selectivity, while saccade rate represents the number of circles visited per second, and is thus a measure of search speed. See Methods.) 

Following the presentation of overall performance as a function of target/distractor discriminability, performance will be re-examined after saccades are broken down according to local factors, namely: the type of location fixated (target or nontarget) both before and after the saccade, and the ordinal position of the saccade in the sequence. 

Hit rate and saccade rate each contributed to successful search

Success in the search task can be assessed by the number of targets fixated per trial. This measure depends on both the hit rate, the probability that a given primary saccade hits a target, and the saccade rate, the number of primary saccades made per unit time, regardless of whether they landed nearer a target or distractor. 

Hit rate

The average proportion of saccades hitting targets (see Methods for definition of hit rate) increased as the distractor width increased (p < 0.01; Table 1) for both the estimation task (Figure 4a) and the look-only task (Figure 5a). Hit rate did not differ between the two tasks, paired t test, t(44) = 0.32, p = 0.75. 

Figure 4

 

Statistical estimation task: saccadic performance as a function of the width of distractors. Primary saccades only, excluding the infrequent revisits. (a) Hit rate: the ratio of the number of target circles fixated divided by the number of total circles fixated. (b) Saccade rate: number of circles fixated/s. (c) Number of target circles fixated/s. Three different lines represent three temporal epochs (blue: 1-s trial; black: first second of the 2-s trial; red: last second of the 2-s trial). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols.

Figure 5

 

Look-only task: saccadic performance as a function of the width of distractors. Primary saccades only, excluding the infrequent revisits. (a) Hit rate: the ratio of the number of target circles fixated divided by the number of total circles fixated. (b) Saccade rate: number of circles fixated/s. (c) Number of target circles fixated/s. Three different lines represent three temporal epochs (blue: 1-s trial; black: first second of the 2-s trial; red: last second of the 2-s trial). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols.

Table 1

 

The repeated measure ANOVA for hit rate in the estimation and look-only tasks (Figures 4a and 5a). Notes: The reported statistics are based on the angular transformation, g(y) = 2arcsin√y – (π)/2, applied to the normalized smoothing measure (which ranges from 0 to 1) in order to improve the normality of its distribution (Hoaglin, Mosteller, & Tukey, 1991). EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Selectivity in estimation task					Selectivity in look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Circle width (W)	2	3.58	1.79	108.25	0.001*	2	3.23	1.62	64.78	<0.001*
Temporal epoch (T)	2	0.12	0.06	18.11	0.0184*	2	0.01	0.01	0.52	0.61
Subjects (S)	4	0.39	0.10			4	0.28	0.07
W × T	4	0.12	0.03	9.43	<0.001*	4	0.07	0.02	4.45	0.01*
W × S	8	0.13	0.02			8	0.20	0.02
T × S	8	0.03	0.00			8	0.10	0.01
Residuals	16	0.05	0.00			16	0.07	0.00
Total	44	4.42				44	3.96

Effects of trial duration (1 s vs. 2 s) on hit rate are also shown in Figures 4a and 5a, with results for the 2-s trials shown separately for the first and last halves of the trials. Differences in hit rates across the three temporal intervals were small and significant only in the estimation task (Table 1). 

Saccade rate

Saccade rates increased, but only slightly (±0.1–0.2 saccades/s), with increasing distractor width (Figures 4b and 5b). Effects of distractor width were significant in the look-only task (Table 2). Saccade rates were also lower during the final second of the 2-s trials than during the other temporal intervals (p < 0.05; Table 2). 

Table 2

 

The repeated measure ANOVA for saccade rate in both estimation and look-only tasks (Figures 4b and 5b). Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Scanning rates in estimation task					Scanning rates in look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Circle width (W)	2	0.14	0.07	3.34	0.09	2	0.68	0.34	10.21	0.0063*
Temporal epoch (T)	2	0.90	0.45	12.94	0.0031*	2	0.54	0.27	17.74	0.0011*
Subjects (S)	4	2.41	0.60			4	0.84	0.21
W × T	4	0.02	0.00	0.39	0.81	4	0.05	0.01	0.80	0.54
W × S	8	0.16	0.02			8	0.27	0.03
T × S	8	0.28	0.03			8	0.12	0.02
Residuals	16	0.18	0.01			16	0.23	0.01
Total	44	4.08				44	2.73

Saccade rates were significantly higher for the look-only task than the estimation task, paired t test, t(44) = 3.75, p < 0.05. Comparing Figures 4b and 5b shows that an effect of task on saccade rate was found only for the easiest target/distractor discriminability levels (widths 6 and 7), where saccade rates were highest. Rates were the same for the two tasks for the most difficult level of discriminability, where saccade rates were lowest. This suggests that the estimation task placed an upper limit on rates, but did not increase saccade rates in all conditions by a fixed amount, which implies that the estimation task and the search for targets could proceed concurrently. 

Rate of fixating targets

The number of targets hit per second, a measure of search success, increased by about a factor of about 1.4 across the three targets' widths (Figures 4c and 5c; Table 3). This is about the same as the proportional increase in hit rate across the distractor widths. Variations in saccade rate with distractor width (Figures 4b and 5b) played little role in increasing the rate of successfully fixating targets (Figures 4c and 5c). Higher saccade rate, however, was the main reason for the greater number of targets hit per second in the look only task (peak 2.4–2.5/s with the widest distractors, Figure 5c) compared to the estimation task (peak ∼2.3 targets/s, Figure 4c). 

Table 3

 

The repeated measure ANOVA for number of targets hit/second in estimation and look-only tasks (Figures 4c and 5c). Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Estimation task					Look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Circle width (W)	2	7.84	3.92	97.78	<0.01*	2	8.78	4.39	39.90	0.0001*
Temporal epoch (T)	2	0.04	0.02	1.25	0.34	2	0.29	0.15	4.62	0.0463*
Subjects (S)	4	0.94	0.23			4	0.51	0.13
W × T	4	0.11	0.03	4.92	0.0089*	4	0.08	0.02	1.23	0.34
W × S	8	0.32	0.04			8	0.88	0.11
T × S	8	0.13	0.02			8	0.25	0.03
Residuals	16	0.09	0.01			16	0.26	0.02
Total	44	9.48				44	11.05

In summary, the small modulation in saccade rates across distractor width (Figures 4b and 5b) suggests that there was little attempt to slow the pace of scanning in order to improve the ability to find and fixate targets (similar to Hooge & Erkelens, 1999). Instead, scanning seems to have proceeded at a uniformly high rate regardless of distractor width. The next set of analyses, however, presents a somewhat different picture. In these analyses, saccades are subdivided according to local conditions, namely, the nature of the fixated location (target or nontarget), and the nature of the goal location. 

Dwell times depended on the nature of the location fixated, the nature of the saccadic goal, and target/distractor similarity

For the remaining analyses, the timing patterns of saccades will be represented by the dwell time. Dwell time is defined as the total pause time between two consecutive primary saccades minus the in-flight time of any secondary saccades (see Methods and Figure 3) and provides an index of the time allotted to planning a given saccade (Wu et al., 2010). 

Dwell time is shown in Figure 6 as a function of the type of location, target, or nontarget that was closest to the line of sight prior to the saccade, with separate functions for saccades that then hit targets and for saccades that missed targets. Only the longer trials (2 s) were included in order to obtain sufficient numbers of samples/trial in each of the four groups (i.e., target→target; target→nontarget; nontarget→target; nontarget→nontarget). In addition, the first saccade of the sequence was not included because it is a special case in which the saccade originated from the initial fixation point. (First saccades will be considered in the next subsection.) 

Figure 6

 

Dwell time as a function of saccade starting location, target or distractor, and landing position (HIT/MISS) in the estimation task (top row) and in the look-only task (bottom row). Data were only for longer trial duration (2 s). Blue lines represent the dwells of saccades that hit a target, and red lines represent the dwells of saccades that missed. The superimposed green lines represent the probability of hitting a target as a function of saccade starting location (green axis on the right hand side). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols. The first saccade in the sequence was not included because the first saccade left from initial fixation.

Figure 6 shows large effects of the type of location fixated prior to the saccade. Dwell times were about 100 ms longer when fixating a target than a nontarget (p < 0.05; Table 4). The effect of the currently fixated location was about the same in both the statistical estimation and the look-only tasks; thus, the longer dwell times on targets was not due to the requirements of the estimation task, but rather to the search process. Figure 6 also shows a small increase in hit rates following saccades launched from targets than from nontargets. 

Table 4

 

The repeated measure ANOVA for saccadic dwell time as a function of the starting location (target or nontarget) and landing location (target or nontarget). Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Dwell times in estimation task					Dwell times in look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Saccade starting location (A)	1	119,873.80	119,873.80	46.90	0.0024*	1	113,981.80	113,981.80	152.39	0.0002*
Saccade landing location (B)	1	886.30	886.30	6.74	0.06	1	1,582.40	1,582.40	16.95	0.0147*
Width (W)	2	26,195.80	13,097.90	21.68	0.0006*	2	32,475.90	16,237.95	21.15	<0.001*
Subjects (S)	4	43,201.10	10,800.28			4	9,024.60	2,256.15
A × B	1	3,199.90	3,199.90	22.16	<0.001*	1	272.50	272.50	2.24	0.15
A × W	2	2,102.50	1,051.25	7.28	0.0026*	2	264.00	132.00	1.08	0.35
A × S	4	10,224.30	2,556.08			4	2,991.90	747.98
B × W	2	567.90	283.95	1.97	0.16	2	5.30	2.65	0.02	0.98
B × S	4	526.10	131.53			4	373.50	93.38
W × S	8	4,833.70	604.21			8	6,141.10	767.64
Residuals	30	4,332.00	144.40			30	3,652.60	121.75
Total	59	215,943.40				59	170,765.60

Dwell times were also affected by whether the impending saccade hit or missed the target. Although it might be expected that longer dwell times should precede hits (i.e., greater accuracy at the cost of latency), the results showed the opposite trend: Average dwell times prior to hits were slightly shorter than dwell times prior to misses. This was particularly true for the small group of saccades that originated from nontargets with the wider distractors, when dwell times were short. The effects are small, but were consistent across distractor widths, tasks, and subjects (see ANOVA in Table 4 and individual subject results in Supplemental Figure S2). The difference between dwell times prior to hits and misses suggests that when a target is not readily available, saccades may be delayed, perhaps in an attempt to find a target during the current fixation, or to formulate an alternative search strategy. 

Figure 6 also shows that once saccades were sorted according to the nature of the currently fixated location, the effect of the width of the distractor is larger than was seen in the overall data (Figures 4b and 5b). Dwell times decreased by about 30–40 ms as the distractor width increased across the range tested (Figure 6; p < 0.01; Table 4). Most of these effects of distractor width were due to the differences between performance with the most difficult target/distractor similarity levels (distractor width 5) and the other two widths tested. The effect of distractor width on dwell time was larger here, in Figure 6, than when overall performance was examined (Figures 4 and 5) because, for overall performance, the decrease in dwell time for the widest distractors was offset by the greater number of saccades launched from targets, which (as Figure 6 shows) had longer dwells. 

In summary, once saccades are subdivided according to both the type of location fixated and the type of location at the saccadic goal, saccadic timing can be seen to depend on three factors: the type of fixated location (target or nontarget), the difficulty of distinguishing targets from distractors, and whether the saccade hit or missed the target. The difference between dwell times on targets and nontargets was the same for both the estimation and look-only tasks, showing that the effects were due to the search process, and not to the evaluation of the tilt of the line inside the targets. The effect of target/distractor similarity, and the difference between hits and misses, implies that saccades are not simply made at a uniformly brisk rate, but rather that saccades may be delayed when a target is not readily available. 

Dwell time depended on the ordinal position of the saccade in the sequence

Saccadic timing was also examined as a function of the ordinal position of the saccade in the sequence. 

Ordinal position had large effects on dwell times (Figure 7; Table 5). The first saccades in the sequences had short latency (∼200 ms), and a poor hit rate. Only 30%–40% of first saccades hit targets, even with the widest distractors. Hit rate jumped to its peak value (>80% for the widest distractors) for the second saccade. These second saccades, however, were preceded by brief dwell times (<200 ms for the widest distractors). The brief dwell times between the first and second saccades (Figure 7) are consistent with prior results showing that second saccades benefit from planning that starts before the first saccade occurs (Araujo et al., 2001; Caspi, Beutter, & Eckstein, 2004; McPeek, Skavenski, & Nakayama, 2000). 

Figure 7

 

Dwell time as a function of the ordinal position of primary saccades for three levels of selection difficulty in the estimation task (top) and the look-only task (bottom). Blue lines represent saccades going to a target (HIT) and red lines represent saccades going to a distractor (MISS). The superimposed individual circles represent the probability of hitting a target (axis on the right-hand side). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols.

Table 5

 

The repeated measure ANOVA for effects of ordinal position on dwell time. Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Dwells in estimation task					Dwells in look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Ordinal position (P)	5	102,148	20,429.50	15.98	<0.001*	5	94,856.90	18,971.38	19.71	<0.001*
Accuracy: hit or miss (A)	1	1,466.9	1,466.90	3.39	0.14	1	1.60	1.60	0.00	0.95
Width (W)	2	7,891.3	3,945.65	5.74	0.0285*	2	20,397.70	10,198.85	8.93	0.0092*
Subjects (s)	4	116,788	29,196.95			4	17,491.20	4,372.80
P × A	5	8,945.8	1,789.16	8.42	<0.001*	5	7,415.40	1,483.08	4.21	0.0015*
P × W	10	9,271.3	927.13	4.36	<0.001*	10	10,766.00	1,076.60	3.05	0.0018*
P × S	20	25,562.7	1,278.14			20	19,250.20	962.51
A × W	2	725.6	362.80	1.71	0.19	2	672.40	336.20	0.95	0.39
A × S	4	1,732.3	433.08			4	1,306.40	326.60
W × S	8	5,501.7	687.71			8	9,141.80	1,142.73
Residuals	118	25,079	212.53			118	41,604.50	352.58
Total	179	305,112				179	222,904.10

Figure 7 also shows a cost in time of hitting targets early in the sequence. First and second saccades that hit targets had longer latencies than those that missed targets (Table 5). 

Dwell time increased (to about 250 ms) for the third-to-the-last saccades in the sequences. Hit rates steadily decreased across these saccades, perhaps due to the steady reduction in the number of available targets. 

Figure 7 also shows no consistent difference in dwell times prior to hits and misses, beginning with the third saccade in the sequence. However, breaking down the saccades according to the nature of the currently fixated location confirms the same pattern shown for overall performance in Figure 6, namely, slightly shorter dwell times prior to hits than misses, particularly for infrequent saccades that left from nontarget locations with the wider distractors (Supplementary Figures S3 and S4). 

A strong predictor of whether a target was hit was, not surprisingly, its eccentricity prior to the saccade. Figure 8 shows that the eccentricities of the nearest missed targets were consistently larger than the eccentricities of hit targets across ordinal positions (p < 0.05; Table 6). Figure 8 also shows that the average eccentricities of the hit or missed targets increased with ordinal position, suggesting a preference to fixate nearby, easily discovered targets first. 

Figure 8

 

Eccentricity of the nearest target circle hit and the nearest target circle missed as a function of the ordinal position of primary saccades for three levels of selection difficulty in the estimation task (top) and the look-only task (bottom). Blue lines represent saccades going to a target (HIT) and red lines represent saccades going to a distractor (MISS). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols.

Table 6

 

The repeated measure ANOVA for effects of ordinal position on target eccentricity. Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	Eccentricity in estimation task					Eccentricity in look-only task
	df	SS	EMS	F	p	df	SS	EMS	F	p
Ordinal position (P)	5	505,663.5	101,132.70	111.88	<0.001*	5	613,707.2	122,741.44	133.42	<0.001*
Accuracy: hit or miss (A)	1	54,288.5	54,288.50	15.52	0.017*	1	138,783.4	138,783.40	61.88	<0.001*
Width (W)	2	52,307	26,153.50	30.95	<0.001*	2	36,839.2	18,419.60	9.46	0.0078*
Subjects (s)	4	4,542.8	1,135.70			4	12,131.6	3,032.90
P × A	5	9,925.3	1,985.06	4.46	<0.001*	5	27,586.1	5,517.22	7.42	<0.001*
P × W	10	17,386.7	1,738.67	3.91	<0.001*	10	13,394.1	1,339.41	1.80	0.07
P × S	20	18,078.3	903.92			20	18,399	919.95
A × W	2	2,780.3	1,390.15	3.12	0.048*	2	4,731.1	2,365.55	3.18	0.045*
A × S	4	13,994.8	3,498.70			4	8,971.8	2,242.95
W × S	8	6,760.6	845.08			8	15,575.5	1,946.94
Residuals	118	52,538.3	445.24			118	87,737.6	743.54
Total	179	738,266.1				179	977,856.6

In summary, the analysis of dwell times as a function of ordinal position reveals that timing strategies changed over the course of a trial. The first saccades had short latencies and low hit rates. The second saccades also had short latencies, but a much higher hit rate, which may be because the analysis of the information started prior to the first saccade. There was also evidence that increasing dwell time was associated with an improved hit rate of the first two saccades, specifically, dwell times of the first two saccades were longer prior to saccades that hit targets than saccades that missed targets, but this pattern was not found for later saccades. The probability of a saccade hitting a target also depended on spatial factors, specifically, the eccentricity of the nearest target. 

Lowering contrast results in longer dwell times, but not in better hit rates

Hooge and Erkelens (1999) found that when dwell time was prolonged by making the foveal discrimination task more difficult (i.e., smaller gaps in their Landolt C's), the probability of hitting targets increased also, presumably as a byproduct of having more time. We conducted an additional experiment to determine whether the same benefit of eliciting longer dwell times held for our task when we used an experimental manipulation, namely, lower stimulus contrast, that would produce longer dwell times. 

Six subjects were run in the same statistical estimation task with the 2-s trial duration; three were subjects tested in the first experiment. The only change to the experiment was that the contrast of visual element inside each circle (either the single dot, for distractors, or the tilted line, for targets) was set to one of three different luminance levels: 101, 51, or 19 cd/m². The background luminance was the same as the first experiment (white background, luminance = 168 cd/m²). The three conditions will be referred to as low, medium, and high contrast. 

Figure 9a shows that, as expected, dwell time increased as contrast decreased (p < 0.05; Table 7). However, contrary to the findings of Hooge and Erkelens (1999) from a different experimental paradigm, the increase in dwell time was not beneficial to the hit rate, which remained the same across the contrast levels (Figure 9b; Table 8). 

Figure 9

 

Saccadic performance with variable contrast levels. Left: dwell times as a function of distractor width for three levels of stimulus contrast (low, medium, and high contrast). Right: selectivity as a function of distractor width for the three levels of stimulus contrast. Bars show ± SE; otherwise, SEs are smaller than the plotting symbols.

Table 7

 

The repeated measure ANOVA for dwell time at different contrast levels. Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	df	SS	EMS	F	p
Circle width (W)	2	2,111.7	1,055.85	3.50	0.07
Contrast (C)	2	3,493.4	1,746.7	5.66	0.02*
Subjects (S)	5	14,740.3	2,948.06
W × C	4	357.2	89.3	3.46	0.03*
W × S	10	3,017.5	301.75
C × S	10	3,087.4	308.74
Residuals	20	516.4	25.82
Total	53	27,323.9

Table 8

 

Repeated measures ANOVA for hit rates at different contrast levels. Notes: The reported statistics are based on the angular transformation, g(y) = 2arcsin√y – (π)/2, applied to the normalized smoothing measure (which ranges from 0 to 1) in order to improve the normality of its distribution (Hoaglin, Mosteller, & Tukey, 1991). EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	df	SS	EMS	F	P
Circle width (W)	2	3.16	1.58	71.91	<0.001*
Contrast (C)	2	0.00	0.00	0.39	0.69
Subjects (S)	5	1.74	0.35
W × C	4	0.01	0.00	0.98	0.44
W × S	10	0.22	0.02
C × S	10	0.04	0.00
Residuals	20	0.04	0.00
Total	53	5.21

It is possible that any benefits of increasing dwell time on hit rates were not realized because of interference from the visual demands associated with using low contrast stimuli. It is also possible that benefits are found only when the dwell times are varied within trials, as was the case in Hooge and Erkelens (1999). Note that we also found that longer dwell times were associated with slightly higher hit rates within trials (Figure 6, and also Figure 10; see below). Differences in outcomes as a result of whether manipulations are made between or within trials point to a role for task strategies on performance. 

Figure 10

 

Dwell time as a function of saccade starting locations (target or distractor) and landing positions (HIT/MISS) in the estimation task of Experiment 2. Blue lines represent the dwells of saccades that hit a target, and red lines represent the dwells of saccades that missed. The superimposed green lines represent the probability of hitting a target as a function of saccade starting locations (green axis on the right-hand side). Bars show ± SE; otherwise, SEs are smaller than the plotting symbols. The first saccade in the sequence was not included because the first saccade left from initial fixation. Each row represents one level of stimulus contrast (low, medium, or high contrast).

This new set of data also provided an opportunity to replicate the effects of the type of location fixated before and after the saccade that were shown for the first experiment in Figure 6. Figure 10 shows the same pattern of results, namely saccades leaving from targets had longer dwells than saccades leaving from nontargets (p < 0.01; Table 9). In addition, and particularly for the widest distractors and for saccades leaving from nontargets, where dwell times were the shortest, dwell times prior to hits were shorter than dwell times prior to misses (Figure 10). Differences between hits and misses were small, but reliable (landing location, p = 0.01; interaction between starting and landing location, p < 0.01; Table 9). Effects of circle width were also once again significant (p < 0.01; Table 9). 

Table 9

 

The repeated measure ANOVA for saccadic dwell time as a function of the starting location (target or nontarget) and landing location (target or nontarget) at different contrast levels. Notes: EMS = expected mean square. SS = sum of squares. Asterisks indicate significance with the p values listed in the table.

Source of variance	df	SS	EMS	F	p
Starting location (A)	1	493,916.70	493,916.70	45.12	0.001*
Landing location (B)	1	7,510.00	7,510.00	16.31	0.01*
Circle width (W)	2	69,018.70	34,509.35	18.86	<0.001*
Contrast (C)	2	11,586.10	5,793.05	5.20	0.03*
Subjects (S)	5	65,681.00	13,136.20
A* × B	1	22,449.60	22,449.60	71.99	<0.001*
A × W	2	7,776.10	3,888.05	12.47	<0.001*
A × C	2	4,104.70	2,052.35	6.58	0.002*
A × S	5	54,728.70	10,945.74
B × W	2	1,040.30	520.15	1.67	0.19
B × C	2	723.20	361.60	1.16	0.32
B × S	5	2,302.10	460.42
W × C	4	1,141.50	285.38	0.92	0.46
W × S	10	18,293.10	1,829.31
C × S	10	11,137.10	1,113.71
Residuals	161	50,205.50	311.84
Total	215	821,614.40

Saccadic strategies and the limits imposed by visual acuity

One way to evaluate the adequacy of a saccadic timing strategy is to ask whether the ability to hit targets is as good as would be expected on the basis of visual acuity, i.e., the ability to distinguish eccentric targets from distractors. To address this question, we compared subjects' hit rates to that of a simple model that was limited only by the ability to distinguish targets from distractors at different eccentricities. The model's ability to distinguish targets from distractors at different eccentricities was obtained from the preliminary psychophysical testing (Figure 2b). The model was also given three other properties: (a) it aims its saccades to the closest target detected; (b) it does not revisit previously fixated targets; and (c) it does not remember information across fixations, other than the locations of previously fixated targets (in order to avoid revisits). Details of the model are presented in Appendix. 

Note that the preliminary psychophysical testing was done for the purpose of verifying that targets and distractors could be distinguished at the line widths tested. The conditions operating during the preliminary testing were not the same as those during search. Specifically, in the preliminary testing there was no need to search: Only one location had to be judged, and there was no uncertainty about which location to judge since a location cue was given. In addition, in the preliminary testing there was no evaluation of the details at the currently-fixated location (as there was in the estimation task) and the available time for the judgments was quite long (1 s). Thus, it would seem highly unlikely that performance (hit rates) during search would reach the levels predicted solely on the basis of acuity given the reduced conditions used to obtain the preliminary data. Despite these caveats, a comparison of performance with a model based solely on acuity is useful because underperformance of subjects relative to the model can be attributed to aspects of the search strategy, in particular, a preference for using saccades to explore, rather than a preference to delay saccades in the attempt to find a target using eccentric vision while maintaining fixation. 

The hit rates of the model and subjects are compared in Figure 11 as a function of the number of targets remaining in the sequence. While subjects did not perform as well as the model, the differences for the two wider distractors were quite small. The only cases in which the model outperformed the subject by a wide margin were for the initial saccades (where, as noted above, subjects performed poorly) and for the hardest target/distractor discriminability (distractor width 5). The poor performance for the most difficult target/distractor discriminability level reflects the preference to explore when a target is not readily available. Reducing the eccentricity of possible targets by exploring with new saccades, rather than waiting in the attempt to get more information, was the preferred strategy. 

Figure 11

 

Hit rate as a function of the number of remaining (not-yet-fixated) targets for three levels of selection difficulty. Blue lines represent observed hit rates in the estimation task, and black lines represent observed hit rates in the look-only task. The red lines are the predicted hit rate from acuity model described in the text and the Appendix.

Discussion

One strategy for making saccades during visual search for multiple targets is simply to make saccades at the highest possible rate, in an attempt to fixate as many locations as possible, with the rate of saccades limited only by the capacity of the saccadic system to produce saccades, and by the time required to identify the fixated details. 

Our initial analyses appeared to support preferences for making saccades at the same high rates, regardless of the difficulty of finding targets. Mean overall saccade rates were about 3/s and varied only slightly with the discriminability of targets from distractors (Figures 4 and 5). These results revealed little strategic adjustment of saccadic timing during search. 

A different picture emerged when saccades were sorted according to local factors, specifically: where the eye was looking just before the saccade (closer to a target or to a nontarget), whether the saccade landed closer to a target (hit) or to nontarget (miss), and the ordinal position of the saccade in the sequence. All of these factors, as well as the discriminability of targets from distractors, affected the timing of saccades. These influences show that saccadic timing is strategically, often subtly, adjusted as search proceeds. We next consider the implications of these adjustments for understanding strategies of saccadic timing during search. 

The role of dwell time in finding or identifying targets

When saccades were subdivided according to whether the currently fixated location was closer to a target or to a nontarget, dwell times were found to vary across the three levels of target/distractor discriminability (Figures 6 and 10). These effects were due primarily to the long dwell times found with the most difficult level of discriminability. 

The effects of target/distractor discriminability can be compared to those reported in prior studies of visual search. Gould and Dill (1969), for example, studied search for multiple targets and found that the duration of fixation pauses increased as the similarity of targets and distractors increased. Hooge and Erkelens (1999) also found effects of discriminability on fixation pause duration, but only in one case, namely, with distractors that were identical to the targets in all ways except for a small feature (the gap in the C's). The gap could be detected reliably only when the C was fixated. Both Hooge and Erkelens (1999) and Gould and Dill (1969) attributed the effects of target/distractor similarity on dwell time to the longer time needed to decide whether a given fixated element was a target or a distractor, rather than to processes connected to the search, i.e., to the selection of saccadic goal locations. 

The time needed to distinguish fixated targets from distractors might also account for the effects of discriminability on dwell time that we observed. This would be expected because for the most difficult level of discriminability, where dwell times were longest, a highly accurate classification (target vs. distractor) could be made only when fixating close to the element (see Figure 2b). 

Did we find any evidence that dwell time was increased to improve the ability to find eccentric targets? One of our observations is consistent with such a strategy. We found that dwell times preceding saccades that hit targets were shorter than dwell times preceding misses. Differences between dwell times preceding hits and misses, although small, were found across tasks, subjects, and levels of target/distractor discriminability (Figures 6 and 10 and Supplementary Figures S2, S3, and S4). 

The difference in dwell times preceding hits and misses may represent attempts to accumulate more information in order to find a nearby target. This interpretation is consistent with the finding that the differences between dwell times prior to hits and misses were most evident when dwell times were short (∼150 ms; Figures 6 and 10). If the benefits to search of adding more time are subjected to diminishing returns, then benefits should be greater with shorter dwell times. With longer times (>250 ms; Figures 6 and 10), little benefit of small additional delays would be expected. 

At some point, however, the decision must be made to stop acquiring information and initiate the saccade. Prior studies of saccadic reaction time have considered how saccades might be triggered in the event that the accumulated visual evidence does not support a clear decision (for example, when a target is not discovered). Several studies suggested that saccades may be launched on the basis of elapsed time, either in place of, or in conjunction with, the available evidence (Churchland et al., 2008; Ludwig, 2009). A prominent role for elapsed time in determining when to initiate saccades has also been considered in analyses of saccadic timing during search, reading, or scene viewing (Hooge et al., 2007; Yang & McConkie, 2001; Nuthmann et al., 2010; Henderson & Smith, 2009). In visual search tasks, decisions to initiate saccades on the basis of elapsed time before a target has been identified are effectively decisions to explore the display to gather new information, rather than wait to acquire or analyze more information from the currently fixated location. 

A bias to explore, rather than to wait, is also consistent with our analysis of target hit rates in comparison to what would be predicted on the basis of visual acuity (Figure 11). Subjects did about as well as the model with the easiest levels of target/distractor discriminability, but underperformed with the most difficult level of discriminability. This suggests that with the most difficult level of discriminability, the saccadic decisions favored exploration, rather than making full use of the available visual information in time-consuming attempts to find a target. A bias to explore makes sense when the expected hit rates are low, as was the case with the most difficult discriminations. Exploration, which has the benefit of quickly reducing the eccentricity of potential targets, is surely a more effective use of time. 

Targeting versus exploratory phases of search

Strategies of saccadic timing also took into account the nature of the fixated location. Dwell times while fixating near targets were about 100 ms longer than dwell times while fixating elsewhere. These are large effects. The longer dwells on targets were not the result of the perceptual requirements of the statistical estimation task since the pattern of results was the same in the look-only task. The longer dwells when fixating targets were also not likely to be the result of the process that attempted to distinguish a fixated target from a distractor because longer dwells on targets were found even for the easiest target/distractor discriminability levels. Thus, the longer dwell times on targets were connected to some aspect of the search process. 

Search for multiple targets can be viewed as consisting of a series of exploratory episodes, each ending with a saccade to a target. The terminal fixations on targets that ended each episode might have been prolonged for several reasons, including confirming that the location was in fact a target, or making new plans about initiating the next exploration. 

The difference between dwell times on targets and nontargets is very similar to results of Epelboim et al. (1995) in a multi-target search task. Their task required either looking at or tapping a set of randomly distributed rods in a specified sequence that was dictated by the colors of the rods. Epelboim et al. (1995) found that before the locations were learned, during what they called “search episodes,” dwell times were 100–150 ms shorter than the dwell times after the locations were learned (referred to as “sequence” episodes). This large difference in dwell times between their search and sequence episodes, just as our differences between dwell times on targets and nontargets, suggests that different operations are needed when exploring a display, as opposed to when fixating the critical targets, even when the task is as simple as looking at a set of targets in sequence. 

The finding of large differences between dwell times on targets and nontargets suggests that dwell times may be used to infer strategies of segmenting performance of complex tasks into exploratory and targeting phases. This may be particularly valuable when the observers' goals or strategies are not known. 

Ordinal position

Dwell times varied with ordinal position. The pattern of dwell times as a function of ordinal position was similar to that reported for other visual tasks requiring sequences of saccades (Irwin & Zelinsky, 2002; Over, Hooge, Vlaskamp, & Erkelens, 2007). One of the most striking effects of ordinal position was the distinction between the first two saccades and the remaining saccades in the sequence (Figure 7). 

Dwell times preceding the first and second saccades were shorter than the remaining dwell times in the sequence. The hit rate of the first saccade was poor, but hit rate improved for the second saccade. The poor hit rate of the first saccade, coupled with the brief dwell times between the first and second saccades, suggests that the planning of the second saccade often began before the first was completed (Araujo et al., 2001; Caspi et al., 2004; McPeek et al., 2000; Phillips & Segraves, 2010; Wu et al., 2010; Zingale & Kowler, 1987). Thus, the high hit rates of the second saccade took time, and likely benefited from information acquired before the first saccade. 

There was other evidence that finding targets early in the sequence was costly in time. For the two easier levels of discriminability of targets and distractors, dwell times preceding hits were longer than preceding misses (Figure 7). Such costs were not observed later in the sequence. 

Why did it cost time to find targets early in the sequence? The costs may represent time needed for visual analyses, or for planning, following the initial appearance of the display. Saccades later in the sequence may take advantage of the information obtained or the plans made at or near display onset. 

The effects of ordinal position point to the risks of drawing general conclusions about search from observations restricted to very brief presentations, or to the first saccade or the first pair of saccades (Araujo et al., 2001; Caspi et al., 2004; Morvan & Maloney, 2012). Search strategies, particularly in real-world tasks where there is typically opportunity to learn display contents, may rely on memory for recently viewed material (Ballard et al., 1995; Epelboim & Suppes, 2001; Kibbe & Kowler, 2011; Melcher, 2001; Melcher & Kowler, 2001). 

Summary and conclusions

The time available to accomplish any task is limited, thus the control of saccadic timing, at the level of individual fixation pauses or at the level of the planning of entire sequences of saccades, can provide insights about strategies of resource management, as well as the fine-grain control of saccadic decisions. 

We studied saccadic timing during visual search for multiple targets and found several distinct influences on the dwell times between successive saccades. Specifically, dwell times prior to target hits were shorter than dwell times prior to misses; dwells on targets were considerably longer than on nontarget locations; dwells early in the sequences were shorter than dwells later; and dwell times increased when targets were very similar to distractors. 

All of these results indicate strategic adjustments of saccadic timing during search. Saccades are not made at the highest possible rates in an attempt to fixate as many locations as possible in the time available, nor are saccades delayed until a target is discovered. Instead, the results suggest that timing patterns are continually adjusted on the basis of incoming information and internal models that determine when to delay and when to explore. Given that the delays of saccades when a target was not readily available were relatively brief, it is likely that timing decisions take into account the diminishing returns of waiting too long relative to the benefits of exploration in search of targets. The very brief dwell times found during periods of exploration, when landing at or near nontargets, contribute to the benefits of exploration since the cost in time of fixating nontarget locations may be minimal, outweighed by the benefits of reducing the eccentricity of potential targets in the display. Finally, the effects of ordinal position on both timing and hit rate, in particular the greater cost in time associated with finding targets early (the first two saccades), and the poor hit rate of the first saccade, suggests either a role for memory or that search strategies take time to develop. 

Investigations of saccadic behavior often emphasize the decisions underlying the selection of future locations to be examined. These strategies are widely considered to be rational in that they take into account both the requirements of the task and the limitations of visual and cognitive processing. The present study of saccadic timing during search for multiple targets suggests that the control of saccadic timing, including subtle adjustments of dwell times, is similarly rational, taking into account the benefits of waiting versus exploring in an attempt to maintain accurate and efficient visual search. 

Supplementary Materials

docx S1 

Acknowledgments

We thank Jacob Feldman, Manish Singh, Michele Rucci, Melchi Michel, John Wilder, Cordelia Aitkin, Min Zhao, Elio Santos, and Nicholas Ross for valuable suggestions. This research was supported in part by NIH EY15522. 

Commercial relationships: none. 

Corresponding author: Chia-Chien Wu. 

Email: [e-mail]. 

Address: Department of Computer Science, University of Massachusetts at Boston, Boston, MA, USA. 

Appendix

The preliminary psychophysical experiment (Figure 2b) showed that the ability to distinguish targets from distractors depended on both circle eccentricity and distractor circle width. The preliminary results (Figure 2b) were used in a simulation to convert the physical stimulus display of targets and distractors into a stochastic visual array based on the eccentricity of each circle relative to current fixation. 

For each simulated trial n, a display of 25 circles (six targets and 19 distractors) was generated. The probability of each target circle T at eccentricity i being categorized correctly as a target (hit), and the probability of each distractor circle D at eccentricity i being incorrectly categorized as a target (false alarm), prior to each saccade, was determined from the preliminary psychophysical results. For cases in which the circle eccentricity fell between the tested values (Figure 2b), linear interpolation was used. 

One difference between the conditions of the preliminary psychophysical data and the conditions of the actual experiment is the prior probability of finding a target, P(T). In the preliminary psychophysical data, P(T) = 0.5. In the actual experiment, P(T) initially was 6/25 = 0.24 and decreased with each target that was discovered. To account for effects of the change in the proportion of targets on the criterion, the P(hit) and P(false alarm) used in the simulation were determined from the values of d′ (Figure 2b) and from a decision criterion, c, set to be equal to d′/ln (β), where β is the ratio of the proportion of distractors remaining (not yet fixated) to the proportion of targets remaining (not yet fixated) in the display (Macmillan & Creelman, 2005, equation 2.6). 

The values of P(hit) and P(false alarm) were then used to label each circle as a target or a distractor by drawing a sample from a binomial distribution with parameter either P(hit), for targets, or P(false alarm), for distractors. The first simulated saccade was then directed to the nearest location that was categorized as a target. This process was repeated for a sequence of six saccades, using the same display generated for trial n, but at the new eccentricities following each of the simulated saccades in the sequence and the new computed criterion, c, that took into account the change in the proportion of available (not-yet-fixated) targets and distractors. Note that the model was given perfect memory for locations fixated (but not for other locations) and was not allowed to revisit. This process was repeated for 10,000 trials for each distractor width.