Saccadic momentum and attentive control in V4 neurons during visual search

Abstract

Saccadic momentum refers to the increased probability of making a saccade in a forward direction relative to the previous saccade. During visual search and free viewing conditions saccadic probability falls in a gradient from forward to backward directions. It has been considered to reflect an oculomotor bias for a continuing motor plan. Here we report that a saccadic momentum gradient is observed in nonhuman primate behavior and in the visual responses of cortical area V4 neurons during a conjunction style visual search task. This result suggests that saccadic momentum arises in part from a biased spatial distribution of visual responses to stimuli. The effect is independent of feature-based selective attention and overridden by directed spatial attention. The implications of saccadic momentum for search guidance are much broader and robust than the inhibition-of-return's presumed role in preventing refixation of recent locations.

Introduction

Do saccadic momentum or inhibition of return (IOR) phenomena result from changes in the neural sensory representation of the visual scene? The present study investigates this issue and differentiates effects associated with saccade momentum and IOR conditions from known attentive effects in visual cortical area V4. 

The guidance of search is modulated by both task strategy and attention to stimulus properties, and ultimately depends on the current sensory representation of the surrounding space. Models of visual search generally proceed by identifying relevant, salient items or their likely locations within the scene during fixations. Search models often incorporate a measure that reduces the probability of returning to previous fixations either by using a memory strategy or by reducing the salience of items/locations that have been recently inspected (Itti & Koch, 2001; Zelinsky, 2008). The IOR phenomenon (Klein, 2000) is often employed as the agent that reduces the probability of returning to previous fixation locations. Initially described as an increase in reaction time to targets placed at previously attended locations (Posner & Cohen, 1984), the oculomotor inhibition of return (O-IOR) refers to the observation that returning to a recently fixated location is less likely and takes longer than to other locations in the scene (Klein, 2000). Klein and MacInnes (1999) proposed that the IOR could be viewed as facilitating a foraging search by preventing the revisiting of prior fixation locations. 

Smith and Henderson (2009) reported that the reduced probability of making a saccade applied to a much broader area in the direction of the previous fixation position. Furthermore, they drew attention to the relative increased probability in the forward progression of saccades, describing their observations as saccadic momentum; the tendency for saccades to continue the trajectory of the previous saccade. They reported such forward saccades to be preceded by shorter fixation durations than backward-directed saccades. Further work has established saccadic momentum as a gradient for both saccade direction and fixation duration under various viewing conditions (Bays & Husain, 2012; Luke, Smith, Schmidt, & Henderson, 2014; Smith & Henderson, 2011; Wilming, Harst, Schmidt, & Konig, 2013). Debate continues to surround the IOR and its relation to saccadic momentum and their respective mechanisms. 

Behavioral measures spurred early speculation that IOR effects reflected a continuing motor plan tied to oculomotor control systems with interest centering on the superior colliculus. A series of studies culminated in the report that while IOR components are clearly present in the discharge activity of superior colliculus neurons, the source of the IOR signal is antecedent to the superior colliculus (Dorris, Klein, Everling, & Munoz, 2002). High on the list of potential sources are frontal and parietal cortical areas involved with oculomotor control (Dorris et al., 2002; Mayer, Seidenberg, Dorflinger, & Rao, 2004). The confluence of both attentional and motor control systems in both frontal and parietal cortex raise the possibility that saccadic momentum and IOR arise from active attentional efforts that bias attention toward new or forward locations and away from prior locations (Bichot, Heard, DeGennaro, & Desimone, 2015; Bichot & Schall, 2002; Mirpour, Arcizet, Ong, & Bisley, 2009; Zhou & Desimone, 2011). The known connections and feedback of frontal and parietal areas to visual association cortex, particularly area V4 (Blatt, Andersen & Stoner, 1990; Schall, Morel, King, & Bullier, 1995), present the possibility that there are sensory correlates of saccadic momentum and the IOR in visual association cortex. Furthermore, these correlates may directly modulate the neural representation of objects in the visual scene and thus influence the guidance prioritization for visual search. 

In visual search research, emphasis has been on forward-looking attentive processing at the site of the next fixation. In this study, emphasis is placed on what has happened in the recent past to influence the current neural representation of the surround. The behavioral and physiological correlates of these questions are examined using a visual search paradigm employing arrays of conjunction style stimuli (Treisman & Gelade, 1980; Wolfe, 1994) and measuring the activity of cortical area V4 neurons during search. Spatial crowding and background effects were minimized by using simple stimulus arrays with spacing set to the receptive field (RF) eccentricity as has been previously employed (Bichot, Rossi, & Desimone, 2005; Shen & Pare, 2014). Search strategies that bias fixation selection or sequencing in scene viewing, and perhaps confound saccadic momentum measures, are reduced by using random placement of the target and distracters within these simpler stimulus arrays. 

Visual processing in V4 can be modulated by attentive mechanisms even when the stimuli are in the periphery and not the target of a following saccade (Bichot et al., 2005; Motter, 1994; Qiu, Sugihara, & von der Heydt, 2007; Reynolds & Chellazi, 2004). Here we explore the responses to peripheral stimuli after the first saccade in visual search and contrast them to the responses at the onset of the visual search array. Peripheral stimuli that share behaviorally relevant features with a target are important in the guidance of saccade selection during search. This report addresses changes in the neural response to those stimuli that result from the search process itself and, in turn, potentially alter the guidance of the next steps in search. Understanding the factors that alter the neural response to stimuli during visual search provides insight into the mechanisms that determine where we look next. 

This report establishes that during visual search nonhuman primates demonstrate saccadic momentum behaviors, and that the sensory response to stimuli in area V4 of cortex is correlated with saccadic momentum behavior. Neural correlates of saccadic momentum in V4 are shown to be independent of feature selective attention, and are overridden by directed spatial attention. 

Materials and methods

Data were obtained from two rhesus monkeys trained for behavioral neurophysiological recording experiments. Standard electrophysiological techniques were used to obtain recordings from neurons in extrastriate area V4 (Motter, 2006, 2018). The impulse activity of single cortical neurons was recorded with glass-coated Elgiloy microelectrodes inserted transdurally into the cortex. Waveforms were isolated by adjusting the position of the electrode during recording. Postmortem examinations confirmed the neurophysiological recording locations. All experimental protocols were approved by the Institutional Animal Care and Use Committee at the Veterans Affairs Medical Center and at State University of New York Upstate Medical University. The study adhered to the Association for Research in Vision and Ophthalmology Statement for the Use of Animals in Ophthalmic and Vision Research. 

Behavioral paradigms

The monkeys performed two different behavioral tasks for these experiments. The first task was a standard fixation paradigm that required the monkey to fixate steadily on a small visual target for several seconds while at the same time ignoring stimuli presented in the near periphery (1°–15° from fixation). This task was used to locate and map each neuron's receptive field (RF) and determine each neuron's response preferences for stimulus shape, size, orientation, color, and position within the RF. The monkeys were required to keep eye position inside a 1.0° window centered on the fixation spot. Eye position was measured with a scleral search coil system. Viewing was binocular. 

The second task was a conjunction style visual search task used to examine the neural responses to stimuli entering the receptive field as the result of a saccade. Stimuli were arranged on a grid pattern with the grid spacing, grid orientation, and number of elements in the array based on each cell's RF size and eccentricity, limited by the 35° × 25° display boundary. Figure 1 illustrates the basic search display, with the spacing and orientation of the grid determined by the position of the center of a RF, depicted by the dashed circle to the lower right of the initial central fixation spot. Arrays typically had 20–36 items, subject to limitations of the display screen to fully display each item and for the RF to always be within the display boundary during search; items clipped by the display boundary were eliminated entirely. 

Figure 1

 

Search array construction. The eccentricity of the receptive field (dashed circle) center and its location with respect to the point of fixation defined the grid spacing (interstimulus distance) and overall orientation of the stimulus array. Stimulus shape, size, orientation, and color varied based on the neurons' tuning preferences. Two shapes and two colors were chosen to configure four stimuli for the conjunction style search task; each trial contained a target and two sets of distracters. Trials began with fixation on a small fixation target (plus) on an otherwise blank screen. Once fixation was established the small target was replaced with a replica of the trial's target stimulus. After 300–500 ms, the cueing replica was removed and the array was simultaneously presented without any stimulus at the center location. As search for the target progressed, each fixation resulted in a stimulus appearing in the RF except for fixations that placed the RF outside of the array, or at its center.

The search task required the animals to find and fixate a stimulus (always present) in the array. The trial started with a fixation spot at the center of the screen. Once fixation was established (within 0.5° for 300 ms) the spot was replaced by a replica of the target stimulus for the trial. After 300–500 ms the replica was removed and the array was simultaneously presented without a stimulus at the center location. Subjects searched through the array until the target was found. Fixation of the target for 600 ms terminated the trial as correct, resulting in a liquid reward. Trials were terminated as incorrect if the final target fixation had not begun within 6 s. The search array remained on until the trial was declared as either correctly or incorrectly completed. The shape and size of stimuli depended on the tuning properties (see the following material) of the neuron being studied. The eye position window that identified final fixation was variable and defined to be a circle that was centered on and just encompassed the edges of the target stimulus plus 0.25° in radius. Targets were randomly selected from four stimuli (see below) on a trial-by-trial basis; a target on one trial could be a distracter on the next. The animals worked daily for about 1,200–1,500 trials at correct performance levels of >85%. 

Stimulus presentation

Stimuli were presented on a SONY GDM F520 monitor (SONY Corp., Japan) set to 22 pixels/degree at a viewing distance of 57 cm. Stimulus generation and presentation was controlled by custom software using standard graphic routines with stimulus timing synchronized with the vertical refresh of the graphics display system (Motter, 2006, 2018). The stimuli consisted of letter-like figures, such as T, I, E, L, F, O, and Z, and their nonrotationally invariant, mirror images. Each stimulus could be varied in orientation, length, width, and color. Stimuli were presented on a gray background of 8 cd/m². A standard set of color levels was used, based on the use of individual RGB guns, combinations of two guns, or white. These colors were adjusted to an average value of 20 cd/m² using an EG&G model 450 photometer fitted with a photometric filter. Individual frames were measured to assure that each frame of a presentation was identical. No attempt was made to select isoluminant colors for individual animals or neurons. A black stimulus (<1 cd/m²) was also used. 

Search stimulus selection

During the initial characterization of RF properties, fixation was maintained throughout the trial on a fixation target. During this period letter-like stimuli having different orientations, shapes, sizes, colors, luminance, or stimulus positions were sequentially delivered to the area of the RF on each trial. These trials characterized an optimal/preferred stimulus for each neuron. A stimulus duration of 200 ms and an interstimulus interval of 400 ms were used for presentation timing (Motter, 2006). The number of stimulus presentations delivered per trial as well as the sequence order was pseudorandomized to avoid prediction of both trial duration and stimulus type. RFs were mapped using a reverse correlation technique employing a slightly faster sequential presentation of stimuli at each location in a 16 × 16 grid placed over the area of the RF (Motter, 2009). 

Based on the neuronal response rate, a preferred set of color and shape features were defined. A second set of color and shape features were chosen that elicited reduced but clear responses, described here as nonpreferred features. Thus four stimulus combinations were made: one stimulus combining both preferred (PR) color and shape, two stimuli having either the preferred color (CL) or preferred shape (SH) combined with the nonpreferred second feature, and one stimulus that combined the nonpreferred features (NP). In the simple conjunction search task, the target is defined by two features, color and shape, whereas distracter stimuli each share only one of the target features. The target for each trial was randomly selected without replacement from a pool containing equal numbers of the four possible stimuli to equate presentations. The positions of the stimuli, both target and distracters, were randomly assigned to the grid locations. The proportions of the two distracter stimuli were balanced for each trial. 

The interstimulus spacing and orientation of the search array grid was set equal to the eccentricity and visual field elevation of the RF center, so that a fixation on one stimulus resulted in the placement of another stimulus at the center of the RF (Bichot et al., 2015; Mazer & Gallant, 2003). For area V4, optimal letter-like stimuli are typically much smaller than the RF (Motter, 2018); at most, one stimulus appeared in the RF on any fixation. RFs did not include the fovea. The trial started with fixation at the center of the array. A stimulus was always in the RF at the time of the array onset. 

Eye movement analysis

Data are reported with respect to two different time points. The first is the time of onset of the stimulus array during the initial fixation period. The response to the stimulus in the RF at the onset of the array measures the relative responsiveness to a stimulus after a prolonged period without any stimulus in the RF. The second time point(s) is the beginning of each midtrial fixation occurring between the initial and final fixations. Data collected during the final fixation on the trial target at the end of the trial are not used in this report. The beginning of a new fixation was defined as the time of the beginning of a 60 ms time window where the instantaneous eye movement velocity fell below and stayed less than 16°/s. Saccade onset was defined as the beginning of a 20 ms interval where velocity exceeded 16°/s. A minimum fixation duration of 120 ms was required for neural analyses. If a secondary, error-correcting saccade was made, the ensuing fixation was merged with the prior fixation if it began less than 60 ms after the end of the prior fixation and was localized to the same stimulus (Wu & Kowler, 2013). This method was applied to 2% of midtrial fixations. 

Saccadic momentum

Behavioral analyses

For the behavioral analysis of saccadic momentum two measures are made. The first measure is the duration of the fixation between the two saccades. The second measure is the vector angle between one saccade with respect to the immediately preceding saccade. The angle is defined in a counter clockwise fashion. Following convention (Smith & Henderson, 2009) a return to the previous fixation position is an angle of 0°, and a forward progression, in the same direction as the previous saccade, is an angle of 180°. The observed momentum angles were evaluated against the distribution of all possible saccade angles; obtained by measuring the angle between saccade vectors for each actual fixation location to each stimulus in the array for each midtrial fixation, that is, all possible targeting saccades. These measures define the expected angle frequencies for a Chi-square hypothesis test. 

Neural analyses

With respect to saccadic momentum, what are the analogous conditions for examining sensory neurons? Consider the diagram of Figure 2A showing an initial saccade (thick arrowed line) to a central stimulus location. Saccadic momentum states that a continuing forward saccade to a location in the lower oval is more likely than a backward saccade to a location in the upper oval. Given a stimulus in the lower oval that is identical to a stimulus in the upper oval, a neural sensory correlate of saccadic momentum can be defined as a difference in the responses to the stimuli based on which locations they occupy with respect to the just completed saccade. Ideally one would like to examine the neural response at all stimulus locations simultaneously, but to make comparisons one also needs to record from neurons with identical response properties at each location and that is not practical. Instead we examine the responses from a single neuron's RF when it falls in a leading, trailing, or any other position with respect to the direction of the just completed saccade (Figure 2B). The angular measure is made relative to the saccade vector, and not in relation to external (screen) coordinate space. The diagram in Figure 2C shows how information about saccadic momentum can be gathered from a RF at a specific retinotopic location by consideration of the angle made between the saccade vector and the RF vector (the line between the fovea and the center of the RF). The coordinate scheme used defines an angle of 180° to represent a leading saccade toward the RF (the RF leads/moves in front of the saccade) and an angle of 0° to represent a trailing saccade that goes away from the RF (thus the RF trails behind the saccade advance). Symmetrical directions to the right or left are pooled across the 0–180 axis in the neuronal data analysis. The neural correlate question is whether there is an overall response imbalance that favors the leading or forward progression. 

Figure 2

 

What is saccadic momentum for a neuron? Arrowed lines portray saccades. (A) Saccadic momentum states that a subsequent forward saccade to a location in the lower oval is more likely than a backward saccade to a location in the upper oval. A neural sensory correlate of saccadic momentum can be defined as a difference in the responses to identical stimuli located in the upper versus lower oval at the end of the first saccade. (B) Circles portray receptive fields (RF). Instead of simultaneously recording neurons with RFs at all grid locations marked by the dots in (A), the evidence for saccadic momentum is derived from repeated measures from a single neuron. The direction of saccadic momentum for a single neuron (a single RF) is derived from a consideration of the angle between each saccade vector and the neuron's RF vector. The RF vector is represented as the line between the RF center and the fovea (the fixation point). A saccade from F1 to F2 with the accompanying shift of the RF from P1 to P2 results in a large angle indicated by α. Whereas a saccade from F3 to F4 with the accompanying shift of the RF from P3 to P4, results in a small angle indicated by β. In this manner, although there is only one RF location, an entire range of saccade angles can be associated with the response to an item appearing in that RF. (C) The different angles with respect to the saccade away from the current fixation at F0 can be classified based on whether the RF leads the saccade, trails behind the saccade, or is displaced sideways.

Neural activity summaries

The activity of each neuron was normalized with respect to its response to the preferred stimulus (preferred shape and color) in the RF at the initial onset of the stimulus array. The average response activity in the interval from 50 to 150 ms after array onset was used for the normalization (Motter, 2006). For all other spike rate analyses the 200 ms response interval from 50 to 250 ms after array onset or after the beginning of a fixation is used. The analysis duration was constrained by the minimum response latency and the average initial saccade latency of 210 ms. Because the animal subjects were unconstrained in their search patterns, the number of specific stimulus and/or behavioral combinations varied, a minimum of five such occurrences per combination per neuron were required for analysis. After rate normalization, data were averaged across neurons. Activity rates in the histograms illustrating the time course of events are binned in 10 ms intervals. For population analyses the average spike rate for each condition was used in a repeated-measures (RM) analysis of variance (ANOVA) design using neurons as subjects. Pairwise multiple comparisons (Holm-Sidak method) were made between treatments. When warranted, a RM ANOVA on Ranks was used, or paired t tests. 

Results

The results are organized around three main observations: (1) behavioral evidence of saccadic momentum and fixation duration as a function of saccadic direction in nonhuman primates, (2) neurophysiological evidence for a correlation between area V4 neural activity and saccadic momentum parameters, and (3) differentiation and comparison between feature selective attention and spatially directed attention components during visual search. The behavioral data are derived from the same sets of trials used to collect the neuronal data. All behavioral data are based on midtrial saccades and fixations, thus excluding the initial saccade after array onset and the final fixation on the target. 

Where do saccades go?

Saccades preferentially targeted nearby stimulus locations of stimuli that share the target color rather than target shape. Saccades landed on stimuli that matched the target in color about 82% (83% and 81% for the two subjects separately) of the time, while landing on stimuli that matched the target shape only 13% (12% and 15% separately) of the time and landing in empty areas of the display (fixation locations > 1.5° from a stimulus center) 5% (6% and 4% separately) of the time. In previous work preferential selective attention to stimulus color has been correlated with the modulation of the neural response in area V4 (Bichot et al., 2005; Motter, 1994). Saccades that landed on shape matching stimuli, as well as empty areas, occurred sporadically within trials that otherwise appeared to be guided by color. These results are largely independent of stimulus eccentricity with the exception of landing on an empty area where probability of landing increases with eccentricity (longer saccades). Whether the latter represents targeting errors or strategy is uncertain, the longer saccades may have represented a strategy of resetting the search to a new area of the grid. The midtrial analyses excluded the initial saccade and the final saccade onto the trial's target. 

Saccadic momentum

In addition to targeting preferentially the nearby items that share the trial target's color, we found that the saccades also display saccadic momentum—the tendency for saccades to continue the trajectory of the previous saccade. Saccadic momentum was initially defined for human performance using search tasks that typically employed a target placed on a natural photographic scene. Here we establish that it also is present for monkey subjects on a regular grid search task. The regular grid also provides a more restricted framework for saccade targets thereby reducing the uncertainty of what item is targeted. The random, trial by trial, placement of stimuli within a grid array defeats any particular search strategy that might account for saccadic momentum within more natural scenes, where the varying densities of stimulus information might serve to help guide search. 

Saccade momentum was investigated by measuring the angle between two consecutive midtrial saccades. These angles were binned in 45° increments centered on the major grid axes and diagonals. An angle of 180° implies a saccade in the forward direction, a continuation of the direction of the previous saccade. The count totals for each direction form the observed outcome measures. The observed counts for both subjects are shown in Figure 3A where the radial extent for each direction is the percentage of counts for that direction relative to the total counts. For these conditions there appears to be a slight gradient in the backward direction. However, the observed saccade direction needs to be evaluated with respect to the possible saccade directions. Because the search arrays are limited in spatial extent, the probability of a saccade being made in any given direction depends on the starting position within the array, for example, saccades from fixations along the edge of an array are limited to directions back into the array. Any bounded area will always have an expected gradient of possible saccades favoring the backward direction. The number of items in the array, as long as their spatial distribution is uniform or random, does not affect the expected distribution. Even if the array is unbounded there would remain a component favoring the backward direction because in an item-to-item sequence of fixations it is always possible to go back to the previous fixation location. Bays and Husain (2012) used fixation densities and simulated saccades to derive possible outcomes. Here, the simplicity of the array allowed a more direct method. To determine the distribution of possible saccade directions, we measured for each actual fixation the angles between the previous saccade vector and those formed to all other array item locations. These measurements yielded expected outcomes under the hypothesis of a random selection of possible saccade directions. The dominance of expected outcomes in the backward direction is quite apparent in Figure 3B. Note that differences in observed and expected outcomes between subjects may reflect both behavioral differences in fixation distribution as well as the fact that the arrays differed, as they were determined by each neuron studied. After normalizing the expected counts with respect to the total observed counts, a Chi-square goodness-of-fit evaluation was made between the observed and expected distributions of outcomes. The distributions are significantly different with p < 0.001 for both subjects individually or for averaged data. To visualize the saccadic momentum hypothesis better, a ratio index was then computed as “observed” divided by “expected” for each direction, yielding a value greater than 1.0 for observed direction exceeding expected direction outcomes. This data is plotted in Figure 3C, and indicate a clear bias favoring a continued forward progression (180°) over a return direction (0°). 

Figure 3

 

Saccadic momentum. (A) Relative saccade directions for midtrial saccade pairs for each subject. Polar plot shows percentage of observed counts for saccade angles binned in 45° increments centered on major axes. Gray radial circles in A and B indicate 10% and 20% of total count levels, respectively. (B) Percentage of expected counts for possible saccade angles. Possible angles are based on the vector of the prior saccade and vectors to all other array item locations. (C) Saccadic momentum expressed as ratio of observed / expected. Ratios greater than 1.0 (inner dashed gray circle in C) indicate increased probability of occurrence, those less than 1.0 indicate decreased probability. Outer dashed gray circle in C represents a 2.0 observed / expected ratio. Magenta line for subject A; blue line for subject B.

Simple counts of saccade directions can be misleading about the probability of saccades in those directions. Of the 44,771 midtrial fixations, saccadic returns to the one-back fixation position occurred 1,308 times of 33,716 opportunities, or 3.9%, where an opportunity is defined as a midtrial fixation sequence across 3 items. The percentage is similar to our previous reports (Motter & Belky, 1998; Motter & Holsapple, 2007), where we considered this consistent with a moderate IOR phenomenon. What was not appreciated before was how saccade amplitude and the spatial distribution of saccade directions as shaped by display boundaries produce a very uneven spatial distribution of possible subsequent fixation locations. In similar work, Keech and Resca (2010) described the apparent forward bias of these results. When the sequence and spatial structure of search is taken into account, saccadic momentum (Smith & Henderson, 2009) is revealed as a forward-based gradient as shown in Figure 3C. Saccadic momentum analysis establishes a broad gradient of probability favoring the forward (180°) direction with a greater than chance ratio in the forward direction as well as a smaller than chance ratio in the reverse direction (0°). The implication for search guidance is much broader and robust than the IOR's presumed role in preventing refixation of recent locations. 

Consideration was given to the hypothesis that an appropriate control might be the distribution of only the target color matching stimuli, given the preferential targeting behavior. The random placement of stimuli actually results in essentially identical distributions, nevertheless, that procedure was done and the outcome did not differ. A second concern was whether the saccadic momentum effect observed was specific to the regular grid spacing or the associated normalization of changes across the axis length or orientation, or simply the difficulty of a search task in which stimuli and array conditions changed with each neuron being studied. A behavioral control experiment was performed with data collected from the same two subjects for search through arrays of 48 stimuli, measured in subject A before and subject B after the regular grid experiments. The stimulus positions were randomly determined, but avoiding overlap, from trial to trial within a 27.2° × 20.4° display area (Figure 4A). The same conjunction style search conditions were used employing a single set of red and blue Ts and Ls as stimuli (see also, Motter & Holsapple, 2000). In addition the individual stimuli had six possible orientations. Analyses were performed in the same manner as before using the angle between consecutive midtrial fixations. For analysis a binning angle increment of 22.5° was used; without the grid, organization items could literally be in any direction. A very similar set of results were found for each subject shown separately in Figure 4B, 4C, and 4D. The random positioning of items resulted in a more even distribution in the observed saccade directions (Figure 4B). The expected saccade directions (Figure 4C) remain biased for backward saccade directions, yielding the observed/expected ratios (Figure 4D) heavily favoring the forward saccade momentum hypothesis. The observed/expected ratios were quite similar to the regular grid search (Figure 3C). 

Figure 4

 

Saccadic momentum in control array series. (A) Search array of 48 items, with 50/50 distribution of distracter types. Items randomly placed in 27.2° × 20.4° display area with only a nonoverlap constraint. Same trial procedure as main experiment. (B) The observed relative saccade directions measured as the angle between consecutive midtrial saccades for search through the full array shown in (A). (C) Expected relative saccade direction count distribution for the full array. (D) The observed / expected ratio for search in the large array, showing saccadic momentum favoring forward (180°) direction over backward (0°) direction. (E) Observed and (F) Expected relative saccade direction distributions when the fixation between saccades was restricted to occur in an 8° × 8° area in the upper right of the array (dashed square). (G) Despite asymmetries induced by nearby boundaries the observed/expected ratio still strongly favors forward saccade momentum. (H) Observed and (I) Expected relative saccade direction distributions when only saccade amplitudes less than 6.5° are considered. This analysis approximates search in a “local” area. (J) The observed / expected ratio for search restricted to small saccades. Dashed gray circles indicate values of 1, 2, or 3 for o/e ratios. Magenta line for subject A; blue line for subject B.

The control experiment allows illustration of two additional points. First, boundaries must effect both the observed and expected saccade distributions, but if the premise of saccade moment holds then the observed/expected ratio should be maintained. This hypothesis was examined by restricting consideration of consecutive saccades to those where the intervening fixation occurred within an 8° × 8° area located at the upper right of the array (the dashed area in Figure 4A). The presence of the nearby boundaries altered the observed and expected saccade distributions; compare Figure 4B with 4E and Figure 4C with 4F. However, the observed/expected ratio Figure 4G continues to show a strong forward momentum; indicating that if a forward direction is possible it is more likely to be targeted than a backward direction. The asymmetry in the expected distributions is due to the rectangular shape of the overall array. Some idiosyncratic differences arise in the observed direction distributions between the two subjects, possibly due to differences in overall trajectory patterns (Keech & Resca, 2010). Second, by restricting saccade amplitude of the outgoing saccade (the second of the pair) rather than the location of the intervening fixation, we can approximate search under “local” conditions. This procedure also substantially reduces the effect of boundaries, thereby modestly approximating search in an unbounded array. Figure 4H, 4I, and 4J show the observed, expected and observed/expected measures when outgoing saccades were limited to being less than 6.5° (median saccade amplitude was 6.9° for control experiment). Under the saccade amplitude restriction, the observed saccade directions have a clear forward direction bias; and the expected saccade distribution (Figure 4I) has a near equal directional distribution, again with the exception of the exact backwards direction. The consequence is that for “local” areas of the search array the forward observed bias, coupled with the backward expected bias together result in an overall strong forward saccadic momentum effect as shown in Figure 4J. 

The saccadic momentum effects are not due to the regular grid array arrangement, nor to the normalization of grid axis orientation or grid length, nor to the number of array elements or saccade amplitudes. By these measures saccadic momentum is a robust phenomenon. Saccadic momentum accounts for a significant aspect of guidance in these random ordered displays. Some of the robustness seen here may be due to the absence of a higher-level search strategy that may substantially impact the expected direction distribution. 

Fixation duration correlate of saccadic momentum and O-IOR

There is a reported delay in the making of a saccade to a previously fixated location, termed an oculomotor IOR (O-IOR). The delay is measured as an increase in fixation duration in the return to a previously fixated item relative to items in other directions. In most cases this has been measured using a probe target stimulus delivered with respect to the current fixation position (Klein & MacInnes, 1999; Smith & Henderson, 2009, 2011; Luke et al., 2014) or during tasks employing fixed patterns of eye movement (Hooge & Frens, 2000) with the result that the fixation duration prior to a return saccade was found to be longer than to other locations. However, other studies have examined sequences of fixations during a visual search for a target embedded in a natural scene (Mills, Dalmaijer, Van der Stighel, & Dodd, 2015; Smith & Henderson, 2009) and find a gradient of fixation durations from shortest for those preceding forward saccades to longest for return saccades. 

Measurements were made of the duration of the fixation between consecutive pairs of midtrial saccades as well as the angle between the two saccades. Polar plots of fixation duration as a function of relative saccade direction were symmetric with respect to clockwise (CW) versus counterclockwise (CCW) directed saccades. This symmetry allowed the data to be collapsed across CW versus CCW directions for analysis. It is not clear where to draw the line between prolonged fixations due presumably to saccade momentum factors versus prolonged fixations or pauses for unknown reasons. We used a cut-off of 500 ms; a choice that should error on the side of O-IOR differences if they exist. Durations longer than that were excluded from this analysis. In Figure 5 the fixation duration as a function of relative saccade direction is plotted for each subject separately. 

Figure 5

 

Saccade momentum gradients for fixation duration. The fixation duration between, as well as the angle formed by, consecutive midtrial saccades were measured. (A) Fixation duration as a function of saccade angle for outgoing saccades less than 6.5° (red) in amplitude and saccades greater than 6.5° (blue) for subject A in the regular grid experiment. (B) Same for subject B. (C) and (D) Same measurements but for the control array experiments. In all cases the shorter saccades show a monotonic increase in fixation duration from forward to backward saccades, a possible correlate of saccadic momentum. The longer saccades do not show a similar consistent correlate. The separated rightmost points at 0.0 are the fixation durations for the specific O-IOR events, also sorted by saccade length. There is no consistent increase in fixation duration increment for the short saccade group. The longer saccades appear to have an increment in duration, but its meaning in relation to the overall gradient is not clear. Note the difference in fixation durations between the two tasks, A and B versus C and D in the same subjects. The standard error bars are often hidden behind symbols.

Initial plots of all data did not show any consistent relation between directional angle and fixation duration. Consequently, as earlier studies had often restricted saccade amplitudes, we divided the data into two sets, based on outgoing saccades with amplitudes less than or greater than 6.5°. Figure 5A and 5B show the data for the regular grid array. Figure 5C and 5D show the data for the control experiment using the array of 48 items. Data for saccade amplitudes less than 6.5° are shown by red lines in each plot and have a modest but clear increase in fixation duration from forward to backward directions. The longer saccade data, blue lines, show either mixed or opposite trends at best. 

Fixation duration does not provide as robust, or consistent, a measure of saccadic momentum as saccadic probability. Many factors not directly related to saccadic momentum can affect the measurement. For example, there are substantial differences in fixation duration in the two experimental series for the same subjects. A time constraint for deciding to saccade can influence the distribution of saccades that are made and either hide or expose an underlying time accumulating assessment of saccade direction. If subject B in Figure 5B has a more relaxed time constraint for making a saccade this could explain the steeper slope observed, and would be consistent with a longer average fixation duration as is apparent in Figure 5B and 5D. 

Do specific O-IOR cases provide a further increase in fixation duration? Regarding returns to exactly the previously fixated object, the far right data points in Figure 5 at the 0.0 direction represent fixation durations preceding exact O-IOR events, again divided into the two saccade amplitude groups. There is not a meaningful increase in fixation duration beyond the trend set across other directions for the O-IOR for the shorter saccades. Although there is clear uptick in fixation duration for the O-IOR events for the longer saccades, it is not clear how to evaluate it in regard to the generally downward fixation duration trend in the near backward direction (60° to 0°). Thus increased fixation duration prior to actual O-IOR events can be detected, just not consistently across these experiments. The reasons for the differences are not clear. 

In summary, fixation durations are subject to a variety of factors, including the direction of the saccade relative to the preceding saccade. We did not find fixation duration evidence consistent with an independent O-IOR behavior. These observations serve as a cautionary note for interpretations of the relationship between saccadic momentum and fixation duration. 

Area V4 neuronal responses during visual search

The responses of 85 area V4 single neurons were collected from two monkey subjects while they performed the visual search task in the regular grid arrays described above and in Figure 1. The interstimulus spacing and orientation of the search array grid was set to the RF center eccentricity and elevation so that a fixation on one stimulus resulted in placement of another stimulus at the center of the RF. The results presented as follows first examine the response to the stimulus in the RF at the initial onset of the array of stimuli. These “Onset” responses provide comparisons to previous work and provide a critical comparison framework for the responses during the midtrial fixations. Following these results the responses associated with midtrial fixations are examined with respect to feature attentive and saccadic momentum correlates, and finally with respect to spatially directed attention correlates. 

V4 neural response at array onset

The stimulus in the RF at the onset of the array occurs with only a blank background preceding the stimulus in the RF, and before any search saccade takes place. Array onset is the principal time point used in many search tasks. The stable conditions afforded by array onset were used to assess the response to the four different stimulus types, PR (preferred color and shape), CL (preferred color), SH (preferred shape), and NP (neither preferred color nor shape) used in the search task (see Methods). For comparisons across neurons, the response rate of each neuron was normalized with respect to the response to the preferred stimulus (see Methods). Figure 6A and 6B show peristimulus time histograms of the average population response at array onset to the four different physical stimuli, also sorted by color matching condition. The amplitude differences in response to the four stimuli were expected because the stimuli were chosen to be different based on the tuning properties of the neurons. The neural response has an onset latency of about 50 ms and a mean peak latency of 115 ms that precedes the onset latencies of the initial saccades whose mean latency is 210 ms. For analysis, spike rates are computed over a 200 ms interval starting 50 ms after array onset. The box plots of Figure 6C show the median and average raw spike rate as well as the range of Onset response rates from the 85 neurons to each of the four stimulus types, additionally sorted according to the color match between the RF stimulus and the target for the trial. For the aforementioned summaries only onset stimuli that were not the target of the first saccade are included. 

Figure 6

 

Response to array Onset. (A) Response histograms for PR stimulus (upper pair) and SH stimulus (lower pair). For each pair the black line represents the response when the stimulus matched the color of the trial's target and the red line when it did not. Small upward triangle along baseline indicates average saccade latency following array onset. Short vertical tics along baseline (at 50 and 250) mark the time window used to measure the response activity. (B) Same for histograms of CL stimulus (upper pair) and NP stimulus (lower pair). (C) Box plots of raw spikes/s activity for each stimulus and for each attentive (color matching) condition. Notes: M = matching; NM – non-matching. Yellow bar within the 25th–75th percentiles box is the mean; black, the median response. Whiskers are the 10th and 90th percentile response rate limits. (D) Response histogram to array onset when next saccade is away from receptive field stimulus (black); response when next saccade is onto the receptive field stimulus (gray). Upper pair for the preferred (PR) stimulus, lower pair for SH stimulus.

In the histograms of Figure 6A and 6B there is a clear enhancement of the response for stimuli that matched the trial's target color. In previous work (Bichot et al, 2005; Motter, 1994) subjects were trained to attend to color in order to identify the target, resulting in an enhanced neural response for color matching stimuli. Here (and e.g., Zhou & Desimone, 2011), the subjects were not trained to use color, but did so overwhelmingly. In the histograms the target color-matching condition results in both an amplitude increment and broader response duration as compared to the target color nonmatching condition. Although the overall response diminishes as a function of the tuning sensitivity, the feature selective ratio (match/nonmatch) between match conditions remains steady at about 115% across the PR, CL, and SH stimuli, and decreases to 105% with the less effective, less preferred NP stimulus as reported by Bichot et al. (2005). Interestingly, despite the SH stimulus having the preferred shape, it is the match of its nonpreferred color to the target that controls its response dynamic, consistent with the behavioral dominance of color. Note that the NP stimulus, though selected to differ from the preferred stimulus in both color and shape, was chosen to have a moderate response that averages about 60% of the PR stimulus. A repeated-measures ANOVA showed both stimulus type and match condition had significant (p < 0.001) differences, but also a significant (p < 0.001) stimulus × match interaction. Two outlier neurons were dropped from this overall analysis to meet normality and equal variance test requirements. Six other neurons did not have sufficient data in one of the eight conditions, resulting in an analysis with 73, 3, and 1 degree of freedom. Post-comparisons (Holm-Sidak) between the color match/nonmatch conditions were significant (p < 0.001) for each stimulus type except the NP stimulus (p = 0.13). Post-comparisons were also significant for stimulus type (p < 0.001) within each match condition except for the SH versus NP conditions (match: p = 0.02, nonmatch: p = 0.09). 

What about spatially directed attentive influences at array onset? For stimuli whose color matched the trial target there were sufficient trials in about 58% of the neurons (49/85) to examine the response to the stimulus when, in fact, it was the destination of the next saccade; and therefore, presumably the recipient of focal, spatially directed attention around the time of the initial saccade (Deubel & Schneider, 1996; Kowler, Anderson, Dosher, & Blaser, 1995). There were too few instances of saccades to noncolor matching stimuli to examine that combination. Figure 6D shows the time histogram curves for the PR and SH stimuli (when matching the target color) at onset when they were (black) or were not (gray) the target of the next saccade. In contrast to the feature selective difference, there was no significant population difference attributable to spatially directed attention at array onset. This array onset result contrasts with observations in Bichot et al. (2005) and in midtrial search as follows, but is consistent with Motter (1993) where stimulus onset differences in the spatially directed attentive condition became significant only with encroachment of additional stimuli into the RF. An alternate and encompassing view is that the stimulus conditions at onset produced a maximal response for each stimulus type, and directed attention effects may only come into play under conditions that otherwise produce diminished responses (see the following material). 

Zhou and Desimone (2011) also separated the Onset response from the midtrial data, differentiating them as early and late search, based on the differences in the pattern of response and latency of attentive effects. Here we examine this difference further, interpreting the midtrial differences as the adaptation, or forward suppression, effects found in Motter (2006). 

Neural correlates of feature selection during midtrial fixations

Midtrial fixations are fixations after the first saccade and before the final fixation on the target for the trial (see Methods). The responses to stimuli appearing in the RF as the result of a saccade were synchronized to the onset of each fixation, averaged across responses for each neuron, and then across neurons for the population response. To examine potential attentive feature selection during the midtrial fixations, the stimuli appearing in the RFs were sorted by the stimulus type and whether the stimulus matched the target color for that trial. Figure 7A–7D shows the population response of V4 neurons to each stimulus type and the blank (>0.33 grid units from any stimulus) condition. For comparison purposes, the array Onset response (Figure 6) for the color match condition is also shown as the gray dashed line. 

Figure 7

 

Response to stimulus during midtrial fixations. (A–D) Peri-event response histograms for each stimulus type, synchronized to the onset of the midtrial fixation at time zero. Black lines represent response to the RF stimulus when it matches the color of the target; red lines when the stimulus does not match the target's color, and cyan lines for the blank condition where there is no stimulus in the RF. Gray dashed line is response to target color-matching stimulus at array onset (from Figure 6). The two short vertical tics along baseline mark the analysis window. (E) Comparisons of response rates for Onset versus midtrial, broken down according to the target color-matching condition for each stimulus type. (F) A nearly constant 75% ratio of midtrial responses to onset responses as a function of both match condition and stimulus type. (G) Feature attentive selection advantage for color matching is comparable for Onset and midtrial conditions.

The average midtrial response is reduced compared to the average Onset response for the four stimulus types as seen in Figure 7A–7D and summarized in Figure 7E. Across all stimulus types, the feature selective advantage for target color-matching stimuli remains clear and is present earlier in the response than in the Onset case. In some cases, it may even persist across the interval from one fixation to the next. For each stimulus type the average midtrial activity for color match and nonmatch conditions are about 75% of their respective Onset response levels (Figure 7F). The constant ratio suggests a multiplicative scaling; however, the reduction is also consistent with the loss of the transient component of the V4 response as observed in flashed sequences of stimuli during maintained fixations (see Figures 4 and 7 in Motter, 2006). The effectiveness of feature attentive selection (the match / non-match ratio) has a steady 18% increment (Figure 7G) for PR, CL, and SH stimulus types, and less (∼6%) for the NP stimulus. The decreased effectiveness for the NP stimulus confirms the Bichot et al. (2005) report that effects of attentive feature selection are more pronounced in cases where the features align with the tuning properties of the neuron. This bears re-emphasis; in assessing the presence of feature attentive effects, it is necessary to evaluate the feature's effectiveness with respect to the neuron's tuning properties. The NP stimulus in this study elicits an average response of about 50 spikes/s (Figure 6C), yet there is little feature attentive differentiation in the response to the NP stimulus. The responses to all stimuli are significantly greater than the blank condition. Although the response to the NP stimulus in Figure 7D is rather unimpressive, especially considering the activity prior to fixation onset, it is clearly different than a blank condition (cyan line). A repeated-measure ANOVA on ranks with multiple comparisons showed that the response to the target color-matching stimulus (black) in the RF is greater than the non-match (red) for the PR, CL, and SH stimulus type (p < 0.01), but not the NP stimulus (p > 0.01). The feature attentive related observations, where comparable, agree with those drawn by Bichot et al. (2005) and Zhou and Desimone (2011). 

The baseline activity is elevated during search compared to the baseline prior to array onset. This is primarily due to averaging activity elicited by other stimuli that is not synchronized to the chosen fixation onset. For example, in the post-response period > 250 ms after the synchronization (and likely into the next fixation period) the activity converges toward the same ∼0.4 response level. The activity prior to the synchronization point needs to be viewed somewhat differently because it represents the tail end of the previous response indirectly synchronized to fixation onset because saccade durations (20–40 ms) are nearly constant. However, because the fixation durations vary over a larger range, the onset of the prior response is lost in the averaging, making the prior response appear to be truncated. Given that the search arrays contained only stimuli that were effective in eliciting a response, a response should occur on each fixation. The exception is the blank condition where there is no stimulus in the RF; there, the response (cyan line) decays toward the pre-onset baseline. The blank condition activity decays at a rate consistent with the decay of the sustained response after stimulus offset, which lasts around 150 ms (Motter, 2006). 

Neural correlates of saccadic momentum

Although the behavioral data for saccadic momentum demonstrate a clear bias for directional heading, the mechanisms of this action are unclear. It could be a downstream oculomotor hysteresis, but given a frontal cortical origin for the attentive strategies of search, it could also arise from a biasing of sensory information. What constitutes a neural sensory analogy of saccadic momentum? A rationale was outlined in the Methods, and diagrammed in Figure 2, for examining the differences in sensory responses in relation to saccade direction. The hypothesis is that responses from RFs that lie in the forward (leading) direction should produce a more vigorous response than those that lie in a backward (trailing) direction. For a single neuron's RF, the analysis is based on the position of the RF with respect to the saccade trajectory. 

Directional data were mirror reflected across the 0° to 180° directional line and subdivided into three groups (Figure 2C; leading, sideways, and trailing) containing approximately equal numbers of possible target locations in the regular grid arrangement. Figure 8A–8C sorts the V4 population response to the PR stimulus into these three groups. Figure 8A depicts the response when the position of the RF is in a leading direction, Figure 8B a sideways direction, and Figure 8C the trailing direction each with respect to the direction of the prior saccade. The responses are synchronized to the onset of fixation at time 0. For comparative purposes both the response when the RF falls on a blank area (cyan) and the response to the color-matching stimulus at the onset of the array (dashed gray) are shown. A diminishment in both response amplitude and duration is apparent in the response to the PR stimulus from leading to trailing positions. This change is observed for both the target color-matching (black) and nonmatching (red) conditions. Instances where the stimulus in the RF was targeted on the next saccade are specifically excluded here. That exclusion does bias the peak response in the leading direction by altering the distribution of available observations. However, it is specifically that condition where it is most likely that attention is directed to the RF during selection and initiation of a saccade (see the following material). In contrast the responses depicted in Figure 8A–8C are all cases where attention is not directed at the RF. In the absence of spatially directed attention there is a clear difference in the response to the PR stimulus that is dependent on saccadic direction relative to the RF position. On the other hand, the nonspatial, feature-selective attentive influence is present, and the directional diminishment applies to both the color match and nonmatch selective conditions. 

Figure 8

 

Neural correlates of saccadic momentum. (A) Peri-event response histogram when the PR stimulus appeared in the RF as a result of a leading direction (LD) saccade. Note the amplitude and breadth of the response in comparison to B and C. (B) Response histogram when stimulus appeared in RF as result of a sideways direction (SD) saccade. (C) Response histogram when stimulus appeared in RF as result of a trailing direction (TR) saccade. Lines in (A–C) represent conditions as in Figure 7; matching (black), nonmatching (red), and blank (cyan) conditions. (D) Population responses to the four stimulus types appearing in RF; in 30° increments of the angle between the saccade vector and the line between the fovea and RF center. Response means and standard errors for feature attentive conditions corresponding to target color-matching conditions (black) and nonmatching conditions (red).

As an aside, the differentiation between matching and nonmatching activity in the −180 to 50 ms interval in Figure 8A is due to a selection bias that alters the balance of different stimuli in the RF on the previous fixation for the leading condition, but not the sideways or trailing conditions shown in Figure 8B and 8C. The leading condition selects for sequential saccades in the general direction of the RF; this means that the stimulus in the RF on the previous fixation is likely to be the target of the next saccade. Although the analysis excluded cases where saccades to the RF were made for the current fixation, that is, after time 0.0, this did not apply to the previous fixation. Because saccades are generally made to target color-matching stimuli, then in the matching condition, the RF stimulus on the previous fixation is likely to be the same as the matching stimulus after time 0.0. On the other hand, when the PR stimulus in the RF after time 0.0 is a nonmatching stimulus then the stimulus in the RF on the previous fixation is likely to be the distracter of the other color, because in those trials that is the color that is being targeted by saccades. These relationships simply do not apply to the sideways and trailing conditions where the RF stimulus on the previous fixation is equally likely to be either distracter stimulus, or even a blank condition, and the response averaging result is therefore very similar in the period before time 0.0. 

Figure 8D summarizes the population responses for all four stimulus types showing in each case a gradient of response to an identical stimulus as a function of saccade direction in 30° increments. The responses are measured during the 200 ms interval starting 50 ms after fixation onset, as shown by the baseline markings in the peri-event time histograms. Figure 8D shows a decline in response amplitude from leading (180°) to trailing (0°) positions for the PR, CL, and SH stimuli, and a suggestion of such for the NP stimulus. The directional effects appear to be independent of the attentive, feature selective effect that separates color matching and nonmatching conditions, with one clear exception. At 0°, they converge; feature attentive differentiation is not present in the return direction. Why the color matching and nonmatching curves converge at 0° is puzzling. An explanation is elusive. Instances of a return to previous fixation (O-IOR) do fall in this bin, but the convergence remains when each actual O-IOR event is specifically excluded. 

The finer grain analysis of saccade direction presented in Figure 8D came at the cost of a reduction in the number of neurons with sufficient data for the comparisons. The matching by direction partitioning (2×6) reduced the number of participating neurons from n = 73 in Figure 8A–8C to n = 60 for PR, n = 54 for CL, n = 54 for SH, and n = 58 for NP stimulus groups. Two-way repeated-measures ANOVAs on matching and direction factors were made for each stimulus type separately. Main effects for both color matching and direction were significant at the p < 0.001 level for PR, CL, and SH stimuli but only for direction for the NP stimulus. There were significant interactions between match and direction factors (p < 0.01) for each stimulus type, that are consistent with the departure from the approximately parallel black and red line slopes depicted in the plots for each stimulus type. In postcomparisons and as seen in Figure 8D the leading (120°–180°) points had higher response rates than the trailing (0°–60°) directions. 

In summary, the results provide a clear indication that stimuli falling in RFs in the leading direction result in a larger response, particularly in the neurons tuned for those stimuli. Thus those stimuli are potentially more salient and likely to attract attention or priority in visual search. These results demonstrate a sensory based correlate of saccadic momentum. 

Neural correlates of spatially directed attention

During visual search, when the RF stimulus is the destination of the next saccade, the V4 response is enhanced compared to when the saccade goes elsewhere (Bichot et al., 2005; Hayden & Gallant, 2005). The goal here is to provide an examination of whether the enhanced response can be viewed as an extension of saccadic momentum or feature attentive processes, and whether the enhanced response exceeds the initial array Onset response. Figure 9A shows the midtrial response (magenta) to the PR stimulus in the RF when it was the destination of the next saccade, and the response (black) when the saccade went elsewhere. The enhanced response nearly matches, in both amplitude and duration the initial Onset response (dashed gray). The data are for 54 neurons where sufficient midtrial data were available for saccades into the RF for the color-match condition. Nonmatch conditions were usually not available as there were too few saccades made onto color nonmatch stimuli that happened to be in the RF. Saccades onto versus saccades away from the stimulus in the RF were associated with significantly different responses for all four stimulus types in the 50 to 250 ms interval after the beginning of fixation. The analysis, repeated-measures ANOVA on ranks (RM ANOVA), was made for the target color-matching condition for treatments of initial Onset, midtrial onto, and midtrial away conditions. Postcomparisons of midtrial onto versus away responses (Figure 9E, triangles vs squares), were significantly different (p < 0.01). This observation concurs with that of Bichot et al. (2005), although examined here for individual stimuli. For the PR, CL, and NP stimulus types, spatially-directed attention enhanced the response for the saccade onto the RF stimulus, resulting in responses rates that approached the response to that stimulus in the Onset condition (Figure 9E, circles vs triangles) and were not significantly different (RM ANOVA ranks, p > 0.05) from it. The midtrial firing rates did not exceed the Onset response. The response to the SH stimulus, however, was significantly less than the Onset response (RM ANOVA ranks, p < 0.01). For the CL, SH, and NP stimuli this is not the result of a simple saturated neuronal firing rate, as those same neurons gave greater responses to the PR stimulus. The result is consistent with a saturation limit for each separate stimulus. There was no apparent difference in the response latencies for Onset, onto and away responses, with the differentiation between them all occurring in the 75–85 ms interval postevent onset (Figure 9A). Despite the SH stimulus result, the general outcome suggests that directed attention accompanying a saccade to a RF stimulus overrides the diminishment (forward suppression) of midtrial responses that are observed when the RF stimulus is not the destination of the next saccade. 

Figure 9

 

Neural correlates of spatially directed attention. (A) Peri-event response histogram showing response to target color-matching distracter in receptive field when next saccade goes to it (magenta) versus away from it (black). Population response based on 54 neurons. (B) Response histograms showing subdivision of blue response in A, into leading (black), sideways (blue), and trailing (cyan) directional groups. Responses are based on different numbers of neurons as indicated due to limitations in availability of different combinations of stimulus and saccade events. (C) Response histograms for target color-matching (black) and nonmatching (red) stimulus conditions when the next saccade was to the RF stimulus, for a subset of 10 neurons. (D) Response histograms to the actual trial target in the RF when the next saccade was to the target (black) and when the next saccade was away from the target (green). A reduced data criterion was used to include neurons for D, as explained in main results section. For A–D the gray dashed line represents response to target color matching distracter at array onset. (E, F). Mean population responses to the four stimulus types in various conditions detailed in the inset legends and main text. Onset refers to array onset response. Onto refers to condition where next saccade goes to stimulus in the RF. Away refers to condition where next saccade goes away from the stimulus in the RF. Notes: dist = distracter; m = matching; nm = nonmatching condition.

Moore and Chang (2009) reported that stimulus discrimination by area V4 neurons differed depending upon whether a saccade went to or away from the RF. Figure 9E compares stimulus discrimination using the differential responses to the four stimulus types, and shows that the curve for the saccade onto a stimulus in the RF (triangles) is much steeper, that is, increased discriminative power, than that for the saccade away case (squares), confirming Moore and Chang (2009). The differences in stimulus discrimination imply a loss of stimulus information in the response during midtrial fixations. Note, in counterpoint, differences in the feature attentive conditions (squares vs diamonds) did not change the slopes of the tuning curves in Figure 9E, nor at array onset (circles and triangles, Figure 9F). 

Next we examined whether either saccade momentum or feature selective differences alter the responses to the RF stimulus when it is the destination of the next saccade. For these comparisons the data were limited to much smaller subsets due to insufficient numbers of the specific stimulus or saccade direction combinations in most neurons. Figure 9B presents the saccade momentum analysis for the PR stimulus in cases where the RF stimulus matches the target color and is the destination of the next saccade. The histograms are formed individually from the set of data available where leading, sideways, and trailing effects were supported by n = 20 (black), n = 32 (blue), and n = 9 (cyan) neurons, respectively. A tiered pattern comparable to that seen in Figure 8 (where the saccade does not go to the RF) is not apparent here and, in particular, there is no evidence for a decreased response in the trailing case. Thus, when the RF stimulus is the target of the next saccade, saccadic momentum response differences are quashed, likely the consequence of response saturation. This also appears to rule out any notion that saccadic momentum is antecedent to or determines directed attention. 

Similarly, feature selective differences between the target color matching and non-matching conditions are greatly diminished when the RF stimulus is the destination of the next saccade. Because saccades to nonmatching color stimuli are infrequent, only a small subset of neurons had sufficient data to examine feature selection under these conditions. Figure 9C shows the averaged midtrial responses for 10 neurons when the PR stimulus in the RF becomes the destination of the next saccade. In this case there is little difference between the target color matching and nonmatching responses (p > 0.05, t test, n = 10). In addition, there is no significant difference between the match condition response and the array Onset response recalculated for those same 10 neurons (p > 0.05, t test). Similar insignificant differences were observed for the remaining stimulus types. By driving the responses to a maximal limit, directed attention appears to quash both saccadic momentum and feature selective effects. 

Finally, the actual target for the trial occasionally appeared in the RF. Is the response to a stimulus different when it is the actual trial target versus when it is a color-matching distracter? By reducing the minimum number of required responses per condition per neuron for this relatively rare stimulus combination to two, a comparison can be made between responses when the trial's target was in the RF and became the destination of the next saccade versus when the saccade went elsewhere. It is the latter “elsewhere” case that is quite rare and required the criterion of two to obtain at least 30 neurons in all stimulus types. The results for the PR stimuli are shown in Figure 9D, and for all stimulus types in Figure 9F. The response to the PR stimulus when it is the trial's target is not different from the response to the PR stimulus when it is one of the distracters and both stimuli are the destination of the next saccade. And, in turn, both are no different from the response to the PR stimulus at array onset for the color matching condition. This result is true for all four stimulus types (Figure 9E, circles and triangles; 9F, circles and squares). Interestingly the response to the target stimulus when the next saccade goes elsewhere is like the response at array onset when the stimulus is a color nonmatching condition (Figure 9F, diamonds and triangles). Earlier in the results (Figure 6D) no difference was found between Onset responses when the stimuli were or were not the destination of the next saccade. Here for midtrial responses there is a difference, both for distracters (Figure 9E, triangles and squares) and targets (Figure 9F, squares and diamonds). It is important to recognize that the response to array onset for the color match condition appears to be an upper limit across all conditions described already. 

In summary, spatially directed attention overrides saccadic momentum and feature selective attention phenomena during the processing of stimuli that are the destination of the next saccade. Note that this increased activity, contingent on the next saccade being made to the RF stimulus, occurs at the beginning of the fixation. This indicates that on average the next saccade target has already been determined, evidence perhaps of a planned sequence of saccades marked by a color cue (Gersch, Kowler, Schnitzer, & Dosher, 2009; McPeek, Skavenski, & Nakayama, 2000). Furthermore, the equivalence of responses under target color-matching conditions at array onset, and to stimuli in the RF that are the destination of the next saccade (whether a distracter or the actual target) argues that these responses reach a saturating level of activity. Thus an “enhanced” response might actually be a return to standard, rather than an active process of augmentation. By comparison with previous work (Motter, 2006) the midtrial responses that do not reach this saturation level lack a significant initial transient component, suggesting that directed spatial attention may somehow restore that component of the visual response in V4 neurons. 

Discussion

In this report, saccadic momentum, the tendency for saccades to continue the trajectory of the previous saccade during visual search is confirmed for conjunction style search by nonhuman primates. The results confirm a gradient from higher probability for forward progression to lower probability for backward regression. The saccadic momentum gradient is robust. Here the use of trial-by-trial randomization of array items defeats any particular search strategy that might account for saccadic momentum, and establishes saccade momentum at some intermediate level of control, consistent perhaps with a foraging facilitator role but absent a higher cognitive strategy associated with picture viewing. The implications for search guidance are much broader than the IOR's presumed role in preventing refixation of recent locations. The saccade momentum gradient was complemented by a similar gradient based on fixation duration in some cases, but no robust evidence for an independent oculomotor IOR based on fixation duration and acting in addition to the saccade momentum gradient was found. Saccadic momentum and O-IOR have been dissociated across different visual tasks (Bays & Husain, 2012; Dodd, Van der Stigchel, & Hollingworth, 2009; Luke et al., 2014; Smith & Henderson, 2009). Whether our result is due to the specifics of this visual task, a sparse array of objects (vs a visual scene) or the task requirement of search (vs free viewing, memorization, or patterned saccade) is unknown. Wilming et al. (2013) also did not find evidence of an O-IOR beyond the saccade momentum gradient when salient areas were segregated in their analysis. 

Our previous results with repeated flashed stimuli indicated there should be a significant diminishment of area V4 neural responses during visual search compared to the initial array onset response contingent on the repeated presence of an effective RF stimulus across fixations (Motter, 2006). The present results confirm this observation (Figure 7). The amount of reduction is consistent with the loss of the transient component of the V4 response (Motter, 2006). Feature attentive selection based on color matching was present in both initial onset and midtrial responses, generally confirming previous reports (Bichot et al., 2005; Zhou & Desimone, 2011), and suggesting feature attentive selection effects are not reliant on the transient response component. After excluding the initial fixation, no correlation was found between the neural activity and the number of fixations prior to finding the target. 

In this study there was not an enhanced response to stimuli that became the target of the next saccade at array onset. Previous search studies (Bichot et al., 2005; Zhou & Desimone, 2011) reported an enhanced V4 response to targeted stimuli at the onset of the search array. The difference may be associated with the longer cue presentation period in those studies, or to the maintained fixation (500 ms) on a neutral fixation target between the cue period and the onset of the search array in those studies compared to this study. These differences may result in a different strategy or initial deployment of attention. In this study there was an enhanced response to stimuli that became the target of the next midtrial saccade, and that enhanced response matched but did not exceed the initial array onset response. This was the case for each of the four stimulus types (Figure 9E), suggesting that the array onset response represents a maximum response for each stimulus. Directed spatial attention apparently restores the response up to this level but does not enhance it further. It is also significant that the directed attention quashes the feature selective differentiation (Figure 9C) as has been reported for popout modulation in a saccade task (Burrows & Moore, 2009). 

A description of the geometry of saccadic momentum with respect to the movement and positioning of neuronal RFs during search was developed (Figure 2). The response of area V4 neurons to stimuli that entered the RF as the result of a saccade was found to be a function of the angle between the saccade vector and the line between the fovea and the center of the RF. The results agreed with the saccadic momentum geometry, finding that the responses from RFs that lie forward of the saccade direction were increased relative to responses from RFs that lie behind the saccade direction. Indeed, the response amplitude measures formed a monotonic gradient varying as a function of the directional angle that matched the behavioral saccadic momentum gradient (Figure 8). The gradient establishes a salience bias that predicts saccade direction (Mazer & Gallant, 2003) and is correlated to the previous saccade. These findings establish a saccadic momentum correlate in the sensory response of extrastriate area V4 neurons. How is that possible? 

Is saccadic momentum the result of attentive guidance? Feature selective attention and spatially directed focal attention are postulated to arise in the frontal cortex and pass to V4 via the frontal eye fields (Bichot et al, 2015; Bruce & Goldberg, 1985; Gregoriou, Gotts, Zhou, & Desimone, 2009; Ninomiya, Sawamura, Inoue, & Takada, 2012; Schall, 2015; Zhou & Desimone, 2011), thus providing a possible linkage to oculomotor events. Feature selective attentive differences between match and nonmatch conditions did not differ as a function of the saccade momentum gradient (Figure 8D). Thus there appears to be no general dependency between feature attentive selection and saccadic momentum. Therefore, spatially directed attention is the most likely attentive guidance candidate. However, the analysis establishing the saccadic momentum gradient specifically excluded any stimulus that was the target of the next saccade, potentially excluding spatial attention as a foundational component. Inclusion of that data (Figure 9E, black triangles) augments the slope of the gradient (Figure 8D). This suggests that saccadic momentum provides a bias for eye movement preparation but not necessarily execution of one. 

Various studies (Gregoriou, Gotts, & Desimone, 2012; Ninomiya et al., 2012; Thompson, Bichot, & Sato, 2005) conclude that V4 neurons receive information only from the visual neurons in frontal eye field (FEF) and not visuomotor or motor FEF neurons. Recently Merrikhi et al. (2017) have found that only visual delay neurons in FEF, which do not respond at the time of the saccade in a delay paradigm, project to area V4. Nevertheless, area V4 does appear to receive saccade preparation information prior to a saccade. Steinmetz and Moore (2014) found evidence of saccadic information being relayed to V4 even when it is dissociated from the direction of spatial attention. Furthermore, Noudoost, Clark, and Moore (2014) found that presaccadic enhancement but not feature discriminability of V4 neurons survives FEF inactivation, raising the probability of other sources. An anatomically based argument can be made for saccade preparatory related information arriving in V4 from lateral parietal (LIP) areas, even via a potential relay from frontal cortex (Blatt et al., 1990; Ipata, Gee, Goldberg, & Bisley, 2006; Ninomiya et al., 2012). If FEF or LIP activity designating a particular region of space associated with preparation for the next saccade modulated V4 neurons it should bias the V4 sensory response (Thompson, Hanes, Bichot, & Schall, 1996; Moore & Armstrong, 2003). However, the timing seems wrong with the V4 response occurring very early in the fixation period, well before the next saccade and not building within the period prior to the saccade (see also, Mazer & Gallant, 2003). 

The previous considerations are all forward-looking without a clear linkage to the previous saccade. This draws into question whether saccadic momentum activity is derived from a preparatory bias for the forthcoming saccade, suggesting instead that the saccadic momentum phenomena itself may reflect a bias caused by the previous saccade. Such a linkage exists if we postulate that the activity in FEF (or LIP) associated with the prior saccade is what biases the response of V4 neurons to the stimuli present after the saccade. The peak visual activity in FEF occurs at the location of the saccade goal in the retinotopic map irrespective of the target's actual salience (Thompson, Bichot, et al., 2005). This activity peaks at the time of the saccade and outlasts the saccade for a short interval. Therefore, the retinotopically organized feedback to V4 during and immediately after a saccade is greatest in the retinotopic direction of the saccade and falls away to either side. Even covert saccade conditions generate this activity (Thompson, Biscoe, & Sato, 2005) and indeed covert shifts of attention as marked by microsaccades are associated with attentional modulation in area V4 (Lowet, et al., 2018). Thus FEF (or LIP) feedback to V4 could underlie the bias in responsiveness to stimuli appearing in the RFs of V4 neurons on the next fixation. Convergence from the large overlapping RFs in FEF or LIP (Hamed, Duhamel, Bremmer, & Graf, 2002; Mayo, DiTomasso, Sommer, & Smith, 2015) may provide the graded response bias that underlies the saccadic momentum gradient with the distribution of the feedback onto V4 being the critical parameter. The visual bias thus established in V4 neurons in turn feeds forward, closing a positive control loop that gives rise to saccadic momentum, possibly establishing a default saccade sequencing strategy. Given this viewpoint the nature of the activity feeding back to area V4 around the time of the saccade, that is, visual versus motor preparation as sorted out by maintained fixation and delayed saccade paradigms, may not matter, rather the envelope of feedback activity surrounding a saccade in a sequence of saccades may be the critical determinant. 

Posed in this manner saccadic momentum seems rather unintentional, somewhat like priming effects that play significant roles in modifying behavior. Perhaps saccadic momentum plays an unrecognized role in other, nonfree viewing paradigms, although it may specifically be a component of the planning of sequences of saccades. Although little evidence supporting a separate sensory IOR mechanism was found, saccadic momentum alone provides a mechanism to bias stimulus salience in a forward direction and against a return to the recently fixated locations in models of visual search. Longer fixations prior to reverse direction saccades may result from having to work against this gradient. Perhaps, unintentional is misleading, and what has been observed is the strength of frontal or parietal areas in the control of sensory processing in terms of expectations or anticipated action; an effect amplified (or isolated) in this search paradigm by the selection of a task that eliminates competing higher-order viewing strategies. 

Acknowledgments

The author thanks Heather Bergsbaken for her excellent technical support. This work was supported by grants from the National Eye Institute, NIH R01-EY018693, and from the Veterans Affairs Bio-Medical Research Program. 

Commercial relationships: none. 

Corresponding author: Brad C. Motter. 

Email: [email protected]. 

Address: Veterans Affairs Medical Center, Syracuse, NY, and Department of Neuroscience and Physiology, SUNY Upstate Medical University, Syracuse, NY, USA.