Eye movements reset visual perception

Abstract

Abstract

Abstract:

Abstract  Human vision uses saccadic eye movements to rapidly shift the sensitive foveal portion of our retina to objects of interest. For vision to function properly amidst these ballistic eye movements, a mechanism is needed to extract discrete percepts on each fixation from the continuous stream of neural activity that spans fixations. The speed of visual parsing is crucial because human behaviors ranging from reading to driving to sports rely on rapid visual analysis. We find that a brain signal associated with moving the eyes appears to play a role in resetting visual analysis on each fixation, a process that may aid in parsing the neural signal. We quantified the degree to which the perception of tilt is influenced by the tilt of a stimulus on a preceding fixation. Two key conditions were compared, one in which a saccade moved the eyes from one stimulus to the next and a second simulated saccade condition in which the stimuli moved in the same manner but the subjects did not move their eyes. We find that there is a brief period of time at the start of each fixation during which the tilt of the previous stimulus influences perception (in a direction opposite to the tilt aftereffect)—perception is not instantaneously reset when a fixation starts. Importantly, the results show that this perceptual bias is much greater, with nearly identical visual input, when saccades are simulated. This finding suggests that, in real-saccade conditions, some signal related to the eye movement may be involved in the reset phenomenon. While proprioceptive information from the extraocular muscles is conceivably a factor, the fast speed of the effect we observe suggests that a more likely mechanism is a corollary discharge signal associated with eye movement.

Introduction

Humans visually explore their environment by making three to four fixations every second, each followed by a saccadic eye movement to the next object of interest. The problem the brain faces interpreting visual input is illustrated in Figure 1: Neural activity in visual areas of the brain is a continuous stream but perception occurs in discrete epochs of time corresponding to eye fixations. A question that has received considerable attention is how we perceive a stable visual world amidst the saccades (Melcher & Colby, 2008; Von Helmholtz, 1924; Wurtz, 2008); i.e., how is our visual experience tied together from one fixation to the next? A complementary problem that has received much less attention is how the brain breaks apart the continuum into neural activity that distinctly codes information on each fixation. In other words, how is the beginning of a new fixation recognized so that subsequent neural activity can be interpreted as the representation of a distinct image? In most laboratory studies of vision this question is circumvented—the experiments impose a stimulus onset time by flashing an image after a subject has held fixation for an extended time. However, in natural vision, objects come into view by saccades rather than flashing and it is unclear how visual analysis is reset so that neural signals can be parsed. 

Figure 1

 

The visual parsing problem. (a) A record of eye position in a macaque freely viewing a complex natural image. A saccade about 6° in amplitude separates two fixations. (b) An unfiltered recording from macaque area V1 shows the continuous stream of neural activity at different frequencies that spans the fixations and intervening saccade. Parsing continuous neural activity into epochs associated with objects seen on distinct fixations would be simplified if visual processing was informed when a new fixation begins. Data are from experiments reported in Ruiz and Paradiso (2012).

One might speculate that neurons' sudden response onset at the start of each new fixation is sufficient to synchronize perception with brain activity. However, there is not a simple relationship between stimulus onset time and neural response onset: From cell to cell, visual response latencies in the visual cortex cover a wide range and the latency of any given cell is influenced by contrast and other parameters (Bair, Cavanaugh, Smith, & Movshon, 2002; Huang & Paradiso, 2005; Lennie, 1981; Maunsell & Gibson, 1992; Tolhurst, Movshon, & Thompson, 1981). Also, V1 responses tend to be transient (Muller, Metha, Krauskopf, & Lennie, 2001), so a decline in response is not a reliable indication of the end of a fixation. Considered together, these factors suggest that visual signals alone may not suffice to precisely infer fixation start and stop times; information from eye movements may be advantageous. The need for rapid visual analysis is underscored by studies demonstrating that behavioral decisions based on visual input can be made extremely quickly (Stanford, Shankar, Massoglia, Costello, & Salinas, 2010; Thorpe, Fize, & Marlot, 1996). 

There is no question that neural activity across many brain structures is modulated by saccades. Lambda waves were first reported over 50 years ago (Evans, 1953). These occipital electroencephalography (EEG) transients are associated with saccadic eye movements used to explore complex images; they disappear in the dark, with uniform visual stimulation or with prolonged fixation (Brigo, 2011). Numerous studies have shown saccade-related neural activity in BOLD signals, local field potentials (LFPs), and spiking activity (e.g., in occipital cortex, Bodis-Wollner et al., 2002; Purpura, Kalik, & Schiff, 2003; Rajkai et al., 2008). In some brain areas and in some situations, there is also receptive field remapping prior to saccades which alters visual responses (Churan, Guitton, & Pack, 2011; Melcher & Colby, 2008; Nakamura & Colby, 2002; Parks & Corballis, 2010). During saccades there are changes in brain activity (e.g., Bremmer, Kubischik, Hoffmann, & Krekelberg, 2009; Reppas, Usrey, & Reid, 2002) that may be related to the perceptual phenomenon of saccadic suppression (e.g., Matin, 1974; Ross, Morrone, Goldberg, & Burr, 2001; Sylvester, Haynes, & Rees, 2005; Yu & Lee, 2000). 

Thus it is well established that brain activity is modulated by saccades. In our own research we find further that macaque V1 neurons carry information sufficient to precisely estimate the start of new fixations (Ruiz et al., 2010). The question addressed in the present study is if there are perceptual indications that vision is reset on new fixations. To our knowledge, there is only one previous experiment that investigated this question, that one using the Necker cube and other bistable stimuli (Ross & Ma-Wyatt, 2004). The Necker cube is typically perceived in one of two states for intervals of 3–5 s. There was prior evidence of an interaction between perceptual state and eye movements, as it was found that fixations are longer in duration just after a state reversal, perhaps reflecting the time needed to establish the new representation (Ellis & Stark, 1978). In the Ross and Ma-Wyatt (2004) study, bistable stimuli were intermittently presented and it was found that saccades shorten the duration of states of ambiguous figures, suggesting that visual input on a previous fixation may be erased (Ross & Ma-Wyatt, 2004). Our goal was to investigate visual reset with a standard measure of visual performance that is thought to tap into low level cortical mechanisms—orientation discrimination. We find that stimulus orientation on each fixation transiently influences the orientation perceived on a subsequent fixation. Saccades significantly reduce the magnitude of this influence, consistent with the hypothesis that vision is reset, at least in part, by an eye movement signal. 

Methods

To explore the possibility that eye-movement information assists in resetting and parsing visual signals, an orientation discrimination experiment with Gabor stimuli was used to measure the extent to which perception on one fixation influences perception on the next (Figure 2). Unlike studies of saccadic suppression (Matin, 1974), the experiments here assessed interactions between stimuli seen during fixations rather than perception of stimuli presented during saccades. Presumably because of the behavioral importance of edge detection, humans are remarkably sensitive to the orientation of lines or edges; under optimal conditions a tilt less than 0.5° can be reliably discriminated (Andrews, 1967; Paradiso & Carney, 1988). 

Figure 2

 

Two experiments used to examine the influence of one stimulus' orientation (Gabor 1) on the perceived orientation of a second stimulus (Gabor 2). (a) Task with real saccades. After a subject fixates a 0.1° radius red target (exaggerated in size for illustrative purposes) for 500 ms, a Gabor stimulus with orientation of ± 8°, ± 4°, or 0° relative to vertical is displayed for 250 ms. At the end of the Gabor 1 display period, a green fixation point appeared 10° to the right. The subject made a saccade to Gabor 2 (orientations were ± 4°, ± 2°, ± 1°, and 0° from vertical) and indicated whether it appeared tilted clockwise or counter-clockwise from vertical. To limit the time interval available to make the discrimination, a binary noise stimulus (mask) was presented at various times after Gabor 2 and stayed visible until a behavioral response was made. (b) Task with simulated saccades. The subject held fixation at the central location throughout the trial. The two Gabor stimuli and the mask appeared in the same temporal sequence as with a real saccade. However, in this simulated saccade condition, Gabor 2 moved into view rather than having a saccade take the eyes to the stimulus.

Subjects and visual stimuli

Three subjects participated in the study, all with normal or corrected-to-normal vision. Subjects sat in a dimly lit room and used a bite bar that positioned the eyes 60 cm from a visual display (160 Hz). The display used in the experiments was an Iyama HM204DT CRT with a P22 phosphor. An infrared eye tracker measured eye position at 240 Hz (IScan Inc). Stimuli were two-dimensional Gabor patterns (Gaussian damped luminance sinewaves) with a spatial frequency of 0.8 c/° and a peak Michelson contrast of 0.1. Two experiments were used to examine the influence of one stimulus' orientation (Gabor 1) on the perceived orientation of a second stimulus (Gabor 2). In one version of the experiment, subjects made a saccade to view Gabor 2 and in the second version subjects held fixation while a simulated saccade moved Gabor 2 into the fovea. In both experiments subjects were instructed to avoid blinking during the trials. This was readily achieved as subjects initiated each trial with a button press and could pause as desired. 

Experiments with saccades

As detailed in Figure 2, in the experiment with real saccades, subjects viewed a Gabor stimulus at various orientations for 250 ms and then made a saccade to view a second Gabor stimulus 10° to the right. The task was a two-alternative forced-choice discrimination of the Gabor 2 tilt (clockwise or counter-clockwise from vertical). The contrast of Gabor 2 was set at a low value so that it could not be detected when the subject fixated Gabor 1. At an eccentricity of 10° contrast threshold is about 0.25 (Rovamo, Virsu, & Nasanen, 1978) and, as we confirmed, the 0.1 contrast Gabor could not be detected above chance levels before initiating the saccade. Indeed, with static fixation, contrast threshold should not be reached until the eye is less than approximately 2° from the Gabor 2 fixation point (Rovamo et al., 1978). Moreover, at saccadic velocities contrast threshold would not be reached until the eyes are significantly less than 2° from the second fixation point (Kelly, 1979; Robson, 1966). All of this is to say that our best estimate is that the temporal interval in which Gabor 2 can be examined does not begin until the eyes are just about at the second fixation point. 

To limit the time available to discriminate the Gabor 2 orientation, a binary noise stimulus (mask) was presented at various times after Gabor 2 and stayed visible until a behavioral response was made. The mask was meant to truncate neural activity on the second fixation (Figure 1) so that the orientation discrimination had to be performed with the early activity on the second fixation. Because of the short duration and low contrast of the stimuli, afterimages were not seen. To estimate the viewing time at the second fixation point prior to masking, we had subjects perform a large number of 10° saccades and we recorded the time it took to plan and execute them. As Gabor 2 was turned on at the same time as the second fixation point, we estimated the foveal viewing time of Gabor 2 by subtracting the average planning and execution time from the display time prior to masking. Gabor 2 duration was adjusted to give estimated viewing times (i.e., time during which Gabor 2 was foveated) of 50, 100, 150, and 400 ms. For brevity in describing the results, we refer to these as “Gabor 2 times.” Note that for all stimulus display times in this study, actual display times are roughly 5 ms shorter than those specified because on the last display frame the P22 phosphor decays before the frame is over (Bridgeman, 1998; Elze, 2010). 

The three observers completed somewhat different numbers of trials. There were trials at each of four Gabor 2 times, seven Gabor 1 orientations, and five Gabor 2 orientations. Observer TL completed 72 sets of data (10,080 trials), AG completed 57 sets (7,980), and DM completed 60 sets (8,400). 

For trials to be accepted, subjects' saccades had to decelerate and remain within a 1.7° window around Gabor 2. In this experiment and the simulated saccade experiment below, each data point in the graphs comes from 57–75 discriminations per subject. Corrective saccades did not cause a trial to be aborted as long as the eyes stayed in the fixation window effectively keeping the stimulus in the fovea. Likewise, no attempt was made to limit fixation saccades. It is worth noting that at 0.8 c/°, the data should not be significantly affected by saccadic suppression (Burr, Morrone, & Ross, 1994). 

Experiments with simulated saccades

To assess the possible role of saccadic eye movements in visual reset, a second experiment was conducted. The sequence and timing of visual stimulation were as similar as possible to the experiment described above, but the subject held fixation at the central location throughout the trial. To simulate the changes in visual stimulation produced by saccades, subjects' eye movements were measured and interpolated at the 160 Hz display rate. On average, it took observers 100 ms to initiate a saccade and the 10° saccade had a duration of 44 ms. The fast latency of the saccades is consistent with reports that express saccades are common when eye movements are made to trained locations, especially on a blank background (Bibi & Edelman, 2009; Fischer & Boch, 1982; Fischer, Boch, & Ramsperger, 1984). 

At the start of a simulated-saccade trial, Gabor 1 appeared for 250 ms as in real-saccade experiments. Gabor 2 then appeared at the peripheral fixation point and was stationary for 100 ms to simulate the time prior to saccade initiation measured in real saccades. Gabor 2 was then presented for single video frames (6.3 ms) at successive locations moving toward screen center based on the accelerations and speeds of real saccades. The stimulus center locations were: 10°, 9.2°, 7.1°, 4.9°, 3.2°, 1.6°, 0.4°, and 0° from screen center (total duration of simulated saccade = 7 × 6.3 ms = 44.1 ms). After Gabor 2 was viewed for a variable duration (50, 100, 150, and 400 ms), a visual mask appeared as in the real-saccade experiment. As the computer controlled when Gabor 2 entered the fovea in the simulated-saccade experiment, we were able to control viewing time prior to masking and match values to those in the real-saccade experiment. In simulated saccade experiments, subjects had to keep their eyes within a central 1.7° window for the entire trial. As in the experiments with real saccades, normal fixation saccades were not limited. 

The three observers completed somewhat different numbers of trials. There were trials at each of four Gabor 2 times, seven Gabor 1 orientations, and five Gabor 2 orientations. Observer TL completed 75 sets of data (10,500 trials), AG completed 57 sets (7,980), and DM completed 60 sets (8,400). 

Unlike with real saccades, in the simulated saccade experiments, fixation was maintained. Due to this, a potential concern in the simulated saccade experiments is that the effective Gabor 1 duration might be longer than desired because of persistence in the CRT phosphor. However, this is not an issue as the P22 phosphor of the Iyama display decays to near zero less than 2 ms after initial excitation (Elze, 2010). 

Control experiments

A control experiment was performed to further assess the influence that a stimulus seen on one fixation has on the perception of a stimulus on a subsequent fixation. Data collected in the real- and simulated-saccade experiments indicated that at short Gabor 2 viewing times the perceived tilt of Gabor 2 was biased by the orientation of Gabor 1. In the control experiment, the real- and simulated-saccade experiments described above were repeated with one change: At the beginning of the sequence in Figure 2, the first fixation point was shown for the same time as in the regular experiments, but no Gabor 1 stimulus was actually shown. Gabor 2 was presented at one of six orientations: +/− 4°, +/− 2°, +/− 1°. Subjects indicated whether Gabor 2 appear tilted clockwise (CW) or counter-clockwise (CCW) from vertical. Rather than computing bias as in the main experiments, we were able to compute the percentage of correct responses to see if the task was doable, particularly at short Gabor 2 time. In real-saccade control experiments, all three observers completed 1,440 trials (four Gabor 2 times at six orientations repeated 60 times). In simulated-saccade control experiments, observer TL completed 60 data sets (1,440 trials), AG completed 45 sets (1,080 trials), and DM completed 75 sets (1,800 trials). 

Results

The data obtained from three observers in the experiment with real saccades are shown in Figure 3. Figure 3a shows a standard psychometric curve for orientation discrimination of Gabor 2 where psychometric curves were fit to the data as logistic functions (origin). In Figure 3a, the probability that the second Gabor was perceived clockwise from vertical at Gabor 2 viewing times of 50, 100, 150, and 400 ms is displayed. As one would expect, the more clockwise Gabor 2 was, the more likely the subjects were to say that it was tilted clockwise. As Gabor 2 time is reduced from 400 ms to 50 ms, the curves flatten, indicating that variations in the tilt are more difficult to discern. 

Figure 3

 

Effect of Gabor 1 and Gabor 2 orientation on the perception of Gabor 2 when a saccade separates the two stimuli. The percentage of clockwise responses averaged across three observers at four Gabor 2 times (50, 100, 150, 400 ms) as a function of Gabor 2 (a) or Gabor 1 (b) orientation. The differently-colored curves show performance separately for trials with different orientations of Gabor 1 (a) or Gabor 2 (b). Data for each Gabor 2 time have a narrow range of Gabor 2 viewing times (checked for variable saccade planning time) so that the data can be compared to simulated saccade data (Figure 5). Error bars show the standard error of the mean. At longer Gabor 2 times, the perception of Gabor 2's orientation is correct with little influence from the tilt of the preceding stimulus. At shorter Gabor 2 times the opposite is true: The perceived orientation of Gabor 2 follows more closely the tilt of Gabor 1 rather than Gabor 2's actual orientation (top row). Bias is a measure of the influence of Gabor 1 on the perception of Gabor 2. Bias is defined as the average distance between data points at the − 8° and + 8° Gabor 1 orientations (bias is calculated at all Gabor 2 times but is shown here with a black arrow at the 50 ms time).

The differently-colored curves in Figure 3a show performance separately for trials with different orientations of Gabor 1. Surprisingly, at all Gabor 2 times, when Gabor 1 was more clockwise, the subjects were more likely to say that Gabor 2 was clockwise (e.g., pink and green traces displaced upward compared to red and black traces). Also apparent as Gabor 2 time decreases is an increasing separation between the curves representing different Gabor 1 orientations (i.e., the influence of Gabor 1 on the perceived tilt of Gabor 2 increases as there is less time to view Gabor 2). Evidently, despite the loss of sensitivity that occurs during saccades (saccadic suppression) (Matin, 1974), there are large trans-saccadic influences at the start of a saccade. Consistent with studies of trans-saccadic memory, subjects did not simultaneously see Gabor 1 and Gabor 2 (Bridgeman & Mayer, 1983; Irwin, 1996; Rayner & Pollatsek, 1983); rather, Gabor 1 biased the perception of Gabor 2. Note however that the direction of the interaction is opposite that seen with adaptation in the tilt aftereffect (Gibson & Radner, 1937). Figure 3b replots the data to directly show the probability that Gabor 2 was perceived clockwise from vertical as a function of Gabor 1 orientation. At the shortest Gabor 2 time there is a strong relationship between Gabor 1 tilt and Gabor 2 perception. This relationship falls off at longer Gabor 2 viewing times. The results in Figure 3 make the important point that the transition from visual analysis on one fixation to the next is not instantaneous. When there is sufficient viewing time (hundreds of milliseconds as in a normal fixation), the tilt of a visual stimulus is veridically perceived and it depends little or not at all on the properties of a preceding stimulus. At intermediate times, the perception of Gabor 2 has a mixture of the attributes of the first and second stimuli and as viewing time decreases, perception is increasingly biased by the preceding stimulus. The timecourse of the falloff is appropriate for the duration of human fixations and consistent with the distinct percepts across fixations in our daily experience. 

Figure 4 shows control data indicating the percentage of correct orientation discriminations that were made based on Gabor 2 tilt when Gabor 1 was not presented. As expected, performance increased with Gabor 2 time (e.g., black traces for 400 ms higher in plots than red traces for 50 ms). Panels A and B show variation between observers in the control experiments with real (Figure 4a) and simulated (Figure 4b) saccades. Three traces are shown at each masking time, corresponding to the three observers. Figure 4c compares discrimination in real (solid lines) and simulated (dashed lines) saccade conditions. The control data make two valuable points. First, though there are individual conditions at which performance is near chance, even at the shortest Gabor 2 time, orientation can often be discriminated above chance (not surprisingly, performance is closest to chance for the smallest tilts). In other words, observers could see the Gabor 2 stimuli with real and simulated saccades. Second, though there are differences, performance in real and simulated saccade conditions are similar. To the extent one can infer from the control data, it is not the case that the Gabor 2 stimuli are significantly more visible or discriminable in one condition or the other. 

Figure 4

 

Control experiments in which Gabor 2 orientation was discriminated but Gabor 1 was not presented. (a) Orientation discrimination in experiments with real saccades. At each of the four viewing times for Gabor 2, data are shown for three observers (color matched). (b) Orientation discrimination in experiments with simulated saccades. (c) Comparison of orientation discrimination in real and simulated saccade experiments where data is averaged across the three observers shown in Panels A and B. Solid lines show real saccade and dotted lines simulated saccade performance.

Figure 5

 

Effect of Gabor 1 and Gabor 2 orientation on the perception of Gabor 2 when a simulated saccade separates the two stimuli. The percentage of clockwise responses averaged across three observers at four Gabor 2 times (50, 100, 150, 400 ms) as a function of Gabor 2 (a) or Gabor 1 (b) orientation when a simulated saccade brings Gabor 2 into view. Error bars show the standard error of the mean. As with real saccades, there is a graded effect in which the orientation of Gabor 2 appears more similar to the actual orientation of Gabor 1 at short Gabor 2 times and progressively becomes more similar to the actual orientation of Gabor 2 at longer Gabor 2 times. The measure of bias is computed as with real saccades (illustrated here at the 50 ms Gabor 2 time).

To assess whether saccadic eye movements play a role in parsing visual signals on distinct fixations, we conducted a second experiment. The visual stimuli were identical to those used in the first experiment but instead of a saccade bringing Gabor 2 into the fovea, the eyes were stationary and Gabor 2 moved (Figure 2b). The simulated saccades were created by using the eye tracker to sample subjects' saccades and interpolated at the visual display's 160 Hz framerate. In this way simulated saccades were shown to fixating subjects; instead of the eyes moving, the stimuli moved with the accelerations and velocities of the real saccades. The key difference was the absence of actual eye movements in the simulated condition and consequently the absence of the associated corollary discharge and proprioceptive signals in the brain. 

Experiments conducted with the simulated saccades gave data that were qualitatively similar to the data obtained with real saccades. Figure 5 shows how the perception of tilt in Gabor 2 varied with the orientation of Gabor 2 (Figure 5a) and Gabor 1 (Figure 5b). Upon closer inspection there are differences in the slopes of the psychometric curves and more pronounced differences in the separation between the curves compared to the real saccade data in Figure 3. To quantify the influence of Gabor 1 on the perception of Gabor 2 in real and simulated saccade experiments, we calculated a measure of bias. Bias is defined, at any Gabor 2 time, as the average separation in Figures 3a and 4a between the curves representing the extreme orientations of Gabor 1 (± 8°). If the perceived tilt of Gabor 2 is based entirely on its orientation, bias should be zero. On the other hand, if the perception of Gabor 2 tilt were based entirely on the orientation of the preceding stimulus (Gabor 1), bias would be one. 

Bias is plotted in Figure 6 across the four Gabor 2 times. The gradual decrease in bias as Gabor 2 time increases shows that the first stimulus influences the perception of the second stimulus for a transient time period. On average across our three subjects, some trans-saccadic bias persists even with fixations of normal durations. Perhaps the most surprising aspect of the data in Figure 6 is the large difference in bias in real and simulated saccade experiments. At the shortest Gabor 2 time (50 ms), bias in the real saccade experiment is 0.65 but in the simulated saccade experiment bias is 0.93. In other words, with very short viewing times when a saccade is used to bring Gabor 2 into view, the perception of Gabor 2 is based on a mixture of the properties of Gabor 1 and Gabor 2. Strikingly, in the same situation, but without the saccade, the perception of Gabor 2 is based almost entirely on the orientation of the previous stimulus. 

Figure 6

 

Summary of perceptual bias measurements with real and simulated saccades averaged across three observers. At long Gabor 2 times corresponding to normal length fixations, there is no difference between bias measures with real and simulated saccades. At short Gabor 2 times, bias is much greater with simulated saccades suggesting that real saccades quickly decrease the influence of the preceding stimulus. Error bars show the standard error of the mean.

Discussion

Saccadic eye movements are a fundamental means by which humans explore the visual environment. However, saccades introduce two complementary challenges for vision. On the one hand, we have a remarkably stable sense of the visual world despite the sudden changes in visual input that saccades produce. It is common experience that moving one's eye by pushing on it appears to displace everything in view, but a similar eye movement resulting from a saccade leaves the world stationary. Consistent with the proposal made by von Helmholtz in the 19th century, a corollary discharge signal (i.e., a signal sent to visual cortex related to the motor command to move the eyes, Sperry, 1950; Von Holst & Mittelstaedt, 1950, 1971) may be used to internally compensate for eye movements and give us a sense of visual stability (Von Helmholtz, 1924; Wurtz, 2008). Physiological studies indicate that corollary discharge signals from the brainstem reach cerebral cortex very quickly, causing a burst of activity in the frontal eye fields in about 2 ms (Sommer & Wurtz, 2004a, 2004b). Such an effect on visual neurons may underlie receptive field remapping that occurs in some extrastriate areas prior to saccades (Colby, Duhamel, & Goldberg, 1993; Duhamel, Colby, & Goldberg, 1992; Melcher & Colby, 2008). While not yet proven, receptive field remapping via corollary discharge may be a neural mechanism of visual stability (Melcher & Colby, 2008; Sommer & Wurtz, 2008), though there are experimental and theoretical reasons for questioning the link between corollary discharge and space constancy (Bridgeman, 2007; Ilg, Bridgeman, & Hoffmann, 1989). Another important component of the visual stabilization process may be saccadic suppression, the reduction in visual sensitivity for objects seen during saccades (Holt, 1903; Matin, 1974; Ross et al., 2001; Wurtz, 2008). 

Resetting perception on each fixation

The complementary problem to visual stability, which is addressed here, is the distinct nature of percepts on each fixation. Rather than asking how we stitch the world together across saccades for the purpose of visual stability, we have investigated how we break vision apart into distinct percepts on each fixation. Vision is based on a continuous stream of neural activity spanning endless fixations and saccades and a mechanism is needed, of which little is known, to extract discrete percepts from the continuum. As discussed in the Introduction, it is not obvious that visual input alone reliably informs the visual cortex when a new fixation starts and stops. 

As thousands of experiments have demonstrated, it is certainly possible for vision to carry on without saccades when stimuli are flashed to a fixating animal or person. However, visual responses are markedly different in natural vision (Gallant, Connor, & Van Essen, 1998; Livingstone, Freeman, & Hubel, 1996; Ruiz & Paradiso, 2012) and it isn't clear if parsing of visual signals proceeds in the same manner in unnatural fixation experiments with flashed stimuli. With a saccade, visual input at every point in the visual field is different on the fixations before and after the saccade. On the other hand, in typical fixation experiments, there is generally a fixation period of 200–300 ms prior to stimulus presentation in which adaptation will occur; when the target stimulus is presented there is presumably a response transient to only the new stimulus. 

It is the case that very brief fixations sometimes occur in natural human vision (Hayhoe, Shrivastava, Mruczek, & Pelz, 2003). If perceptual bias occurs for a brief period of time at the start of each new fixation, this suggests that there might be imprecision in perception on brief fixations. To our knowledge this has not been studied, in contrast to many experiments on the perception of briefly flashed stimuli during longer fixations. To examine perceptual bias on brief fixations, it would be necessary to quantify perception (e.g., orientation estimation) in the context of visual input on the preceding fixation. 

The hypothesis explored here is that in natural vision, eye movements play an important role in resetting visual analysis on each fixation. The results show that perception is a graded process—there is a brief transition period at the beginning of each new fixation during which perception blends information from the old and new fixation. We do not experience such blended percepts in daily experience presumably because visual processing is not abnormally curtailed by masking stimuli. However, the implication is that the neural representation of the stimuli takes a finite time to transition from one percept to the next, presumably because perception is based on the integration of information over tens of milliseconds. With a fixation of normal duration (and no masking), the information would come from the image on the current fixation. However, for brief masked stimuli, the neural activity used to make a perceptual judgment may extend back into the brain's response to the previous stimulus. Most importantly, we find that when a saccade brings an image into view, the influence of the preceding stimulus is reduced much more quickly than when the visual input is the same but no actual eye movement occurs (i.e., simulated saccades). It is this difference between perceptual bias between fixations with real and simulated saccades that implies that the saccade is important for resetting perception. 

Alternative interpretations

An alternative to the interpretation made here is that perceptual bias across fixations results from the tilt aftereffect in which adaptation to a line at one orientation shifts the perceived orientation of a subsequent line (both lines presented in sequence during a single fixation) (Gibson & Radner, 1937). Melcher has shown that the tilt aftereffect is found even when the adaptation and test stimuli are separated by a saccade (Melcher, 2005, 2007, 2008). However, our experiment was structured to avoid adaptation and the bias we observed is incompatible with the tilt aftereffect in two critical ways. First, adaptation to a line tilted clockwise from vertical makes a subsequent line look more counterclockwise rather than clockwise as in our study (i.e., the direct tilt aftereffect that occurs at small angles similar to what we used is a repulsion rather than an attraction as we observe). Second, the duration of the Gabor 1 stimulus (250 ms) in our experiment is much shorter than the adaptation periods generally found to give a significant tilt aftereffect (Gibson & Radner, 1937; Greenlee & Magnussen, 1987). (In the Melcher studies examining trans-saccadic aftereffects, each trial involved adaptation for 3–5 s in contrast to the 250 ms of our first stimulus.) 

Besides the tilt aftereffect, one might speculate that some other form of aftereffect, adaptation, masking, or priming is responsible for the trans-saccadic orientation bias we observe. However, these explanations are unlikely to be correct because they don't account for the large performance difference in real and simulated saccade conditions despite similar retinal stimulation. Another possibility is that as the task became more difficult at shorter Gabor 2 times, the subjects unwittingly resorted to reporting the orientation of the first stimulus. One reason for thinking this is not the case is that in control experiments we found that subjects could discriminate the second Gabor stimulus above chance levels even at the shortest Gabor 2 time (Figure 4). But even aside from this observation, it isn't clear why there would be quite different levels of response bias in real and simulated saccade experiments that are similar in most every way. 

The significant difference between orientation bias measurements with real and simulated saccades suggests that a signal related to the real eye movement is involved. However, this interpretation hinges on the similarity of the trials with real and simulated saccades. Ultimately, it is impossible to completely match real and simulated saccades. Even with largely equivalent visual stimulation, motion in the simulated saccade condition is discretized by the computer display's refresh rate, and there are possible differences in the moment-to-moment allocation of attention. We have not found a straightforward account of the findings based on these differences, but it is conceivable that additional factors are involved. 

Possible mechanisms

The resetting of perception by eye movements is consistent with the finding that the transient states of unstable figures are disrupted by saccades (Ross & Ma-Wyatt, 2004). The lack of carry-over between fixations is also reminiscent of visual memory studies that show that little information is stored from one fixation to the next (Bridgeman & Mayer, 1983; Irwin, 1996; Rayner & Pollatsek, 1983). In seeking a neural correlate of perceptual reset, there is a daunting diversity of changes in neural activity at the start of fixations that is conceivably related. For example, in macaque LGN, saccadic and post-saccadic activity is complex, including both suppression and facilitation (with facilitation dominant) and a more rapid response to visual input (Buttner & Fuchs, 1973; Ramcharan, Gnadt, & Sherman, 2001; Reppas et al., 2002). In macaque area V1, Rajkai et al. (2008) found that when macaques make saccades in the dark there is a phase alignment of neural activity shortly after the start of each new fixation; they speculated that this may enhance responses to new stimuli (Rajkai et al., 2008). Maldonado et al. (2008) reported that 30–90 ms after the start of new fixations there is enhanced response synchronization to natural image stimuli (Maldonado et al., 2008). In macaque areas MT and MST, activity suppression during saccades is followed by enhanced responses to moving stimuli (Ibbotson, Price, Crowder, Ono, & Mustari, 2007). 

The large difference in orientation bias that we found between real and simulated saccades indicates that something about the eye movement is responsible for decreasing the influence of the preceding stimulus (i.e., resetting analysis on the new fixation). This process may be related to saccadic suppression, but there are notable differences. Rather than suppressed visual sensitivity to objects in view during saccades, our analysis is of interactions between objects seen on successive fixations. It has become increasingly clear over the years that the degree of suppression is dependent on multiple stimulus dimensions. Specifically relevant in the present experiments, it is important to note that there is little or no saccadic suppression at the spatial frequency (0.8 c/°) used in our study (Burr et al., 1994). Finally, the fact that we observe significant bias at short Gabor 2 times even with real saccades suggests that a stimulus on one fixation can influence perception on the next fixation despite some suppression of stimuli viewed during the saccade. Our data suggest that it takes a significant portion of each new fixation period before the influence of the previous stimulus is lost. 

It is not known by what neural mechanism(s) the visual system is able to simultaneously parse information from distinct fixations and create a sense of perceptual stability that spans saccades. Receptive field remapping driven by corollary discharge has been proposed as a mechanism that might underlie visual stability (Duhamel et al., 1992; Melcher & Colby, 2008; Wurtz, 2008), though recent studies have raised questions about the effect (Cavanagh, Hunt, Afraz, & Rolfs, 2010; Churan et al., 2011). In order to parse visual input, a reliable link between a saccade and the neural activity associated with a new fixation is desired. We propose that a corollary discharge signal may play a significant role in tightening this temporal relationship. Clearly, further physiological research will be needed to pin down this hypothesis and the connection to the suggestion by others that there are dual space representations, one disrupted by saccades and the other not (Burr, Morrone, & Ross, 2001; Ross & Ma-Wyatt, 2004). 

There are two routes by which eye movements might alter visual processing. One possibility is that feedback from proprioceptors in the extraocular muscles influences vision (Sherrington, 1918). Neural activity evoked by activation of proprioceptors has been recorded in Area 17 of anesthetized cats (Buisseret & Maffei, 1977) and in the nonhuman primate thalamus (Tanaka, 2007) and somatosensory cortex (Wang, Zhang, Cohen, & Goldberg, 2007; Xu, Wang, Peck, & Goldberg, 2011). However, the slow timecourse of proprioceptive feedback (Buisseret & Maffei, 1977; Wang et al., 2007; Wurtz, 2008) suggests that this is not the basis for the rapid reset that occurs just after a saccade. The alternative to proprioception is corollary discharge, a copy of the motor command used to move the eye. Neural activity correlated with saccades has been recorded in many brain areas with a variety of techniques (e.g., Bodis-Wollner et al., 2002; Green, 1957; Purpura et al., 2003) and there are numerous paths by which corollary discharge information may reach the cerebral cortex (Sommer & Wurtz, 2004a, 2004b, 2008b) Recently, one circuit for corollary discharge has been demonstrated and there is significant evidence that corollary discharge rather than proprioception provides eye position signals used in planning saccades (Richmond & Wurtz, 1980; Sommer & Wurtz, 2008; Wurtz, 2008). We hypothesize that visual cortical areas responsible for perception use corollary discharge as an indicator of new fixations so that the temporal continuum of neural activity can be parsed into perceptual epochs associated with distinct fixations. The fact that bias in our simulated saccade experiments decreases at longer Gabor 2 times shows that parsing can occur without saccades, a point consistent with experiments using flashed stimuli. However, our data suggest that parsing visual signals is faster when the eyes move, presumably an increase in efficiency that takes advantage of the corollary discharge signals. Because perception must lead to behavioral decisions in very short periods of time, rapid responses in the visual system are highly significant (Stanford et al., 2010; Thorpe et al., 1996). 

Conclusions

An orientation discrimination task was used to investigate a possible role of saccades in the resetting of visual analysis on each fixation. Subjects saw two tilted Gabor stimuli separated by a saccade and discriminated the orientation of the second. The results show that for a short period of time the tilt of the second stimulus is significantly influenced by the tilt of the first. This indicates that perception is not immediately reset on each new fixation. A second experiment was conducted in which visual stimulation was nearly identical to the saccade experiment, but subjects held fixation while the saccade was simulated. The degree to which the first stimulus biased the perceived tilt of the second stimulus was much greater with stimulated saccades. We propose that in the natural situation with saccades there is less trans-saccadic bias because the eye movement is being used to reset analysis on the new fixation. The most likely candidate for the signal involved in resetting perception is a corollary discharge related to the saccade. 

Acknowledgments

We thank Aaron Gregoire, Theresa Lii, and David Freestone for technical assistance. Dar Meshi was supported by a NIMH postdoctoral training grant (T32-MH019118) and an award from Brown University's Center for Vision Research. 

Commercial relationships: none. 

Corresponding author: Michael A. Paradiso. 

Email: [e-mail]. 

Address: Department of Neuroscience, Brown University, Providence, RI, USA.