Contributions of fixational eye movements to the discrimination of briefly presented stimuli

Abstract

Although it is known that images tend to disappear when they are stabilized on the retina for tens of seconds or minutes, the possible functions of fixational movements during the brief periods of visual fixation that occur during natural viewing remain controversial. Studies that investigated the retinal stabilization of stimuli presented for less than a few seconds have observed neither decrement in contrast sensitivity nor image fading. In this study, we analyzed the effect of retinal stabilization on discriminating the orientation of a low-contrast and noisy small bar that was displayed for either 500 ms or 2 s. The bar was randomly tilted by 45° either clockwise or counterclockwise. For both exposure durations, percentages of correct discrimination were significantly lower under conditions of visual stabilization than in the presence of the normally moving retinal image. These results are consistent with the predictions of recent computational models that simulated neuronal responses in the early visual system during oculomotor activity and support the hypothesis that visual processes deteriorate rapidly in the absence of retinal image motion.

Introduction

Our eyes are never at rest. During the periods of visual fixation, small eye movements continuously move the projection of the image on the retina. These fixational eye movements include small saccades, slow drifts, and physiological nystagmus, a high-frequency tremor with amplitude smaller than 1’ (Ratliff & Riggs, 1950; Ditchburn, 1955; Steinman, Haddad, Skavenski, & Wyman, 1973) (see Figure 1). Under natural viewing conditions, when the head is not immobilized by a chin rest, movements of the head and body combine with eye movements to further amplify the jittering of the images on the retina (Murakami & Cavanagh, 1998). It is remarkable that the visual system is able to create a steady percept of the visual scene despite such a high degree of variability in the input signals. 

Figure 1

 

An example of macroscopic and microscopic eye movements. A recorded trace of eye movements is shown superimposed on the original image. The panel on the bottom right shows a zoomed portion of the trace in which small fixational eye movements are present. The color of the trace represents the velocity of eye movements (red: slow movements; yellow: fast movements). Blue segments mark periods of blink.

A number of studies have investigated vision under stabilized conditions, in which fixational instability is eliminated by constraining the head and moving the visual scene to compensate for the movement of the eye (Ditchburn & Ginsborg, 1952; Riggs & Ratliff, 1952; Yarbus, 1967). These studies have shown that in the absence of retinal motion, images tend to fade away over a period of several seconds and minutes. Although controversies remain on whether perfectly stabilized images disappear completely (Arend & Timberlake, 1986; Ditchburn, 1987), it is clear that contrast sensitivity is reduced in stabilized conditions with long stimulus presentations, especially at low spatial frequencies (Koenderink, 1972; Kelly, 1979; Tulunay-Keesey, 1982). 

To minimize the motion of the image on the retina, most experiments on stabilized vision have focused on conditions of sustained fixation, in which stimuli were presented for long periods of time. Few studies have examined the consequence of eliminating fixational instability during brief stimulus presentations similar to those that occur during natural viewing conditions. Experiments on visual acuity and contrast sensitivity have found either no significant effect of image stabilization (Keesey, 1960; Tulunay-Keesey & Jones, 1976) or an improvement of performances under stabilized conditions with brief stimulus exposures (Riggs, Ratliff, Cornsweet, & Cornsweet, 1953). 

The results of more recent psychophysical, neurophysiological, and computational studies, however, argue in favor of an important role of fixational instability even during the brief periods of visual fixation. For example, it has been observed that entoptic images generated by casting shadows of the foveal capillaries onto the retina tend to disappear in less than 80 ms (Coppola & Purves, 1996), an interval that correlates well with the rapid decay of neuronal responses in the monkey’s primary visual cortex (V1) with unchanging stimuli (Ringach, Hawken, & Shapley, 1997; Mazer, Vinje, McDermott, Schiller, & Gallant, 2002). Neurophysiological recordings with awake monkeys have shown that fixational eye movements strongly modulate the responses of neurons in different cortical areas (Gur, Beylin, & Snodderly, 1997; Leopold & Logothetis, 1998; Martinez-Conde, Macknik, & Hubel, 2000). In particular, in area V1 of the macaque, different populations of neurons respond selectively to the two main components of fixational eye movements: small saccades and drifts (Snodderly, Kagan, & Gur, 2001). Furthermore, large-scale computer simulations of neuronal responses in the early visual system during oculomotor activity suggest that fixational instability profoundly alters the structure of correlated activity. In these simulations, fixational eye movements induced synchronous modulations of neuronal responses in the lateral geniculate nucleus (LGN) (Rucci, Edelman, & Wray, 2000) that were detected and amplified by neurons in the primary visual cortex (Rucci & Casile, 2003). 

This body of literature raises the hypothesis that previous experiments on stabilized vision, with their focus on evaluating visual acuity or contrast sensitivity, may have not explored conditions adequate to unveil possible effects of fixational instability in the presence of brief stimulus presentations. In this work, we report the results of a forced-choice discrimination task that was designed on the basis of our recent modeling work to enhance the effect of fixational instability on the structure of correlated activity. We show that in this task subjects perform differently under stabilized and unstabilized conditions, even when stimuli were presented for only 500 ms. 

Methods

Subjects

Four subjects with normal vision participated in the experiments. Three subjects were naive about the purposes of the experiments and were paid to participate. A fourth subject was one of the authors. Informed consent was obtained from all subjects following the procedures approved by the Boston University Charles River Campus Institutional Review Board. 

Apparatus

Stimuli were generated on a Millenium G550 graphics card (Matrox Graphics Inc., Dorval, Quebec, Canada) and displayed on a 21″ Trinitron CRT at a resolution of 800 × 600 pixels and vertical refresh rate of 75 Hz. Subjects were kept at a fixed distance of 110 cm from the monitor by means of a dental imprint bite bar and a head rest that prevented movements of the head. 

Eye movements were monitored and recorded by a Generation 6 Dual-Purkinje-Image (DPI) eyetracker (Fourward Technologies Inc., Buena Vista, VA) originally designed by Crane and Steele (1978). The nominal resolution of this eyetracker is about 20″ with a time delay of approximately 0.25 ms (Crane & Steele, 1985). Vertical and horizontal eye position data were sampled at 1 kHz, digitally low-pass filtered (Butterworth filter with 100 Hz cutoff frequency), and recorded for subsequent analysis. To determine the synchronism between traces of eye movements and the stimulus, a small square was periodically flashed at one of the corners of the screen, and the voltage of a photocell covering the square was simultaneously sampled and recorded. Subjects gave their responses by pressing one of two keys on a joypad. 

Image stabilization was maintained by a stimulus deflector coupled to the DPI eyetracker (Crane & Clark, 1978). When properly calibrated, this optical-electronic device shifts the image in the opposite direction and by the same amount as the eye movements with a total response time of approximately 6 ms and a spatial resolution of approximately 10”. Stimuli were always viewed through the deflector in both stabilized and unstabilized conditions. 

Stimuli

Visual stimuli were designed on the basis of our recent modeling work to enhance the possible impact of fixational eye movements in visual discrimination. Each stimulus consisted of a 30 × 4 pixel light gray bar (approximately 30′ of visual angle) embedded in a 42 × 42 pixel gray square. In each trial, the bar was tilted by +45° or −45° with equal probability (angles are measured counterclockwise from the vertical axis). Results from our simulations have indicated that modulations in neuronal responses due to small eye movements may become particularly relevant in the presence of noisy stimuli. Noise was added to the stimulus according to the following algorithm: Each pixel of the square matrix had a fixed probability (the noise density) of being affected by noise. The intensity values of noisy pixels were replaced with random values selected from a uniform distribution between 0 and 255. In all experiments, a noise density of 80% was used. Stimuli were displayed on a gray background of uniform luminance equal to the mean luminance of the stimulus (22.2 cd/m²). Figure 2 shows typical examples of the stimuli. Contrast levels were individually adjusted for each subject so that performances in the presence of the normally moving retinal image were around 70–80% (see “Procedure”). Different contrasts were used in Experiment 1 and Experiment 2. Contrast levels ranged from 2.0% to 4.3% in Experiment 1 and from 3.7% to 4.9% in Experiment 2. The screen background was kept isoluminant at a fixed value of 22.2 cd/m². It should be noted that the luminance experienced by subjects was attenuated by the stimulus deflector. 

Figure 2

 

Examples of stimuli used in the experiments. Left: +45° bar. Right: −45° bar.

Procedure

Each subject participated in several experimental sessions of approximately 30 min each. Each experimental session started with preliminary setup operations that lasted between 10 and 15 min and allowed the subject to adapt to the low level of light in the room. These preliminary operations included positioning the subject optimally and comfortably in the apparatus; tuning the eyetracker until successful tracking; calibrating the stimulus deflector until successful stabilization (see below); and running a brief procedure that allowed conversion of the eyetracker output voltages into degrees of visual angle. This conversion was achieved by performing a quadratic regression on the basis of nine points for which spatial positions and output voltages were known. 

Calibration of the stimulus deflector, a critical step for achieving accurate stabilization, was performed following the afterimage method described by Kelly (1979). In this procedure, the subject is responsible for adjusting the offsets and gains of the deflector. These parameters depend on several factors, including the morphology of the eye. In a preliminary coarse calibration, subjects adjusted the deflector settings by comparing the movement of a stabilized dot to crosshair landmarks that were displayed in the unstabilized field. In the successive fine-tuning phase, a small, bright bar was displayed on a dark background. Subjects used the negative afterimage that developed from fixating the bar for 30 s to refine stabilization while performing a number of small saccades along the horizontal and vertical axes. After a saccade, the displayed bar moved farther than its dark afterimage if the gain was set too high, whereas it moved short of the afterimage if the gain was set too low. Using a vernier potentiometer, the subject finely adjusted the gains so that the afterimage always remained hidden behind the bar during eye movements. 

After these preliminary setup operations, subjects were presented with blocks of 25 experimental trials. A brief break between two consecutive blocks allowed the subjects to relax and occasionally check the accuracy of stabilization by repeating the calibration routine. Overall, subjects were never constrained in the experimental setup for more than 30 min in a row. 

In the experimental trials, subjects reported in a forced-choice procedure whether the stimulus bar was tilted by +45° or −45°. Three experiments were run. Stimuli were presented for 2 s in Experiment 1 and for 500 ms in Experiments 2 and 3. In all experiments, blocks of 25 trials alternated between two different conditions: stabilized and unstabilized. As shown in Figure 3, the temporal sequence of events in an unstabilized trial consisted in (a) presenting an initial fixation dot at the center of the screen for 1.57 s; (b) at the offset of the fixation dot, cueing the location of stimulus. Cueing was performed by four arc segments that surrounded the chosen position for 240 ms; (c) after an interval of 240 ms, displaying the stimulus at the cued location for a fixed duration of either 2 s (Experiment 1) or 500 ms (Experiments 2 and 3); (d) masking the stimulus by a high-energy mask that was displayed for 1.33 s. In the unstabilized trials of Experiments 1 and 2, to allow the normal fixational instability that occurs after saccades in natural viewing conditions, the stimulus was presented at a fixed distance from the fixation dot and the subject was required to make a saccade toward it. Stimuli were displayed 240 ms after the cue to ensure that subjects would fixate the cued location at the time of stimulus appearance. This interval was selected on the basis of preliminary experiments that evaluated subject saccadic and reaction times. In the control experiment (Experiment 3), stimuli were cued and displayed at the center of fixation. 

Figure 3

 

Main experimental procedure (Experiments 1 and 2). Subjects were required to detect the orientation of a noisy bar that was displayed for either 2 s (Experiment 1) or 500 ms (Experiment 2). Performances in two conditions, stabilized and unstabilized (i.e., with the normally moving retinal image), were compared. Left: unstabilized trials. Right: stabilized trials. In the control experiment (Experiment 3), the procedure was identical in both stabilized and unstabilized trials; the cue and stimulus were always presented at the center of fixation.

In the stabilized condition, the stimulus deflector eliminated retinal image motion by compensating for the subject eye movements. In this case, the stimulus always appeared immobile at the center of the fovea. As illustrated in the right panel of Figure 3, the sequence of events in the stabilized condition was similar to that of the unstabilized condition. The main differences were the absence of the saccade (which, in this case, would have disrupted retinal stabilization) and, in Experiments 1 and 2, the absence of the saccade cue, which was eliminated to avoid the annoying afterimages that develop with repetitive presentation of identical visual stimulation in the same retinal location. In Experiment 3, stabilized and unstabilized conditions were identical and included a lower intensity cue at the center of fixation (and thus no saccade). 

To compensate for individual differences in contrast sensitivity, contrast levels for each subject were determined in a preliminary session, in which we systematically varied stimulus contrast. Contrast levels were chosen so that percentages of correct discrimination were between 70% and 80% correct in the unstabilized condition. 

Contrast sensitivity functions were also determined for each subject in three different conditions: stabilized with a 2-s presentation; stabilized with a 500-ms presentation; and unstabilized with a 500-ms presentation. Consistent with previous reports (Tulunay-Keesey & Jones, 1976), visual stabilization had little or no effect on the sensitivity thresholds obtained with these brief stimulus durations. The contrast sensitivity functions measured under visual stabilization were similar to those obtained in unstabilized conditions, and the curves measured with stabilized exposures of 2 s and 500 ms were almost identical to each other. An example of contrast sensitivity functions measured for one subject in the various conditions is given in Figure 4. 

Figure 4

 

Contrast sensitivity functions for GD, measured in different experimental conditions.

Results

Experiment 1

In the first experiment, we examined the effect of visual stabilization with a stimulus presentation of 2 s. Although longer than the typical durations of visual fixation, a 2-s period of observation provides a good reference point for experiments with shorter durations, as it allows a relatively long interval for fixational instability to exert its possible influence on neural activity. Figure 5 shows the percentages of correct discrimination obtained in Experiment 1. The individual subject data as well as their overall means are shown in separate graphs. Each graph compares performances in the stabilized and unstabilized conditions. In the unstabilized condition, following the preliminary contrast selection procedure (see “Methods”), percentages of correct discrimination were 72% for BE (N=164), 69% for TC (N=97), and 76% for GD (N=74). The mean percentage of correct discrimination over all subjects was 72%. Percentages of correct discrimination dropped to chance level for all subjects when the image was stabilized on the retina. Under stabilized conditions, percentages of correct discrimination were 51% for BE (N=82), 53% for TC (N=51), and 47% for GD (N=59). In this case, the mean percentage of correct discrimination over all subjects was 51%. One-tail z tests of the differences in the percentages of correct discrimination under stabilized and unstabilized conditions were all significant at the .05 levels. (BE: z=3.21, p < .05; TC: z=1.94, p <.05; GD: z=3.35, p < .05). 

Figure 5

 

Percentages of correct discrimination obtained in Experiment 1 when the stimulus was presented for 2 s. The results for each subject as well as the overall means are shown. The two bars illustrate the results obtained under stabilized and unstabilized conditions. Error bars at the .05 significance levels are shown.

Experiment 2

While substantially shorter than the durations of stimulus presentation used by most previous experiments with stabilized vision, 2 s is still a long interval compared to the periods of visual fixation that occur during natural viewing conditions. To investigate whether a similar impairment in visual discrimination is also present with shorter exposures, in Experiment 2 the stimulus exposure was reduced to 500 ms. A period of approximately 500 ms has been reported as the average duration of visual fixation for free-viewing of simple patterns stimuli similar to the ones used in our experiments (Harris, Hainline, Abramov, Lemerise, & Camenzuli, 1988; Andrews & Coppola, 1999). 

Figure 6 shows the percentages of correct discrimination obtained with this shorter stimulus duration. As in Figure 5, performances in the stabilized and unstabilized conditions are compared in different graphs for the three subjects. In the unstabilized condition, percentages of correct discrimination were 72% for BE (N=249), 80% for TC (N=143), and 83% for GD (N=98). The mean percentage of correct discrimination over all subjects was 76%. Similar to Experiment 1, lower percentages of correct discrimination were found for all subjects when the image was stabilized on the retina. In the stabilized condition, percentages of correct discrimination with 500-ms exposure duration were 62% for BE (N=221), 66% for TC (N=119), and 66% for GD (N=80). The mean percentage of correct discrimination over all subjects was 64%. One-tail z tests of the differences in the percentages of correct discrimination under stabilized and unstabilized conditions were all significant at the .05 levels (BE: z=2.18, p < .05; TC: z=2.44, p < .05; GD: z=2.52, p < .05). It is interesting that in the unstabilized condition all subjects required higher levels of stimulus contrast to produce levels of performance similar to those obtained in Experiment 1. Because corresponding reductions in contrast sensitivity thresholds were not observed when measuring contrast sensitivity functions (see Figure 4), it appears that this impairment occurred specifically in the discrimination experiments. It should also be noted that although direct comparison of Figures 5 and 6 suggests that percentages of correct discrimination were more severely affected by image stabilization in Experiment 1 (with a 2-s stimulus exposure) than in Experiment 2 (with a 500-ms stimulus exposure), a quantitative analysis is complicated by the fact that contrast levels could not be finely tuned to exactly match the percentages of correct discrimination in the unstabilized conditions of the two experiments. 

Figure 6

 

Percentages of correct discrimination obtained in Experiment 2 when the stimulus was presented for 0.5 s. The results for each subject as well as the overall means are shown. The two bars illustrate the results obtained under stabilized and unstabilized visual conditions. Error bars at the .05 significance levels are shown.

Previous experiments on stabilized vision have reported a reduction in contrast sensitivity with prolonged exposure to stabilized stimuli. With the brief stimulus presentations of our experiments, contrast sensitivity functions measured in the absence or presence of retinal image motion (i.e., stabilized or unstabilized conditions) produced similar thresholds (e.g., see Figure 4). Nevertheless, it is still possible that some degree of image fading occurred due to the continuous presence of a uniform background during and in between trials. In Experiment 1, subjects occasionally reported a partial fading of the image toward the end of a block of trials (the trials in which this occurred were removed from data analysis). In Experiment 2, image fading was never experienced. Nonetheless, to test if a decrement in contrast sensitivity could account for the impairment in discrimination performances under stabilized conditions, data were analyzed to distinguish early trials (the first 10 trials in each block of 25 consecutive trials) from late trials (the last 10 trials in each block). The results of this analysis are shown in Figure 7. In the stabilized condition, only TC exhibited slightly better performances in the first part of a block of trials. Percentages of correct discrimination for TC were 73% in the first 10 trials and 66% in the last 10 trials. However, this difference was well within the range of statistical variability (z = 0.95, p > .05). The other subjects performed almost identically in early and late trials. Percentages of correct discrimination in the stabilized condition were 62% in the first 10 trials and 62% in the last 10 trials for BE, and 68% in the first 10 trials and 66% in the last 10 trials for GD. For all subjects, no statistically significant differences were found between early and late trials, neither in the unstabilized nor in the stabilized conditions. Thus, the reduction in percentages of correct discrimination measured in Experiment 2 was not due to a corresponding long-term fading of the image. 

Figure 7

 

Percentages of correct discrimination in early and late trials of Experiment 2. The results for each subject as well as the overall means are shown. Percentages of correct discrimination obtained in the first 10 trials of each block of 25 consecutive trials are compared to those obtained in the last 10 trials. Bars of different intensity illustrate the results obtained under stabilized and unstabilized conditions. Error bars at the .05 significance levels are shown.

Experiment 3

In the previous experiments, unstabilized and stabilized trials differed not only in retinal image motion but also in the procedure of stimulus presentation. To allow the normal instability of visual fixation, in the unstabilized trials, subjects performed a saccade toward a cued location at which the stimulus was presented. In contrast, to ensure a high quality of retinal stabilization, in the stabilized condition, stimuli were presented at the center of the screen while the subject maintained fixation. 

To examine the possible influence of this procedural difference, in Experiment 3, we matched the conditions of stimulus presentation in stabilized and unstabilized trials. In this control experiment, the stimulus was always preceded by a cue (also in stabilized trials) and presented at the center of fixation in both stabilized and unstabilized trials. 

It is known that under conditions of sustained fixation subjects tend to show a lower degree of fixational instability than in natural viewing conditions (Steinman, Cunitz, Timberlake, & Herman, 1967; Kapoula, Robinson, & Hain, 1986). In Experiment 3, two subjects (GD and TC) exhibited a clear reduction of fixational instability, whereas the third subject (AS) remained at approximately the same level. While for GD, fixational instability decreased of a factor of two with respect to Experiment 2 (1.7 arc min² vs. 3.4 arc min², in correct trials), TC exhibited a more pronounced reduction to about ¼, as shown in Figure 10 (2.5 arc min² vs. 9.0 arc min², in correct trials). 

Figure 10

 

Mean area covered by fixational eye movements in unstabilized trials as a function of the subject response (correct or incorrect) for GD and TC, who took part in all three experiments.

Figure 8 shows subject performances in Experiment 3. The two subjects with little or no reduction in fixational instability maintained a statistically significant difference between stabilized and unstabilized conditions. Percentages of correct discrimination dropped from 75% (N=177) in the unstabilized condition to 68% (N=162) in the stabilized condition for GD, and from 76% (N=148) to 67% (N=98) for AS. Both differences were significant at the .05 levels according to one-tail z tests (AS: =1.87, p < .05; GD: z=1.68, p < .05). In contrast, TC, who exhibited a substantial reduction in fixational instability, performed at an equally low level in both conditions: 66% (N=95) in the unstabilized condition and 64% (N=91) in the stabilized condition (z=0.25, p > .05). 

Figure 8

 

Percentages of correct discrimination obtained in Experiment 3. The results for each subject as well as the overall means are shown. The two bars illustrate the results obtained under stabilized and unstabilized visual conditions. Error bars at the .05 significance levels are shown.

Thus, the difference in performance that was found between unstabilized and stabilized conditions in Experiments 1 and 2 was still visible under sustained fixation in Experiment 3, for the two subjects who maintained a substantial degree of fixational instability. 

Oculomotor Activity

During free-viewing, when the head is not restrained and movements of the eye combine with movements of the head and body, a considerable degree of retinal image motion occurs during the periods of visual fixation. In our main experiments (Experiments 1 and 2), to approximate free-viewing conditions and enhance fixational instability, stimulus presentation was preceded by a saccade. Table 1 summarizes oculomotor activity in the unstabilized condition (i.e., when subjects were free to move their eyes on the stimulus) of Experiments 1 and 2. To estimate fixational instability during a trial, we measured the spatial amplitude of drift periods and the amplitude of fixational saccades. Drift amplitude was defined as the length of the segment joining the initial and final positions of the eye during a fixation. Saccades were detected by a velocity threshold of 10 deg/s. Although individual differences were present, all subjects exhibited a significant degree of fixational instability with both drift and saccades: On average, the amplitude of drift was 7.7′ in Experiment 1 and 4.5′ in Experiment 2. The mean amplitude of saccades was 27′ in both Experiments 1 and 2. Not surprisingly, subjects executed more saccades during the longer stimulus presentation of Experiment 1 (2 s) than in Experiment 2 (500 ms). The mean number of saccades was 3.0 in Experiment 1 and 0.2 in Experiment 2. 

Table 1

 

Oculomotor Activity in the Unstabilized Conditions of Experiments 1 and 2.

Experiment 1 (2 s)	BE	TC	GD	Means
Average number of saccades	4.5	1.9	2.5	3.0
Average saccade amplitude	24.8	38.3	17.8	27.0
Average drift amplitude	8.7	7.8	6.5	7.7
Experiment 2 (500 ms)	BE	TC	GD	Means
Average number of saccades	0.3	0.1	0.3	0.2
Average saccade amplitude	24.8	39.6	17.9	27.4
Average drift amplitude	5.3	4.9	3.4	4.5

Although fixational saccades appeared to improve performance, they were not sufficient to account for the difference between percentages of correct discrimination in the stabilized and unstabilized conditions. In Figure 9, the unstabilized trials were classified depending on whether or not they included a fixational saccade. The presence of a fixational saccade clearly enhanced performance for all three subjects. However, a significant difference was still present between stabilized trials and unstabilized trials in which no saccade occurred. 

Figure 9

 

Percentages of correct discrimination in unstabilized trials that included one fixational saccade (left) and no fixational saccade (center), and in stabilized trials (right).

To estimate the overall spatial extent of fixational instability, we also evaluated the area of the rectangle defined by the SDs of the vertical and horizontal components of the eye position. Figure 10 shows the average area covered by fixational instability for GD and TC, the two subjects who participated in all three experiments. The mean area covered by fixational eye movements was 15 arc min² in Experiment 1 and 2.8 arc min² in Experiment 2 for GD, and 76 arc min² in Experiment 1 and 7 arc min² in Experiment 2 for TC. In Figure 10, trials are sorted according to the subject’s response (i.e., whether the subject correctly or incorrectly reported the orientation of the target). Not surprisingly, the spatial span of fixational eye movements was much larger in Experiment 1 (with 2-s trials) than in Experiments 2 and 3 (with 500-ms trials). More interestingly, the average span of fixation was larger in correct trials than in incorrect trials. The only exception was TC in Experiment 3, the only case among all subjects and experiments in which the difference between percentages of correct discrimination in stabilized and unstabilized conditions was not statistically significant. A larger fixational instability in successful trials is consistent with the hypothesis of a contribution of fixational eye movements in visual discrimination. 

Discussion

During visual fixation, small movements of the eyes and the head keep the projections of the scene onto the retina in constant motion. It is unclear whether this fixational instability serves a useful purpose in natural viewing conditions, when fixation typically lasts a few hundreds milliseconds. By showing an impairment in visual discrimination of briefly presented stimuli, the results of our experiments are consistent with the hypothesis that the motion of the image on the retina plays a role in refreshing, and possibly structuring, neural activity during the brief periods of visual fixation. 

Stabilization of Briefly Presented Stimuli

The analysis of visual performances in the presence of stabilized retinal images dates back more than half a century. Initial studies were stimulated by dynamic theories of visual acuity, which argued for a role of the motion of the eye in hyperacuity (Averil & Weymouth, 1925; Marshall & Talbot, 1942). These theories were disproved by the first pioneering efforts in eliminating fixational eye movements (Riggs et al., 1953; Keesey, 1960). Interest in stabilized vision was then renewed by the discovery that images tend to fade away in the absence of motion on the retina. While many experiments have investigated image fading when stimuli are presented for various durations ranging from tens of seconds to minutes (Riggs & Ratliff, 1952; Ditchburn & Ginsborg, 1952; Barlow, 1963; Evans, 1965; Gerrits, de Haan, & Vendrik, 1966; Yarbus, 1967; Keesey, 1969; Koenderink, 1972; Kelly, 1979), few studies have considered fixational instability within the context of the brief interval of natural visual fixation. 

Previous experiments on stabilization of briefly presented stimuli have found no significant differences with respect to the case in which normal retinal motion occurs. Indeed, it has been reported that acuity for such diverse targets as black lines, vernier displacements, and gratings improves as a function of exposure duration in a similar way for the stabilized and the normally moving retinal image, and similar absolute values of acuity were obtained in both cases (Keesey, 1960). Furthermore, similar thresholds were found for detecting stabilized and unstabilized gratings with exposure duration ranging from a few milliseconds to 4 s (Tulunay-Keesey & Jones, 1976). These previous findings are in agreement with our measurements of contrast sensitivity functions, which produced very similar thresholds with stimulus exposures of 500 ms and 2 s in stabilized and unstabilized conditions. However, they are in sharp contrast with the results of our discrimination experiments in which the absence of retinal image motion significantly impaired subject performances. This different effect of retinal stabilization in contrast sensitivity and orientation discrimination may originate from a number of sources: The experiments differed not only in the task but also in the stimuli and procedure. 

A first important difference was the level of stimulus noise. Whereas contrast sensitivity was measured by means of noise-free gratings at different spatial frequencies, a high level of noise corrupted the stimuli of our discrimination experiments. As described in the second part of the “Discussion,” stimulus noise may enhance the impact of modulations of neural responses due to fixational eye movements. This is consistent with the results of some preliminary experiments with different noise densities (data not shown), which showed a more significant impact of stabilization with higher levels of noise. 

A second important difference was the degree of fixational instability. In previous studies (Keesey, 1960; Tulunay-Keesey & Jones, 1976), subjects maintained fixation during presentation of gratings. It is known that eye movements are reduced in this condition of sustained fixation (Steinman et al., 1967; Kapoula et al., 1986). It is possible that a reduction in the amount of fixational instability attenuated the difference between stabilized and unstabilized contrast sensitivity thresholds. In our main experiments (Experiments 1 and 2), to approximate the fixational instability that occurs during natural viewing, subjects were required to perform a saccade toward a cued location in the unstabilized trials. 

Despite the brief durations of stimulus presentation, the retinal image moved considerably in the unstabilized trials. Both small saccades and drifts contributed to move the stimulus. All subjects made small saccades in both Experiment 1 and 2. No systematic relationship could be seen between subject performances and the shifts in fixation point operated by these saccades. Given the small size of the bar, which could be comfortably seen with a single fixation, it is unlikely that saccades were used to redirect the fovea to different regions of the stimulus. Similarly, the opposite hypothesis (i.e., that saccades were performed in an attempt to move the fovea away from the stimulus, thus low-pass filtering the image by means of the lower resolution of the visual periphery) is also unlikely given the small size of saccades. We did not attempt to distinguish between possible contributions of different types of small eye movements for two main reasons. First, during natural vision, eye movements combine with other movements of the head and body, and it becomes difficult to extrapolate data obtained with a constrained head to more natural viewing conditions. Second, the results of our computer simulations suggest that if a sufficient degree of retinal image motion is present, different types of eye movement have a similar effect on the second-order structure of neural activity, as long as they occur within a spatial window of comparable size. 

As in any study involving retinal stabilization, one may question the accuracy with which retinal motion was eliminated. In our experiments, particular care was taken in eliminating movements of the head, calibrating the stimulus deflector individually for each subject, and minimizing possible sources of noise. Image stabilization was achieved by means of a stimulus deflector directly coupled to the DPI eyetracker, a device with a response time of 6 ms and spatial resolution of 10″ (Crane & Clark, 1978). While perfect retinal stabilization of exoptic images is not possible, both the disappearance of afterimages during the calibration phase and the different levels of subject performance measured in the stabilized and unstabilized conditions argue for a high quality of retinal stabilization. 

Predictions From Neural Modeling

The discrimination experiment described in this work was designed on the basis of our recent computational work on modeling neuronal responses during oculomotor activity. Neurophysiological investigations of the visual cortex of the macaque, a species with visual and oculomotor characteristics similar to those of humans, have shown that neurons in the striate cortex respond to small changes in the visual signals produced by fixational saccades and ocular drift (Gur et al., 1997; Leopold & Logothetis, 1998; Martinez-Conde et al., 2000; Snodderly et al., 2001). Our simulations of LGN and V1 neuronal responses during eye movements indicate that the jittering of the image on the retina contributes to shaping the second-order statistics of thalamic and thalamo-cortical neural activity during visual fixation (Rucci et al., 2000; Rucci & Casile, 2003). The second-order statistical structure of neural activity acquires particular importance in the light of theories that propose a role for synchronous modulations of neural activity in transmitting visual information (Singer & Gray, 1995; Singer, 1999). 

Figure 11 illustrates the putative effect of fixational instability on the statistical structure of neural activity for the task considered in this work. The receptive fields of three geniculate cells are shown: Cells A and B are centered on the stimulus bar (a −45° bar), while the receptive field of cell C is located on the background in a position that would be covered by a bar of opposite orientation (a +45° bar). Due to the noise in the stimulus pattern, it is assumed that cells A and C possess a high mean level of activity, whereas cell B settles on a lower level. During visual fixation, the motion of the retinal image modulates the activity of the three cells in different ways. Fixational modulations tend to be synchronous in cells A and B that are activated by the stimulus bar, and occur with independent temporal dynamics in cell C. The results of our modeling work suggest that these short-lived modulations of thalamic responses are effective in activating cortical cells and may thus help disambiguate confusing input signals. For the stimulus configuration of Figure 11, the emerging prediction is that subjects are more likely to report the presence of a −45° bar (covering the receptive fields of cells A and B) under normal visual conditions, and a +45° bar (covering the receptive fields of cells A and C) under stabilized conditions when, in the absence of fixational modulations, the mean levels of neural responses have a stronger influence on the activity of postsynaptic neurons. 

Figure 11

 

The rationale leading to the design of the experiments. The receptive fields of three ON-center geniculate cells are shown overlapped on the bar stimulus. The graphs illustrate the response of each cell during a stabilized and an unstabilized trial. The table shows the levels of correlation and covariance of activity.

Mathematically, the prediction of our model can be characterized in terms of a different influence of the levels of correlation (the mean of the product of two signals) and covariance (the mean of the product of two signals with their averages removed) of cell responses. Due to the high average level of activity, cells A and C are more strongly correlated than cells A and B. Cells A and B, however, possess a higher level of covariance during the normal jittering of visual fixation, because their responses tend to be modulated synchronously by fixational eye movements. Our model predicts that responses of neurons at later stages in the visual hierarchy are more strongly affected by patterns of correlated activity during stabilized vision and by levels of covariance during normal vision. 

Both the impairment in visual discrimination observed under retinal stabilization and the higher degree of fixational instability observed in successful trials are consistent with the hypothesis that fixational instability plays a role in structuring neural activity during the brief periods of visual fixation. In addition, a comparison of the results of Experiments 1 and 2 suggests that the difference in performance between stabilized and unstabilized conditions is larger with a 2-s stimulus duration. This is also consistent with the predictions of the model that a longer presentation of the stimulus should result in an improvement in discrimination performances by allowing both a more prolonged period for affecting the statistical structure of neural activity and a larger instability of visual fixation. 

Regardless of the actual sources of instability, small saccades, ocular drifts, and/or combinations of movements of the eye and the head, a substantial degree of retinal image motion occurs during natural vision. In addition to humans, fixational instability has been observed in every species for which eye movements have been recorded, including the monkey (Skavenski, Robinson, Steinman, & Timberlake, 1975; Motter & Poggio, 1984; Snodderly & Kurtz, 1985; Snodderly, 1987), the cat (Pritchard & Heron, 1960; Hebbard & Marg, 1960; Winterson & Robinson, 1975; Conway, Timberlake, & Skavenski, 1981), the rabbit (Collewijn & van der Mark, 1972), the turtle (Greschner, Bongard, Rujan, & Ammermüller, 2002), and even the owl (Steinbach & Money, 1973), a species whose eyes are often considered immobile. The results of this work provide support to the hypothesis that this jittering plays a role even within the brief periods of visual fixation. Further studies are needed to determine the precise nature of this role and its relevance in the presence of more natural visual stimulation. 

Acknowledgments

This work was supported by NSF Grant EIA-0130851. The authors thank the late Professor Jacob Beck for helpful discussions and Andrew Schwartz for his valuable help in the preparation and execution of some of the experiments. Commercial relationships: none.