Retinal image shifts, but not eye movements per se, cause alternations in awareness during binocular rivalry

Abstract

Particularly promising studies on visual awareness exploit a generally used perceptual bistability phenomenon, “binocular rivalry”—in which the two eyes' images alternately dominate—because it can dissociate the visual input from the perceptual output. To successfully study awareness, it is crucial to know the extent to which eye movements alter the input. Although there is convincing evidence that perceptual alternations can occur without eye movements, the literature on their exact role is mixed. Moreover, recent work has demonstrated that eye movements, first, correlate positively with perceptual alternations in binocular rivalry, and second, often accompany covert attention shifts (that were previously thought to be purely mental). Here, we asked whether eye movements cause perceptual alternations, and if so, whether it is either the execution of the eye movement or the resulting retinal image change that causes the alternation. Subjects viewed repetitive line patterns, enabling a distinction of saccades that did produce foveal image changes from those that did not. Subjects reported binocular rivalry alternations. We found that, although a saccade is not essential to initiate percept changes, the foveal image change resulting from a (micro)saccade is a deciding factor for percept dominance. We conclude that the foveal image must change to have a saccade cause a change in awareness. This sheds new light on the interaction between spatial attention shifts and perceptual alternations.

Introduction

Perhaps as old as the interest in perceptual alternations in visual awareness is the question to what extent eye movements play a role in this process, for eye movements cause the retinal images to change. To test the role of eye movements in awareness alternations, a technique using retinally fixed afterimages of flashed bright visual stimuli has been used. In those studies, each eye was flashed with a different stimulus, resulting in binocular rivalry in which the two eyes' stimuli competed for awareness (output) without a changing retinal input. Those studies revealed that perceptual alternations can occur although the stimulus did not shift on the retina (Blake, Fox, & McIntyre, 1971; Lack, 1971; McDougall, 1903). 

Although there is now general consensus that alternations in awareness can occur without eye movements, there is also evidence that eye movements do play a cardinal role. It has been demonstrated that the typical percept duration for afterimage rivalry and real-image rivalry differs by a factor of at least 2 (durations of 4 and 1.5 s, respectively; Wade, 1974; see also Wade, 1975), which does suggest a role for eye movements in triggering perceptual alternations (Wade, 1974, 1975). Further, Kaufman (1963) reported that the spread of suppression was wider when a horizontal line was partially suppressed than when a vertical line was partially suppressed. He related this to nonconjugate eye movements, which occurred more frequent in the horizontal direction than in the vertical direction. Moreover, for saccades (van Dam & van Ee, 2006) as well as for simulated microsaccades (Sabrin & Kertesz, 1983), it has been demonstrated that there is a positive correlation between the saccades and perceptual alternations. 

To examine the precise role of retinal image changes due to saccades, we used conventional orthogonal gratings, which, by definition, are repetitive in space, which means that not all eye movements lead to foveal image changes. The left panel of Figure 1 shows two examples of saccades that are in the same direction. For the saccade from A to B, the local retinal image has not changed after the arrival at Point B. However, for the saccade from A to C, the local retinal image gets entirely reversed in luminance. By tracking saccades while subjects viewed the rivaling gratings, we were able to determine the correlation between saccades as well as their resulting retinal image changes and perceptual alternations. 

Figure 1

 

Examples of saccadic retinal image shifts for a left-oblique grating. A saccade from A to B shifts the whole image on the retina, but in the fovea, the image has not changed (right, top panel). For a saccade from A to C, the foveal image does change (right, bottom panel). We examined the effect of such retinal changes on percept dominance in binocular rivalry.

Methods

We used conventional red–green anaglyph stimuli on a monitor (40 × 30 cm; 1,600 × 1,200 pixels at 75 Hz). The intensities of the red and green half-images were adjusted until they were equiluminant when viewed through the red and green filters, which were custom-made (Bernell, Belgium), so that their transmission spectra matched the emission spectra of the monitor (minute amounts of green and red light leaked through the red [0.4%] and green [0.2%] filters, respectively). Head movements were restricted by a chin rest positioned at 55 cm from the monitor. Gaze positions were measured using an SMI EyeLink system (250 Hz). 

The stimulus consisted of ±45° sinusoidal gratings (0.75 cycles/deg; diameter, 6.5° in Experiments 1 and 3; 4.5° in Experiment 2). For all three experiments, the stimulus was displayed in a black window (9.2° × 10.0°) within a reference background (40.0° × 30.5°), which consisted of small squares (0.4°). Only 80% of the squares in the reference background were shown to prevent subjects from experiencing the wallpaper effect. 

Subjects were either free to make eye movements ( Experiment 1) or instructed to fixate a fixation dot of 0.2° ( Experiment 2) while they reported perceptual alternations during 120-s trials. To obtain a response reaction time after a perceptual alternation and to check whether subjects responded reliably in Experiments 1 and 2, we constructed a nonrivalrous stimulus condition for which only one grating was presented binocularly. The orientation of this grating was changed by 90° at random moments. Such stimulus alternation trials were distributed across the separate sessions and were shown intermixed with the perceptual alternation trials. Each experimental session contained eight perceptual alternation trials that were randomized for eye and grating orientation combinations. 

In Experiment 3, a black disk (1.6°) and a fixation target (0.4°) were added to the grating stimulus (see Figure 4a), which was presented alternately (for 0.5-s intervals) with a blank stimulus in which only the background and the fixation target were present (for 53-ms intervals). After the second blank, the luminance of none, one, or both of the gratings was reversed (counterphased grating). Subjects' task was to indicate the percept they had the instant before a mask (6.5°) of dynamic random overlapping dots (density, 9.8 dots/deg ²; size, 0.4°) appeared (at 0.25, 0.5, 0.75, 1.0, 1.25, 1.50, 1.75, or 2.0 s from the moment of stimulus manipulation). 

Eye movement analysis

The filtering of the saccade begin and end marks included the detection of microsaccades (for a detailed description, see van Dam & van Ee, 2006). From the begin and end marks for each saccade, we calculated the resulting retinal phase shifts, p_lo and p_ro, of the left-oblique and right-oblique gratings, respectively (on a scale from 0 to 1 cycle). If p_lo was close to a half-cycle (between 0.25 and 0.75) and p_ro was close to a whole cycle (<0.25 or >0.75), the saccade was categorized as leading to a retinal change for the left-oblique grating only (change-LO saccades). For the reverse situation, the saccade was categorized as leading to a retinal change for the right-oblique grating only (change-RO saccades). All other saccades (p_lo and p_ro similar) were excluded from further analysis (for the free eye movement condition in Experiment 1, on average, 65% of the saccades were excluded, and for Experiment 2 [fixation condition], 71% of the saccades were excluded). 

Temporal correlation between saccades and alternations

We made occurrence histograms (Perkel, Gerstein, & Moore, 1967; van Dam & van Ee, 2005, 2006) in which we plotted the occurrences of saccades relative to the moments of the button presses using a bin width of 100 ms. Only the change-LO and change-RO saccades, as described above, were used in this analysis. From the obtained histograms, the intervals −10.0 to −5.0 and +5.0 to +10.0 s with respect to the button press were used to calculate the mean and the standard deviation of the bin height (as a reference level). We considered a peak or trough in the interval −5.0 to +5.0 s to be significant when two or more neighboring bins within the peak or the trough differed more than 2 SD from the mean (see van Dam & van Ee, 2006). A measure for the differences between the histograms was obtained by applying the Kolmogorov–Smirnov test in a pairwise manner on the raw saccade perceptual alternation intervals within the ranges −2 to 0 and 0 to 2 s. 

Correlation between saccades and the dominant percept

Apart from the correlation between saccades and perceptual alternations, we calculated the correlation between saccades and the percept during percept intervals. We omitted effects correlated to the perceptual alternations by excluding the last 750 ms of each percept interval (as measured from button press to button press) from this analysis. This 750 ms corresponds to the reaction time for the succeeding percept plus the temporal width of the previously found peak of saccades that is correlated to perceptual alternations (van Dam & van Ee, 2006). For each of the thus obtained percept intervals, we determined the number of change-LO saccades and the number of change-RO saccades during the percept interval. We summed the number of change-LO saccades and the number of change-RO saccades across all the left-oblique percept intervals and similarly across all the right-oblique percept intervals. From the sums of change-LO saccades and change-RO saccades, we could then obtain the proportion of change-LO versus the change-RO saccades during each specific percept. For each subject, the proportions of change-LO saccades for both the perceptual states were compared to 0.5 (chance level) using the exact test. 

Experiment 1: Binocular rivalry and free eye movements

Figure 2 displays the correlation between saccades and perceptual alternations, demonstrating the deviation from the baseline saccade probability versus the time relative to the moment of the button press for alternations to the left-oblique grating (LO, left column) and alternations to the right-oblique grating (RO, right column). The top and bottom panels show the histograms for the change-LO saccades (saccades that resulted in a retinal change for the left-oblique grating but not for the right-oblique grating) and the change-RO saccades, respectively. Within each panel, three color representations of the occurrence histograms are shown, one for each subject. At the bottom of each panel, the average deviations across the three subjects (black line) and the average deviation ±1 SE across the subjects (gray lines) are shown. 

Figure 2

 

The panels show the deviation from the baseline saccade probability versus the time relative to the button press for alternations to the left-oblique grating (left column) and alternations to the right-oblique grating (right column). The top panels show the histograms for saccades that resulted in a retinal change for the left-oblique grating but not for the right-oblique grating (change-LO). Similarly, the bottom panels show the histograms for the change-RO saccades. Within each panel, three color occurrence histograms are shown, one for each subject, for which the color represents the deviation within a bin from the reference bin height, expressed in the number of standard deviations. The graph at the bottom of each panel depicts the average deviations (black line) ±1 SE (gray lines) across the subjects. The thick gray vertical bar represents an estimate of the moment that the actual alternation occurred relative to the button press. The top-left and the bottom-right panels show a clear positive correlation between saccades and perceptual alternations at about the moment of the perceptual alternations. In the top-right and bottom-left panels, this correlation is absent. This indicates that retinal image changes due to saccades are correlated with the perceptual alternations.

The top row in Figure 2 (the change-LO saccades) reveals a clear positive correlation between the change-LO saccades and perceptual alternations to the left-oblique grating at about the moment of the perceptual alternations (left panel). For alternations to the right-oblique grating (right panel), this positive correlation is not significant and disappears completely if slightly stricter thresholds for the “no retinal change” and the “retinal change” saccade categorization are applied (for instance, a “no change” categorization if the phase difference was <0.2 or >0.8 and a “change” categorization if the phase difference was between 0.3 and 0.7; see also the Methods section). This indicates that saccades that result in a retinal image change for solely the left-oblique grating can result in left-oblique grating dominance but not likely in right-oblique grating dominance. Similarly, the bottom row of the panels in Figure 2 shows that there is a clear positive correlation at about the moment of the perceptual alternation between the change-RO saccades and the perceptual alternations toward the right-oblique grating (right panel) but hardly between the change-RO saccades and perceptual alternations toward the left-oblique grating (left panel). These results indicate that it is not the saccade itself that correlates strongly with the perceptual alternation, but rather it is the local retinal image change. 

If local retinal image changes were not important, we would predict all the histograms of Figure 2 to appear very similar. Note that all histograms show the decrease in saccade probability between the moment of the perceptual alternation and the moment of the button press, indicating, intriguingly, that there is a reduced probability of making saccades just after a perceptual alternation. Although this decrease could, in principle, be caused by the fact that there is a peak in saccades just before the perceptual alternation, there is independent evidence (which is convincing for, for instance, slant and Necker cube rivalry; van Dam & van Ee, 2005, 2006) that this decrease does reflect a reset of saccade planning. 

Apart from the correlation between saccades and perceptual alternations, we were interested in the correlation between saccades and the percept during percept intervals (see the Methods section). During the left-oblique percepts, the proportion of change-LO saccades was significantly higher ( p < .01) than chance for all individual subjects (on average, the proportion was 63%). During the right-oblique percept intervals, the proportion of change-LO was significantly lower than chance (on average, 30%; p < .01) for all subjects. The difference in the proportion of change-LO saccades during the two different perceptual states indicates that there is also a correlation between retinal image changes and the dominant percept. This suggests that, apart from triggering perceptual alternations, saccades can help to maintain a specific percept by inducing retinal changes for the dominant grating only. 

In sum, the results indicate that retinal image changes are crucial for the correlation between saccades and perceptual alternations and that retinal image changes determine, at least in part, perceptual dominance. Here, it might be of interest to note the similarity with monocular rivalry, for which it has been reported that retinal image changes determine the dominant percept (Bradley & Schor, 1988; Georgeson & Phillips, 1980), a finding that we have recently replicated using the current setup and analyses (van Dam, 2006). Note that sinusoidal gratings of 0.75 cycles/deg can induce monocular rivalry (Atkinson, Campbell, Fiorentini, & Maffei, 1973) and perceptual fading of the gratings (Tulunay-Keesey, 1982). Such “monocular rivalry” alternations might play a role in our binocular rivalry paradigm. Therefore, we repeated the experiment for two subjects, using square-wave gratings, for which the number of monocular rivalry alternations and the amount of perceptual fading should be reduced (Atkinson et al., 1973; Campbell & Howell, 1972; Tulunay-Keesey, 1982). The results for the square-wave gratings were very similar to the results described here for the sinusoidal gratings. 

Experiment 2: Binocular rivalry and fixation

To investigate how microsaccades affect grating visibility in binocular rivalry, we repeated Experiment 1 with strict fixation. Microsaccades are known to correlate with target visibility in the Troxler effect (Martinez-Conde, Macknik, Troncoso, & Dyar, 2006) and with perceptual alternations in binocular rivalry (Sabrin & Kertesz, 1980, 1983). We combined the occurrence histograms of the change-LO saccades with the histograms of the change-RO saccades to obtain, first, a histogram for saccades that produce a retinal change in the grating that becomes dominant but not in the grating that gets suppressed (top panel of Figure 3) and, second, a histogram for saccades that produce a retinal change in the grating that gets suppressed but not in the grating that becomes dominant (bottom panel). 

Figure 3

 

Results for Experiment 2 in which we investigated the correlation between fixational saccades and perceptual alternations for binocular rivalry. Respectively, the top and bottom panels show the histograms for change-to-dominance saccades and change-to-suppressed saccades. In each panel, three color representations are shown, one for each subject and, as before, the average deviation across the three subjects is shown in the bottom graph of each panel. There is a strong positive correlation between the change-to-dominant saccades and the perceptual alternations (peak at the moment of the perceptual alternations). For the change-to-suppressed saccades, this positive correlation is either absent or much reduced. These results reveal a clear correlation between microsaccadic retinal image changes and perceptual alternations for binocular rivalry.

The results again demonstrate a strong correlation between the retinal image changes and the grating that becomes dominant (there does appear to be a small peak in the change-in-suppressed saccades for subject J.B., but again, this correlation disappears with stricter saccade division and the difference between the two histograms is significant even for subject J.B.; p < .05). The proportion of change-to-dominant saccades during the percept intervals was again significantly higher than chance level (on average, 59%, with p < .01 for two subjects and p < .05 for the third subject), indicating that microsaccades can help to maintain a percept even when the subjects are instructed to fixate. 

Experiment 3: Retinal image changes without saccades

Our suggestion that the retinal image changes due to saccades, rather than the saccades per se, cause perceptual alternations naturally leads to the hypothesis that local retinal image shifts without co-occurring eye movements trigger perceptual alternations. To test this hypothesis, we presented alternately the grating stimulus and a blank (see Figure 4b). After the second blank, one of four possible stimulus manipulations occurred: The luminance of the left-oblique grating could be reversed (counterphased grating); the right-oblique could be counterphased; both gratings could be counterphased or the gratings remained unaltered. After the manipulation, the stimulus was removed at varying intervals and substituted by a mask. The subjects' task was to report the percept they had the instant before the mask appeared. 

Figure 4

 

Panels a and b show the setup of Experiment 3 in which we investigated, by using physical stimulus shifts, whether retinal shifts without saccades were sufficient to trigger perceptual alternations. The grating stimulus (0.5 s) and a blank stimulus (53 ms) were presented alternately during the entire trial. In the second blank, none, only one, or both of the gratings were counterphased. At various, but distinct, moments after the stimulus manipulation, the stimulus was substituted by a mask and subjects had to indicate their last percept. Panel c shows the proportions of left-oblique grating perceived versus the time relative to the moment of stimulus manipulation. Error bars represent standard errors across subjects. The results demonstrate that if only one grating is counterphased (filled symbols), that grating becomes dominant, implying that retinal image changes without saccades are sufficient to manipulate the percept.

Trials in which subjects either blinked or made saccades that resulted in more than a 0.25-cycle phase shift for one or both gratings were excluded. The proportion of left-oblique grating perceived is depicted in Figure 4c, showing clearly that the left-oblique grating either became or remained dominant when only the left-oblique grating was counterphased (red disks). Similarly, the right-oblique grating either became or remained dominant when the right-oblique grating was counterphased (blue filled squares). When either both or none of the gratings were counterphased, subjects did not show a preference for either the left-oblique or right-oblique grating (open symbols). 

In sum, we conclude that retinal changes without saccades can cause perceptual alternations. Another interesting aspect in Figure 4c is that after about 1 s of percept dominance, the proportions of left-oblique grating start returning to 0.5, indicating that perceptual alternations occur after the initial period of percept dominance. If retinal image changes would account for all perceptual alternations, we would predict that the results in Figure 4c would look like horizontal straight lines because after the stimulus manipulation, there would be no more events that could trigger an alternation. Also, the return to chance level in Figure 4c cannot be due to retinal changes caused by saccades larger than 0.25 grating cycle because trials in which such saccades occurred were removed from the analysis. Therefore, assuming that other eye movements (drift, tremor, etc.) did not significantly affect the results, the return to chance level in the proportions of left-oblique grating perceived suggests a central alternation process. 

Discussion

We found retinal image changes to be crucial to have a saccade cause a perceptual alternation for both free eye movements ( Experiment 1) and for fixation ( Experiment 2) and that retinal image changes without saccades can indeed determine percept dominance ( Experiment 3). 

Although the experiments reported here were conducted using a relatively low spatial frequency, it is very likely that the results extend to binocular rivalry stimuli in general. A high positive correlation between saccades and perceptual alternations has already been reported for two other binocular rivalry stimuli: one consisting of 2.9 cycles/deg gratings and one consisting of an image of a house and an image of a face (van Dam & van Ee, 2006). It is unlikely that the previously found correlation had some other origin than the correlation reported here. Monocular effects like perceptual fading are also unlikely to have affected the results in a significant manner because similar results are found when square-wave gratings (for which monocular fading effects should be much reduced; Atkinson et al., 1973; Campbell & Howell, 1972; Tulunay-Keesey, 1982) are used instead of sine-wave gratings. 

The findings reported here are also consistent with previous reports that abrupt changes in one eye's stimulus (i.e., either phase or frequency changes of a grating; Walker & Powell, 1979; transient contrast changes; Blake, Westendorf, & Fox, 1990; Walker & Powell, 1979; Wilson, Blake, & Lee, 2001; or a delay in the onset of the image of one eye; Wolfe, 1984) can determine percept dominance. Our findings indicate that saccades, due to their resulting retinal image changes, can truly cause perceptual alternations in a similar manner as the abovementioned changes in the stimulus. 

The occurrence of binocular rivalry with retinal afterimages has frequently led to the assumption that the role of eye movements in binocular rivalry is negligible. Here, we have shown that for real images, this assumption is not valid, but, even for afterimages, a role of eye movements cannot be ruled out because the perception of afterimages is known to depend on eye movements. For instance, afterimages tend to disappear (e.g., Kennard, Hartmann, Kraft, & Boshes, 1970) and reappear after the execution of a saccade (e.g., Ditchburn, 1973; McDougall, 1903). For stabilized image rivalry (by controlling for occurring eye movements), the stabilization of the image on the retina is usually not perfect, otherwise the images would perceptually disappear (Ditchburn & Ginsborg, 1952; Martinez-Conde et al., 2006; Riggs, Ratliff, Cornsweet, & Cornsweet, 1953), which means that the influence of eye movements during stabilized image rivalry remains to be verified. 

Perceptual alternations can occur without saccades and without stimulus changes (return to chance level for the perceived grating orientation in our Experiment 3), indicating a central process. Covert spatial attention is such a central process through which percept dominance can be influenced (Brouwer, Tong, Schwarzbach, & van Ee, 2005; Slotnick & Yantis, 2005; von Helmholtz, 1910). Our finding that retinal image changes due to microsaccades can trigger perceptual alternations must be taken into account when studying the role of attention. In addition, several studies demonstrated that spatial attention shifts are usually accompanied by microsaccades in the direction of the attention shift even under strict fixation tasks (Engbert & Kliegl, 2003; Hafed & Clark, 2002; Laubrock, Engbert, & Kliegl, 2005). Thus, a correlation between attention shifts and perceptual alternations might potentially be due to the retinal image changes resulting from co-occurring microsaccades rather than the attention shifts per se. 

Because only a restricted class of eye movements (those that change the foveal image) can cause a change in awareness, the preparation of an eye movement is not sufficient to cause a change in awareness (unless the saccade landing position would be accurately known, which presumably is not the case given the large errors of saccades to well-defined targets; van Opstal & van Gisbergen, 1989). The “premotor theory” (Rizzolatti, Riggio, Dascola, & Umiltá, 1987) posits that the allocation of covert spatial attention is equivalent to planning but not executing a saccade. According to this theory, our results suggest that covert spatial attention shifts (top–down influence) cannot be responsible for perceptual alternations in binocular rivalry or at least that this top–down influence is much weaker than the bottom–up influence through the retinal image changes. Indeed, for binocular rivalry, there is evidence that voluntary control has only little influence on percept dominance (Meng & Tong, 2004; van Ee, van Dam, & Brouwer, 2005) unless the bottom–up stream of information is changed either continuously (Chong, Tadin, & Blake, 2005) or abruptly (Chong & Blake, 2006; Mitchell, Stoner, & Reynolds, 2004). 

Conclusions

In sum, existing research (and our Experiment 3) implies that perceptual alternations can occur without saccades and without stimulus changes, indicating a central process. Here, we demonstrated that the retinal shift due to an eye movement can be a deciding factor for percept dominance: The foveal image must change to have a saccade cause a change in awareness, and saccades that favor the currently dominant percept help to maintain that percept. In addition, there is evidence for a reduced probability of making saccades just after a perceptual alternation. These findings shed new light on the role of overt and covert spatial attention shifts for perceptual alternations in binocular rivalry. 

Acknowledgments

The authors were supported by a grant from the Netherlands Organization for Scientific Research (NWO) assigned to R.v.E. 

Commercial relationships: none. 

Corresponding author: Raymond van Ee. 

Email: [e-mail]. 

Address: Princetonplein 5, 3485 CC Utrecht, The Netherlands.