Object form discontinuity facilitates displacement discrimination across saccades

Abstract

Stimulus displacements coinciding with a saccadic eye movement are poorly detected by human observers. In recent years, converging evidence has shown that this phenomenon does not result from poor transsaccadic retention of presaccadic stimulus position information, but from the visual system's efforts to spatially align presaccadic and postsaccadic perception on the basis of visual landmarks. It is known that this process can be disrupted, and transsaccadic displacement detection performance can be improved, by briefly blanking the stimulus display during and immediately after the saccade. In the present study, we investigated whether this improvement could also follow from a discontinuity in the task-irrelevant form of the displaced stimulus. We observed this to be the case: Subjects more accurately identified the direction of intrasaccadic displacements when the displaced stimulus simultaneously changed form, compared to conditions without a form change. However, larger improvements were still observed under blanking conditions. In a second experiment, we show that facilitation induced by form changes and blanks can combine. We conclude that a strong assumption of visual stability underlies the suppression of transsaccadic change detection performance, the rejection of which generalizes from stimulus form to stimulus position.

Introduction

As human observers make saccadic eye movements to explore a scene, its projection on the retina is subject to large shifts of various directions and sizes. Despite the challenge that this succession of lateral input displacements would seem to pose to the visual system, observers are provided with a subjective experience of a stable and richly detailed world. This intriguing behavior and the mechanisms underlying it have received ample attention from vision scientists. 

Intuitively, the visual system would seem to possess all necessary information to stabilize perception across eye movements, since the motor signals that drive saccades are initiated from within the same biological system that has to compensate for the visual input displacements produced by them. If the extra-retinal motor signal of the eye movement is simply subtracted from the retinal displacement signal of an unmoving object, a spatially stable and therefore veridical percept could ensue. This suggestion of a cancellation approach to the visual stability problem is indeed an old one (Sperry, 1950; von Helmholtz, 1866; von Holst & Mittelstaedt, 1950), but on a neurophysiological level both types of signals are too different to be simply subtractable (Sommer & Wurtz, 2008). Rather, the currently most popular theory holds that a corollary discharge of the saccadic motor signal is indeed present but serves a different purpose: By briefly shifting the receptive fields of retinotopic neurons in anticipation of the change in correspondence between the retinotopic and spatiotopic coordinate systems, the same neurons can transsaccadically encode information on the same parts of a scene despite the retinal image shift induced by the eye movement. Such receptive field remapping has been observed in parietal cortex, extrastriate visual cortex, the frontal eye fields, and the superior colliculus; using single-cell recording, ERPs, fMRI, or psychophysical studies; and both in monkeys and in humans (Duhamel, Colby, & Goldberg, 1992; Mathôt & Theeuwes, 2010; Melcher, 2005, 2007; Merriam, Genovese, & Colby, 2003, 2007; Nakamura & Colby, 2002; Parks & Corballis, 2008; Umeno & Goldberg, 1997; Walker, Fitzgibbon, & Goldberg, 1995). Note however that horizontal activation transfer between neurons encoding a salience map could constitute an alternate explanation to some of these data (Cavanagh, Hunt, Afraz, & Rolfs, 2010). 

It seems unlikely that any mechanism compensating for the retinal image shift through extra-retinal information alone could fully account for the human perceptual experience of visual stability though, as the corollary discharge signal and the processes it drives are clearly not free of noise and biases. When briefly flashing isolated stimuli in darkness around the time of a saccade, significant errors have been observed to occur in their localization. Starting up to 100 ms before the initiation of the eye movement perceived locations are biased in the direction of the saccade, whereas an opposite tendency is present at the end of the saccade and for several tens of milliseconds afterward (Honda, 1989, 1991). Moreover, when stimuli are flashed before the saccade and in the presence of visual references, human localization data display not only a lateral bias but also a compression of visual space onto the saccade target (Lappe, Awater, & Krekelberg, 2000; Ross, Morrone, & Burr, 1997). Perisaccadic localization errors thus appear to vary in both direction and magnitude across space and time and can be observed to be present during a significant part of the successive fixation periods. If the extra-retinal mechanisms relevant to especially the lateral shift component of perisaccadic localization behavior would also be driving the stabilization of real-life vision, absolute space perception would indeed be non-veridical during the majority of the time spent exploring a scene. Therefore, the corollary discharge theory of visual stability merely shifts the problem. Rather than having to account for the lack of large apparent scene displacements with every saccade made, we are now left to wonder why observers have no experience of smaller instabilities as a result of both transient and absolute errors in compensating the retinal effects of the saccadic motor command. 

Change detection across saccades

Corroborating this conclusion, actual external instabilities in the form of stimulus displacements applied during a saccade go largely unnoticed to the observer, especially when the saccade length is long (Bridgeman, Hendry, & Stark, 1975; Li & Matin, 1990). This implies that there is no need for a perfect extra-retinal compensation mechanism, since the visual system is apparently able of resolving transsaccadic instabilities even when they do not result from an eye movement but merely coincide with one. The interpretation of these results has often been in terms of the precision with which presaccadic information is encoded and transferred across the saccade, suggesting a coarse transsaccadic representation of positional information (Bridgeman, Van der Heijden, & Velichkovsky, 1994; Irwin, 1992; O'Regan, 1992). The argument then is that we do not perceive stimulus displacements across saccades because we do not have a sufficiently accurate postsaccadic representation of presaccadic stimulus position information. This interpretation is congruent with a second class of experimental results, demonstrating that transsaccadic change detection of stimulus form is similarly impaired (Deubel, Schneider, & Bridgeman, 2002; Grimes, 1996; Henderson & Hollingworth, 2003), and that only coarse and abstract form properties of presaccadic stimulation contribute to the speed and accuracy of postsaccadic stimulus recognition (Henderson, 1994, 1997; Pollatsek, Rayner, & Collins, 1984; Pollatsek, Rayner, & Henderson, 1990; Rayner, McConkie, & Zola, 1980). 

Change detection across saccades is not always severely impaired, however: Stimuli that are defined by their orientation (Pollatsek & Rayner, 2002; Verfaillie, De Troy, & Van Rensbergen, 1994), embedded in a configuration (Verfaillie, 1997; Verfaillie & De Graef, 2000), or moving (Gysen, De Graef, & Verfaillie, 2002) allow better task performance. One explanation could be that some aspects of a scene or an object are simply more accurately represented than others. However, as Mitroff, Simons, and Levin (2004) point out, absent change detection does not necessarily imply the absence of a representation that could in principle enable successful task completion: It only demonstrates that such a representation is not being employed. That is, the visual system may be unable to adapt to the artificial demands of a psychophysical task. Indeed, the priority of real-life transsaccadic perception would seem to be upholding subjective stability rather than accurately detecting highly unlikely and behaviorally irrelevant events, such as minor object displacements that are exactly synchronized with a saccade. As a result, a detailed stimulus position representation could be suppressed from conscious perception despite the explicit instruction to retrieve it. It could however still be covertly employed for other purposes, such as motor coordination (Prablanc & Martin, 1992). 

Blanking and landmark effects

Deubel, Schneider, and Bridgeman (1996) delivered the most convincing evidence for this idea. In their study, subjects were required to indicate the horizontal displacement direction of a small dot, when performed intrasaccadically. Predictably, task accuracy was low for stimulus shifts of up to 2 visual degrees in size. However, the simple insertion of a short blank interval during and immediately after the saccade, before the onset of the postsaccadic stimulus, improved displacement discrimination performance dramatically. This strongly suggests that the presaccadic position representation was indeed detailed enough to solve the task, but without an immediate postsaccadic blank this representation could not be explicitly compared to the incoming postsaccadic position information, or at least not in a veridical manner. 

Deubel, Bridgeman, and Schneider (1998) further report that if a spatially stable but postsaccadically blanked stimulus was displayed in the vicinity of an intrasaccadically displaced but continuously present second stimulus, the latter stimulus was seen as stable and used as a reference for inferring the apparent displacement of the former. These findings indicated that the encoded presaccadic position information is not merely being suppressed, but actively being used in a visual stability mechanism based on retinal information: Assuming that the real world is stable, presaccadic and postsaccadic vision will be aligned with one another based on transsaccadically persistent landmarks. Only when this is not possible, for instance when the visual system does not encounter any immediate stimulation after a saccade, will it fall back on different mechanisms of visual stability, which are presumably extra-retinal and based on the corollary discharge. The fact that this fallback to imperfect position information actually improves task performance demonstrates the strength of the visual stability assumption. Thus, while extra-retinal information can certainly contribute to the stability of transsaccadic vision, visual information is dominant. This precedence of vision over conflicting information sources has also been called “visual capture” (Matin et al., 1982). 

The role of object form in attaining visual stability

Surprisingly, Deubel et al. (2002) report that landmark effects do not require form similarity between the presaccadic stimulus and its corresponding postsaccadic landmark. To be exact, they observed that the performance improvements induced by target blanking can be undone by inserting flanking stimuli around the position of the displaced saccade target during the blank interval, as long as these flankers are clearly localized within the spatial dimension on which the saccade displaces the retinal projection (e.g., long horizontal bars will not work for a horizontal saccade). Similarly, Koch and Deubel (2007) demonstrate that the landmark effect can go as far as using the gravitational center of two postsaccadic objects as a spatial reference for the position of a single presaccadic stimulus. Thus, when the target object is not available immediately after the saccade, other nearby objects can be used as a reference even if they were not present before the saccade. Their position on the spatial dimension traveled by the saccade will then be assumed to correspond to that of the saccade target. 

This appears to run counter to the notion of a stable world null hypothesis as the main determinant of visual stability in real-life vision. Indeed, it has been suggested but never proven that the visual stability assumption is primarily evaluated by attempting to relocate the saccade target object itself on the basis of certain critical locating features immediately after saccade landing (Deubel, Wolf, & Hauske, 1984; McConkie & Currie, 1996). In the study of Deubel et al. (2002), the saccade target object was however absent when the saccade landed, possibly invoking a separate visual stability mechanism that could utilize flanking landmarks (Higgins & Wang, 2010). In the present study, we will therefore investigate whether displacement detection can be observed to be improved when the saccade target does remain present at saccade landing but is unstable in its form properties as well as its position. 

Looking at this research premise from a somewhat different angle, Deubel et al. (2002) also show that an immediate postsaccadic blank improves transsaccadic detection performance for form changes as well as displacements. This suggests that a similar mechanism could be underlying both types of saccadic change detection suppression, encompassing an assumption of visual stability that is maintained and used as a basis for a transsaccadically stable perceptual experience as long as the discrepancy between both perisaccadic pieces of information can reasonably be assumed to be the result of an error internal to the human body (Niemeier, Crawford, & Tweed, 2003). If the rejection of such an inbuilt assumption of visual stability is the key to detecting intrasaccadic stimulus changes of both object position and object form, could then indeed the successful detection of an object form change be sufficient to refute visual stability and improve detection of simultaneous displacements of the same object? Or, to make a perhaps counterintuitive prediction: Is displacement detection across saccades easier between dissimilar stimuli? 

To test whether a rejection of visual stability generalizes from form to position, we asked subjects to judge the direction of saccade target displacements while the task-irrelevant visual form of the saccade target could be altered simultaneously. In addition, we compared the effect of a form change to the effect of a postsaccadic blank, the most commonly used technique to induce better displacement discrimination performance. If both manipulations trigger the same mechanism of disrupting visual stability, we would expect them to have similar but non-additive effect sizes. That is, if a postsaccadic blank has already refuted the visual stability assumption, a form change to the reappearing saccade target will do nothing to improve performance further. If, on the other hand, they activate independent mechanisms of performance facilitation, they should be additive. 

In sum, four questions were initially asked:

1.  
   Does a postsaccadic blank improve intrasaccadic displacement discrimination in our paradigm?
2.  
   Does a form change to the saccade target improve displacement discrimination?
3.  
   Is displacement discrimination performance improved to the same degree by a postsaccadic blank and an intrasaccadic form change?
4.  
   Does their joint occurrence lead to an additive effect?

Experiment 1

Methods

Subjects

Six subjects, five males and one female, between the ages of 24 and 47 participated in this experiment. Two of these subjects were authors on this paper; the remaining four subjects were completely naive with regard to the aims and conditions of the experiment. All had normal or corrected-to-normal vision. 

Apparatus

Stimuli were presented on an Iiyama Vision Master Pro 541 22-inch monitor, with a temporal resolution of 200 Hz and a spatial resolution of 800 by 600 pixels, subtending 17 by 13 visual degrees. Participants were seated at a distance of 135 cm from the monitor, with their head stabilized by a head rest and a bite bar with dental impression compound. Eye movement data were collected using an analog Dual Purkinje Image eye tracker (Crane & Steele, 1985) sampled at 1000 Hz and processed by custom software on a Windows XP platform. Stimulus presentation and analog-to-digital conversion were performed by a Cambridge Research Systems Visage stimulus generator. The response buttons used were of the analog “breaker” type, interpreting an interruption of the current as a button press, and were read in through the parallel port on the Visage. The monitor was gamma-corrected by the automatic routines included with the Visage system. 

Stimuli

We used four simple geometrical shapes as stimuli, to make form changes as qualitative as possible while still retaining easy identification of their centers. This enables subjects to accurately detect displacements even between dissimilar shapes. The shapes were a cross, a circle, a square, and a diamond (see Figure 1). They subtended an area of 26 by 26 pixels (0.55 by 0.55 visual degree on the screen). The intrasaccadic location changes performed relative to the center of each stimulus ranged from 0.11 to 0.44 visual degree, resulting in an overlap between 0.06 and 0.44 visual degree. A background monitor luminance of 7.7 cd/m² and a Michelson contrast of 0.71 were used for all stimuli. 

Figure 1

 

The four different stimuli used in Experiment 1, referred to as “diamond,” “square,” “circle,” and “cross.”

Procedure

We used a three-way fully factorial design. First, we manipulated the size of the saccade target displacement to either 0.11, 0.22, 0.33, or 0.44 visual degree. A vertical displacement (upward or downward) was chosen in order to avoid introducing a systematic bias related to the horizontal undershooting of the saccade target. Second, on half of the trials we introduced a postsaccadic blank, on the other half we did not. Third, we changed the stimulus form on half of the trials. All 12 combinations of possible stimulus form changes (cross to circle, square to diamond, etc.) were used randomly. 

Figure 2 illustrates the procedure. In a dimly lit room, participants were instructed to fixate a small cross 2.7 degrees to the left of the center of the screen. They had two buttons at their disposal. During fixation, they could press the right button to instantly apply drift correction to the eye tracker calibration setup, as long as the deviation was smaller than 1 visual degree of angle. In the beginning of the experiment, the experimenter gave the explicit instruction to use the drift correction button when the fixation position was found to be inaccurate, but after the first block most subjects started using it spontaneously following a failed fixation. As soon as the left button was pressed, a random fixation period lasting between 500 and 1500 ms started, after which a stimulus appeared 5.4 degrees to the right of fixation, 2.7 degrees to the right of the center of the screen. When fixation on the initial cross was insufficiently accurate (eye position outside a region of 0.5 by 0.5 degree enveloping the fixation cross, plus a 0.3-degree tolerance zone to each side), the trial was aborted and the subject was admonished with both visual and auditory signals. In the vertical direction, the center of presaccadic stimulus could be located up to 0.33 degree away from the center of the screen, either upward or downward. Subjects were instructed to saccade toward the presaccadic stimulus as quickly as they could. Saccade latencies shorter than 150 ms and longer than 400 ms resulted in an abortion of the trial. In practice, the mean saccade latency was 188 ms, and on 95% of all trials it was shorter than 250 ms. Once the saccade was initiated and the gaze left the fixation zone, the stimulus was immediately displaced and could either disappear (blanking conditions) or not, and/or change form (form change conditions) or not. We estimate the intrasaccadic stimulus changes to have become effective on the screen 10–15 ms into the saccade. The center of the postsaccadic stimulus was never more than 0.22° removed from the horizontal centerline of the screen, and every postsaccadic stimulus position was equally likely to have resulted from an upward or a downward displacement. This way, postsaccadic position information was by itself not informative to solve the task. A blank lasted 100 ms, after which the postsaccadic stimulus was shown. Due to the partial overlap with the saccade, we estimate the effective postsaccadic blanking duration to have been around 80 ms. Both in blanking and non-blanking conditions, the postsaccadic stimulus remained on-screen for 150 ms and was never followed by a mask or any other visual stimulation. Within 2000 ms after the offset of the postsaccadic stimulus, the subject had to press the left button to indicate that the stimulus had moved upward, or the right button to indicate that it had moved downward. Aborted trials were recycled after each block of 50 trials; twice-aborted trials were not recycled again but dropped from the experiment. A total of 3.4% of all trials were lost. 

Figure 2

 

Subjects started each trial by fixating a cross on the left of the screen. After a random delay, a stimulus appeared to the right, to which they were instructed to saccade as quickly as possible. The vertical position of the stimulus always changed during the eye movement, and on half the trials the stimulus form was altered as well. The postsaccadic stimulus was either immediately present or preceded by a brief blank screen (on half the trials). The task was to ignore form changes and judge the direction of displacement as being upward or downward.

In total, 16 conditions were measured in each subject, 100 trials each, across typically four 1-h sessions in the DPI eye tracker. Trials were collected in blocks of 50 trials, after which subjects could rest their eyes for a little while. 

Results

Figure 3 shows the average proportion of correct responses in each condition. The logistic mixed model regressions, which were fitted to these data using the lme4 package for R, are also shown. This type of generalized linear model analysis accounts both for the binomial nature of the response variable (through transforming proportions to logits) and the within-subject nature of the experimental design (through modeling the random subject variability around each fixed effect). The full model consisted of a fixed intercept and seven fixed effects (Size, Form, Blank, and their two-way and three-way interactions), and a random subject variability parameter for each of these fixed terms. First, we reduced the random effects structure through sequentially removing those effects whose removal did not result in a significant drop in model fit (likelihood ratio test against a χ ² distribution). We started with the highest order terms, and only dropped lower order terms when they were not involved in a retained higher order interaction. Readding any of the dropped terms to the final reduced model did not result in an improved fit. Second, we reduced the fixed effect structure in a similar manner. Third, we readded those removed random effects for which the fixed effects were retained, to maximally account for subject variability in the hypothesis testing. This was the final model: 

$logit\left(p\right)={\beta }_0+r_0+({\beta }_1+r_1)Size+({\beta }_2+r_2)Form+({\beta }_3+r_3)Blank+({\beta }_{23}+r_{23})Form\times Blank+r_{13}Size\times Blank,$

(1)

where p is the proportion correct, β ₀ and r ₀ are the fixed intercept and its random subject variability, and β _i and r _i are the fixed and random regression weights for the remaining main and interaction effects, respectively. 

Figure 3

 

Proportion correct results for all subjects. Symbols represent data points averaged across subjects, with error bars indicating one standard error of between-subject variability. Blue and green lines represent the logistic regression model fit to these data points (see Table 1); the orange line illustrates the theoretical prediction of an additive combination of the isolated form change and blank effects. Main effects of displacement size, form change, and blanking are clearly present, but the departure from additivity (full green line vs. dotted orange line) is not statistically significant.

Table 1 shows the fixed effect estimates, as well the standard errors of their estimation and a Wald-Z significance test. All main effects are significant, whereas the Form by Blank interaction is only marginally significant. Fixed effects that are not listed were dropped from the model during the model selection procedure and can therefore be considered highly non-significant. In a direct comparison between these beta weights, a blank has a significantly greater effect than a form change (β ₃ > β ₂, t(8) = 2.18, p = 0.03), and the combined effect of a blank and a form change does not result in a significantly better performance than is measured with the blank alone (β ₃ > −β ₂₃, t(10) = 0.73, p = 0.75). Summarized, an intrasaccadic form change facilitated displacement discrimination, but only around half as much as a postsaccadic blank. The present data neither allowed to reject the null hypothesis that a form change improved performance on top of a blank (β ₃ = −β ₂₃), nor that full additivity existed between both effects (β ₂₃ = 0). 

Table 1

 

Fixed regression weights and significance levels of the logistic regression model on the proportion correct data.

Effect	Weight	Estimate	SE	p(z)
	β 0	−0.405	0.137
Size	β 1	1.140	0.095	<0.001
Form	β 2	0.611	0.157	<0.001
Blank	β 3	1.340	0.293	<0.001
Form × Blank	β 23	−0.403	0.239	=0.092

Figure 4 shows the model fit results when each type of form change (for instance, cross to diamond) received a separate β estimate under Model 1. It can be seen in the left panel that all form changes except one (square to cross) are, like the overall estimate (β ₂ = 0.61), situated in between the full blanking effect (β ₃ = 1.34) and the absence of any performance improvement (β = 0). The overall variation across form changes is not significant in a one-way ANOVA on these beta estimates (F(11,5) = 2.79, p = 0.13). The right panel shows the results for the Form × Blank interactions; though highly variable, these weights are negative for 10 out of 12 form change types, which is significantly more than could be expected if these values were in reality centered around 0 (full additivity; Yates-corrected χ ² _{(df = 1)} = 4.08, p = 0.04). 

Figure 4

 

(a) Estimated beta values for each type of form change, in the absence of blanking. Analogous to Figure 3, the interrupted blue line indicates the baseline performance in the absence of a form change, the interrupted green line indicates the average form change effect, and the full blue line indicates the average blanking effect. All but one form change type result in an intermediate performance level, lending support to the notion of an overall intermediate effect of these form changes. (b) Similarly, estimated beta values for the Form × Blank interaction. The dotted orange line illustrates the additivity hypothesis, the full blue line a performance level equal to that of the isolated blanking effect alone. The full green line indicates the average Form × Blank effect. It can be seen that although the overall Form × Blank effect is not significant, all but two beta weights are negative. The error bars in both figures denote one standard error of estimation.

Figure 5 demonstrates the interdependence of the estimated beta weights for Form, Blank, and Form × Blank across subjects. These values were obtained by applying the fixed effect part of Model 1 as a separate logistic regression to each subject's data. In the left panel, the size of the form change effect can be seen to be strongly correlated with the size of the blanking effect across subjects (ρ = 0.90, slope = 0.40). The right panel shows the relation between the form change effect and its interaction with the blanking effect (ρ = −0.75, slope = −0.44). Thus, while the absolute beta values may vary considerably, their relative values are consistent across subjects. 

Figure 5

 

(a) The estimated Form and Blank beta weights for individual subjects. Analogous to Figure 3, the interrupted blue line illustrates the absence of a form change effect, the full blue line illustrates a form change effect equal to the blanking effect. The interrupted green line is a linear fit on these individual subject data, whereas the triangle indicates the estimated average Form and Blank effects of Table 1. The form change effect can be described as a relatively fixed proportion of the blanking effect across subjects. (b) The relation between the Form and Form × Blank beta weights across subjects. The dotted orange line indicates the additivity hypothesis, the full blue line is equal to the absence of any form change effect on top of the blanking effect. The full green line is a linear fit to the estimated individual beta weights, whereas the triangle again indicates the estimated average beta weights of Table 1. All subjects but one are compatible with a strongly negative relation between the Form and Form × Blank effects. Error bars in both figures indicate one standard error of estimation.

Discussion

Referring back to the four initial questions, the following findings were critical. 

First, the blanking effect of Deubel et al. (1996) is replicated in the present study. After a postsaccadic blank, intrasaccadic displacements of the saccade target object were discriminated between with a higher average accuracy than in conditions without blanking. 

Second, a clear effect of form change is present, as manifested in improved displacement discrimination performance on trials where an intrasaccadic form change was applied, when compared to non-blanked conditions without such a form change. This suggests that discontinuous stimulus properties pertaining to its visual form can reject an assumption of visual stability relating to its position and disrupt the transsaccadic stabilization process. A more veridical representation of the presaccadic stimulus position then becomes available for solving the task, probably based on extra-retinal signals. While this representation might still be perturbed by imperfect compensation for the retinal image shift, in the present experimental task it clearly does allow for a more accurate displacement discrimination performance. Importantly, the form changes used were qualitative in nature and had a clear, common center relative to which displacements were identifiable between different stimuli; if these conditions are not met, we suspect the form change effect could be absent because a smaller form change would not signal a disruption of visual stability, or could be severely diminished or even reversed because a form change could act as a noise factor on the attribution of a given stimulus to a single spatial coordinate. 

Third, the results indicate that a form change was less effective in improving performance than a blank. One possible explanation could be that across different types of form changes (for instance, square to circle versus square to diamond) an averaging occurs of form change effects that are absent and other form change effects that are equal to the full blanking effect. Figure 4 and the analysis associated with it show that this is not the case, however. No significant variation across form changes was present, and all except one of the separate beta values are estimated to be at an intermediate value close to the overall average effect of a form change. A similar argument might be made for individual subjects, where some subjects might show a form change effect equal to the blanking effect whereas others show none. Figure 5 instead paints a rather different picture: The form change effect is a relatively consistent proportion of the blanking effect within each individual subject. Thus, we conclude that there is in the present data an overall intermediate effect of a form change compared to a postsaccadic blank. This implies that at least the type of form changes we used result in a weaker facilitative effect than could theoretically have been achieved on the basis of the spatial representation present in transsaccadic memory, as evidenced by the superior performance of blanking conditions. 

Fourth, even though the simultaneous presence of postsaccadic target blanking and a form change to the saccade target object diminishes the sum of their effects by about two thirds of the form change effect, the formal statistical rejection of the additivity hypothesis (β ₂₃ < 0) was not significant. Only a weaker version of this hypothesis rejection—more than half of the individual β ₂₃ estimates for different form change types are negative—could be demonstrated to be present. The high performance induced by the blanking effect alone makes it difficult to obtain reliable statistical estimates for the interaction effect. In addition, it is unclear whether the improvement expected from a full additivity hypothesis could ever have been achieved. Possibly the precision of the transsaccadic positional representation limits the maximal discrimination performance for a given displacement size. 

In the next experiment, we will address this issue by studying the interaction between postsaccadic blanking duration and the presence of a form change. It is known from previous studies (Deubel et al., 1996) that the blanking effect occurs for blanking durations as short as 50 ms but does not reach its maximal potential for improving performance until around 250 ms. Thus, the combined effect of a form change and a shorter blank can be compared against the potential for improvement as observed using a longer blanking duration. In addition, we will measure how well subjects could detect the presence of the form changes, blanks, and displacements employed in this experiment. This should provide explicit support to what we have up to now assumed to be true, namely that form changes and blanks were easily noticed by subjects, whereas failing to discriminate between displacement directions in the absence of either of these manipulations was underlain by subjects having perceived an unmoving stimulus. 

Experiment 2

Methods

Five male subjects between 26 and 37 years old participated in this experiment, of which one was the first author. The apparatus was identical and the procedure similar to that of Experiment 1. However, a larger saccade size of 8° was used, and the screen was moved closer to the subject. At a distance of 60 cm, it now occupied 32 by 24 visual degrees of the subject's field of view. This was done to allow finer grained displacements at the screen resolution used and to make the task slightly more difficult. In addition, we selected only two stimulus forms for this experiment, square and cross, in order to minimize irrelevant potential variability in the data. In a 2 × 2 × 4 factorial design, we combined the presence or absence of a form change with two different displacement sizes (0.08° and 0.16°) and four blanking durations (0, 50, 100, and 250 ms). Around 30 ms of the blanks overlapped with the saccade itself. In total, each subject performed 100 trials in each of these conditions, amounting to 1600 trials across three to four 1-h sessions. The median saccadic latency was 206 ms, with 95% of all latencies shorter than 302 ms. Less than 1% of all trials was aborted twice due to late responses, incorrect eye movements, or measurement errors. These trials were not included in the analyses. 

After these sessions, additional change detection (rather than discrimination of the direction of the change) measurements were done on the same five subjects, using the same stimulus presentation procedure. The aim was to gauge to what degree subjects were aware of the different manipulations applied to the stability or continuity of stimulus presentation. First, a single 100 trial block was run, without blanks but including both displacement sizes, in which subjects were to detect whether a form change had been absent (left button) or present (right button). Square-to-cross and cross-to-square changes were used on 50% of the trials, whereas on the remaining 50% stimulus form remained stable at either a square or a cross. Second, 400 trials were run in which the saccade target object could either be displaced (50% of all trials, either 0.08° or 0.16°) or spatially stable (50% of all trials). This manipulation was factorially combined with either a 0-ms (that is, no) or a 50-ms blank. Object forms did not change and were either squares or crosses. Subjects were instructed to respond whether the saccade target object had been stable (left breaker) or unstable (right breaker). Both stimulus displacements and stimulus interruptions counted as an instability for the purpose of this task. Subjects were explicitly made aware that only 25% of all trials warranted “stable” responses, and that ideally they should respond with the left button only in around one fourth of trials. This was done to induce some conservatism in giving a “stable” response given the clear displacements and long blanks subjects had been perceiving previously. The median saccadic latency on these additional measurements was 190 ms, with 95% of all latencies shorter than 273 ms; 2.6% of all trials were dropped from the analysis after being aborted twice. 

Results

Figure 6 shows the proportion of correct responses on the main task of the experiment, in which the direction of displacement was to be identified, for both displacement sizes. A separate Form by Size factorial mixed model logistic regression was applied for each blanking duration. The beta weights of Form contributed significantly for blanking durations of 0 ms (z = 4.01, p < 0.01), 50 ms (z = 2.09, p = 0.04), and 100 ms (z = 2.05, p = 0.04), but not of 250 ms (z = 1.05, p = 0.29). This reflects the effects for the smallest displacement size. Form interactions with a displacement size increase were significantly positive for 0-ms (z = 2.99, p < 0.01), marginally positive for 50-ms (z = 1.91, p = 0.06), and significantly negative for 100-ms blanking durations (z = −2.17, p = 0.03). The negative interaction effect at 250-ms blanking duration was insignificant (z = −1.4, p = 0.15). A form change effect can therefore be said to exist at 0-ms and 50-ms blanking durations for both displacement sizes, at 100-ms blanking duration for the smallest displacement size, and not at all at 250-ms blanking duration. Note how performance saturates at a much lower level for the 0.08° displacements. We conclude that the facilitative effects of form changes and blanks can combine with each other up to a certain internal performance ceiling, which is dependent on the displacement size. 

Figure 6

 

Proportion correct results for Experiment 2, subdivided by blanking duration. The colors and symbols have no relation to those of the previous figures. In the absence of a blank, a form change greatly improves performance. Longer blanking durations lead to less improvement, up to an internal ceiling performance dependent on the displacement size applied. Error bars indicate one standard error of between-subject variability for each data point.

Figure 7 shows the additional change detection data. When asked to detect a form change, subjects performed at 97% correct. Similarly, even a 50-ms blank was responded to as an instability in stimulus presentation in 98% of all trials when no simultaneous stimulus displacement was present, and 99% when there was. Subjects used the prior information provided, namely that only one fourth of all trials during the blank and displacement detection parts of the experiment was in reality stable, relatively well: Only 35% “stable” responses were given. However, this did not result in successful detection of either the 0.08° or 0.16° displacements: When pooled together with the actually stable trials to control for the false alarm rate, average performances of 49.08% and 51.49% correct were reached, respectively. We conclude that while subjects could easily detect the presence of the form change and blanking manipulations, they failed to detect any instability when these manipulations were absent, regardless of the exact displacement size. 

Figure 7

 

Additional change detection results for Experiment 2. The colors have no relation to those of the previous figures. Form changes and 50-ms blanks were almost always correctly detected, whereas displacements in the absence of either of these manipulations could not be distinguished from non-displaced trials, independent of the exact displacement size. Error bars denote one standard error of between-subject variability.

Discussion

The main finding of Experiment 1—form changes facilitate displacement discrimination performance—was clearly replicated in Experiment 2. This effect remained weaker than the full blanking effect induced by longer blanks but was equally effective as a short 50-ms blank. Relating these discrimination data to change detection rates, we observe that the failure to discriminate the direction of stimulus displacement in conditions without a blank or form change is indeed underlain by a perception of positional stability rather than positional uncertainty. That this is the case both for the smaller and the larger displacement sizes agrees well with the finding that whereas blanked performances differ greatly between these displacement sizes, their non-blanked discrimination performances are comparable. One striking finding is that in these subjectively “stable” trials, discrimination performance is still above chance. Possibly the forced choice of the discrimination task was weakly driven by other information when no displacement could be perceived, such as the initial saccade landing position against the displaced stimulus. In addition, we observe that the form changes and blanks that led to the facilitative effects described were themselves easily detected by all subjects. Combined with the earlier finding of submaximal facilitation for form changes and shorter blanking durations, this brings on the inevitable conclusion that successful explicit detection of an instability or discontinuity in stimulus presentation does not suffice to render an optimally precise transsaccadic representation of the saccade target object position available for use in a transsaccadic displacement discrimination task. 

For these relatively small displacements, the maximally attainable performance in the discrimination task was strongly limited by the displacement size. This is easily explained as being the resultant of positional noise induced by either the poor visual acuity of peripheral vision, or the mechanisms compensating the retinal image shift based on a corollary discharge of the saccade motor command; both can be expected to affect the discrimination of small displacements more than that of larger displacements. It is clear from the data that form changes can improve performance on top of blanks up to this internal ceiling performance. This renders the search for formal statistical additivity rather fruitless, since the individual effects of form changes and blanks are too strong to have a predicted additive performance below the empirically observed actual ceiling performance. However, both effects do combine with one another into a performance level superior to each isolated effect, in both displacement size conditions. This leads to the same conclusion as the previous paragraph: A simple instability detector, granting full access to a maximally detailed and precise positional representation as soon as any sort of discontinuity in stimulus presentation is detected at saccade landing, does not suffice to explain the data. Otherwise, performance should not improve further by the presence of a form change after the presence of a blank had already been detected. We will return to this in the General discussion section. 

General discussion

Object form subserves visual stability

MacKay (1972) already proposed that eye movements should be seen as questions, answered by their resulting retinal information. Decades later, the scientific consensus appears to have indeed converged on this intuition: Across the saccadic interruption of useful visual input, the visual system will assume that previously stable objects have remained stable, unless convincing evidence to the contrary is present at saccade landing. Under natural circumstances, this is a reasonable assumption, as it is improbable that otherwise stable objects would displace themselves in exact synchrony with the observer's saccadic eye movements during scene exploration, and rather more likely that the visual system's extra-retinal compensation mechanisms for saccade-induced retinal image shifts are imperfect. The breakthrough finding of Deubel et al. (1996) that this null hypothesis of visual stability could be disrupted under the specific experimental condition of postsaccadic target blanking provided direct evidence for this view. The role of the corollary discharge for the stabilization of everyday vision is thereby reduced to that of a predictor of the impending postsaccadic visual input, specifying the null hypothesis to be tested by retinal information. It has long been hypothesized that certain form features of objects in the saccade landing region could be factored into these spatial updating mechanisms of transsaccadic vision (e.g., Deubel et al., 1984; McConkie & Currie, 1996). Here, we offer the first empirical proof that this is indeed the case: Subjects were significantly better at discriminating between displacement directions when saccade target objects changed form as well as position, implying that these form changes contributed to the abolition of an illusion of positional stability, which was strongly present when these displaced objects also remained stable in their form properties. 

However, the current results in addition demonstrate that successful transsaccadic change detection by itself does not automatically give rise to the maximally attainable discrimination performance for a given displacement size. This is true for form changes when compared to blanks, but also for shorter blanks when compared to longer blanks, as already noted by Deubel et al. (1996). Moreover, though not formally additive, these facilitative effects do stack up to the empirically defined maximal task performance. Both of these facts argue against an explanation where any detected instability immediately renders available the maximally precise transsaccadic representation. Several explanations could be brought forward. First, successful explicit change detection might not always equal successful rejection of the visual stability null hypothesis, resulting in rejection rates well below 100% for form changes and shorter blanks, and in an additive increase in rejection rates when form changes and blanks are combined. Second, even when rejected in its most strict form, the visual stability assumption might continue to play a role as prior knowledge in determining the most likely displacement size given all other sources of information. Under this hypothesis, form changes or shorter blanks retain a greater weight for the stable world assumption in reaching a perceptual solution than is the case with longer blanking durations. This view agrees well with Niemeier et al.'s (2003) Bayesian approach to the visual stability problem and predicts graded apparent displacement effects. Third, additional mechanisms might be involved. A prime candidate is a masking effect on the contents of detailed transsaccadic memory, as proposed by De Graef and Verfaillie (2002) and empirically supported by Germeys, De Graef, Van Eccelpoel, and Verfaillie (under revision; see also McRae, Butler, & Popiel, 1987). The transsaccadic representation of object position and form would then remain available to a greater degree of precision when more read-out time to a masking-resilient memory store is provided through longer blanks, or when dissimilar form information is a less efficient mask than identical form information. 

The less than perfect correspondence between, on the one hand, the explicit detection of a form change to a transsaccadically present object and, on the other hand, the degree to which the discrimination performance of the object's displacement is improved leaves the possibility open that not every easily detectable form or feature change to the saccade target object will result in facilitative effects such as we observed. Indeed, McConkie and Currie (1996) in their saccade target object theory already proposed that only a subset of critical locating features would constitute a relevant input to the spatial stabilization mechanisms of transsaccadic vision. More research will be needed before these relevant inputs can be characterized exactly and exhaustively, however. 

Form discontinuities and the landmark effect

An interesting contrast between our present results and those of Deubel et al. (2002) can be noted. In their study, the postsaccadic target blanking effect could be reverted by intrasaccadically inserting flanking objects around the displaced position of a blanked saccade target object. Therefore, it appears as if the position of these flankers was taken as an indication of the position of the target object itself, despite the obvious form dissimilarity. That is, they served as a landmark for the saccade target object position in the absence of the saccade target object itself. In the present study, on the other hand, we have observed that the visual system does not treat the saccade target object as loosely as its own spatial reference when it is discontinuous in its form properties. 

One could speculate that the form change dimension used by Deubel and his colleagues did not include any “critical locating features,” whereas ours did. However, a more plausible explanation is still to be found in the recent work of Higgins and Wang (2010). These authors suggest that landmark and blanking effects rely on different mechanisms altogether, based on the observation that flanking landmarks affect the veridicality of spatial representation similarly both across and within fixations, whereas the blanking effect of isolated saccade target objects is only facilitative in transsaccadic vision. Seeing how landmark effects therefore occur in the clear presence of the visual transient associated with a stimulus displacement—which is masked by the saccade itself in a classical transsaccadic paradigm—it would appear that they do not require an assumption of visual stability to be upheld at all. 

What is specific about the transsaccadic situation is that it involves the visual system in a leap of faith across an interruption of input, prompting it to recruit a mechanism aimed specifically at testing the validity of the assumptions made. Interestingly, failure to verify these assumptions for an isolated postsaccadic object such as we use does then not lead to a fallback on landmark mechanisms, even though they have been demonstrated to operate even in the presence of clear stimulus discontinuities. However, while the isolated postsaccadic object will not serve as its own landmark, Deubel, Koch, and Bridgeman (2010) did recently demonstrate that a transsaccadically blanked stimulus can still serve as a landmark for a second stimulus with a longer postsaccadic blanking duration. We theorize that the form-unspecific landmark effect is essentially a two-step process (identify potential landmark, use it at the next stimulus onset), which does not apply to paradigms with a single postsaccadic stimulus presentation. There, only the form-specific visual stability assumption is relevant. 

Puzzling together the scene

Observers not only experience the visual world as being stable but also as being widely detailed, despite the anatomical reality that the human retina only allows high-acuity processing in a small central part of the visual field. O'Regan (1992) argued that this seemingly wide field of view is but an illusion drawn up by the visual system, and that no detailed transsaccadic representation and no quantitative solution for its integration into postsaccadic vision is needed to explain the human perceptual experience. Since attention itself restricts what we are aware of, and attention is closely linked to saccadic eye movements, wondering about a specific part of the visual scene will automatically bring about a detailed representation retrieved from the external world itself. The blanking effect demonstrated that a far greater amount of information is retained across saccades than change detection tasks would indicate, offering evidence against the idea of such sparse transsaccadic representation. 

However, O'Regan's view still has merit. It is indeed equally clear from the current as well as previous studies on the visual stability problem that transsaccadic perception cannot be equated with passively combining snapshots of the visual input, but that it is actively involved in constructing illusions optimized for natural situations. We would argue that transsaccadic perception of object form and features can be seen in a similar light. The fact that far more visual detail is retained across saccadic eye movements than is evident from classical change detection tasks (Demeyer, De Graef, Wagemans, & Verfaillie, 2010; Deubel et al., 2002; Germeys et al., under revision) does not imply that a master scene representation of photograph-like qualities is being filled with local visual detail as foveal vision collects it across successive fixations. What it does allow is for detailed perceptual solutions reached during the previous fixation to remain relevant for postsaccadic processing down to the level of detailed visual form (Demeyer, De Graef, Wagemans, & Verfaillie, 2009; Demeyer et al., 2010; Melcher, 2005), and thus provide processing speed benefits as well as transsaccadic representational continuity. The illusion of a widely detailed scene representation, richly populated with stable and unified transsaccadic object representations, is then the resultant of both the tunnel vision imposed by spatial attention and the carryover of information on detailed perceptual solutions into the next fixation(s). 

Conclusions

We empirically confirm that the human visual system employs the saccade target object form to ensure a perceptual experience of visual stability, and that this process is disrupted when object form is discontinuous across the saccade. A more veridical perception of artificially applied intrasaccadic displacements then ensues. Thus, a rejection of the visual system's assumption of stability generalizes from object form to object position. The exact mechanisms involved in reaching a transsaccadic spatial representation when stability is rejected as well as the exact set of object features underlying the saccade target object relocalization process remain fascinating topics for future research. 

Acknowledgments

This research was supported by the Concerted Research Effort Convention GOA of the Research Fund K.U. Leuven (GOA/2005/03-TBA) granted to Géry d'Ydewalle, Karl Verfaillie, and Johan Wagemans, by the European Community through GazeCom Project IST-C-033816 to Karl Verfaillie and Peter De Graef, and by a Methusalem grant to Johan Wagemans (METH/08/02). 

Commercial relationships: none. 

Corresponding author: Maarten Demeyer. 

Email: [e-mail]. 

Address: Laboratory of Experimental Psychology, Tiensestraat 102, B-3000 Leuven, Belgium.