Transsaccadic identification of highly similar artificial shapes

Abstract

Multiple times per second, the visual system succeeds in making a seamless transition between presaccadic and postsaccadic perception. The nature of the transsaccadic representation needed to support this was commonly thought to be sparse and abstract. However, recent studies have suggested that detailed visual information is transferred across saccades as well. Here, we seek to confirm that preview effects of visual detail on postsaccadic perception do indeed occur. We presented subjects with highly similar artificial shapes, preceded by a congruent, an incongruent, or no preview. Postsaccadic recognition performance was measured, while the contrast of presaccadic and postsaccadic stimuli was manipulated independently. The results show that congruent previews provided a benefit to the recognition performance of postsaccadic stimuli, compared to no-preview conditions. Incongruent previews induced a recognition accuracy cost, combined with a recognition speed benefit. A second experiment showed that these effects can disappear when stimulus presentation is interrupted with a postsaccadic visual mask. We conclude that visual detail contained in transsaccadic memory can affect the postsaccadic percept. Furthermore, we find that the transsaccadic representation supporting this process is contrast-independent, but that postsaccadic contrast, through its effect on the reliability of information, can affect the degree to which it is employed.

Introduction

The anatomy of the human visual system is such that only a small, central part of the retina can deliver high-acuity visual information about the outside world. This limitation can be overcome by moving this high-acuity region around the visual field by means of, on average, two to five saccades per second. This poses a new problem to the visual system, however: how can we achieve continuity and stability in visual perception from such a spatially unstable input signal? 

A key part in solving this problem is to understand what information, if any, is retained from one fixation to the next in what is often called transsaccadic memory. Initial speculation that a fully detailed pattern-level integrative visual buffer would exist was quickly refuted by a number of studies (Bridgeman & Mayer, 1983; Irwin, Yantis, & Jonides, 1983; Irwin, Zacks, & Brown, 1990; O'Regan & Lévy-Schoen, 1983; Rayner & Pollatsek, 1983). What kind of memory could then instead support transsaccadic integration has since been the subject of a long and rich line of research that can roughly be categorized by the experimental task used. 

First, change detection tasks require the subject to detect a discrepancy between a presaccadic and a postsaccadic stimulus. Inferences about the contents of transsaccadic memory then rely on the assumption that discrimination performance reflects the amount of task-relevant presaccadic information that is retained until after the saccade. The classical finding in this line of research is that intrasaccadic displacements of stimuli are significantly more difficult to detect than displacements occurring during a fixation (Bridgeman, Hendry, & Stark, 1975), leading to the speculation that little or no visual information is retained across saccades, and perception starts from scratch following each saccade (Bridgeman, Van Der Heijden, & Velichkovsky, 1994). However, Verfaillie, De Troy, and Van Rensbergen (1994) noted in their study on transsaccadic perception of biological motion that while position information was poorly retained, detection of changes in orientation was only slightly affected by the presence of a saccade. Moreover, Verfaillie (1997) showed that detection of intrasaccadic position changes could be moderately improved by providing more biological motion walkers as reference objects (also see Deubel, 2004). Measuring change detection during visual scene exploration, Hollingworth and Henderson (2002; Henderson & Hollingworth, 2003) found that even within-category object changes could be detected rather well, though global changes in scene luminance or contrast were still almost imperceptible (Henderson, Brockmole, & Gajewski, 2008). But perhaps the most striking finding was reported by Deubel, Schneider, and Bridgeman (1996), who showed that briefly blanking the display immediately after saccade landing could improve intrasaccadic change detection performance of stimulus location significantly (also see Gysen, Verfaillie, & De Graef, 2002). In another study (Deubel, Schneider, & Bridgeman, 2002), it was shown that this blanking effect could be generalized to change detection of stimulus identity. This suggests that the results of change detection tasks are not so much a function of whether information was retained in transsaccadic memory, but rather whether it was retained and subsequently available for comparison to the postsaccadic stimulus. De Graef and Verfaillie (2002) proposed that a detailed visual store exists that could support transsaccadic integration, the so-called visual analog. Also described as “informational persistence”, this memory store is precategorical, short-lived (up to a few hundred milliseconds), and maskable (Irwin & Yeomans, 1986). In further experiments, Germeys, De Graef, Van Eccelpoel, and Verfaillie (submitted for publication) presented subjects with circular arrays of letters and observed that presenting a short location cue during a blank interval immediately following the saccade could drastically improve detection performance of changes to individual items within such an array. They propose that this cue and blank interval are needed to quickly read out detailed information from the visual analog to a more durable store, before its contents are masked away by the postsaccadic stimulus. In general, this line of research can be summarized as showing that the inability to detect small intrasaccadic changes does not necessarily indicate the absence of the task-relevant visual information in transsaccadic memory. 

Second, adaptation aftereffect tasks have recently been employed. Melcher (2005) asked his subjects to fixate an adaptor stimulus for several seconds before initiating a saccade and found that spatially matched postsaccadic test stimuli showed an adaptation aftereffect that increased with stimulus complexity but was absent for stimuli that only differed in contrast, suggesting that the transsaccadic memory representation involved has already abstracted contrast from the input. For non-spatially matched stimuli, a similar but much smaller effect was found. In another task, Melcher (2007) observed that this transfer of adaptation to the postsaccadic location already started before the actual execution of the saccade. Visual adaptation itself cannot be considered sufficient to support transsaccadic information transfer, but these results do suggest that pre- and postsaccadic stimuli, though retinotopically in a different location, use common visual neurons for processing. The author proposed that this is due to perisaccadic remapping of their receptive fields to a new retinal location, as described in neurophysiological literature (Colby, Duhamel, & Goldberg, 1996; Duhamel, Colby, & Goldberg, 1992), and speculated that it is this mechanism that supports transsaccadic information transfer and integration, and that constitutes a more visually detailed transsaccadic memory. 

Third, postsaccadic identification tasks require the subject to identify the postsaccadic stimulus and rely on the assumption that the presence of presaccadic information in transsaccadic memory will affect task performance. As such, this task not only probes the contents of transsaccadic memory but also measures the functional advantage it can give to an observer during postsaccadic stimulus identification. This advantage is called a transsaccadic preview benefit. Pollatsek, Rayner, and Collins (1984) observed such a benefit by comparing latencies to name drawings of realistic objects between conditions where the same line drawing was shown presaccadically and conditions where a different line drawing was shown. They found that observers reacted fastest when the preview image was identical and also found significant facilitation when the general visual shape of both stimuli was similar. In a subsequent study (Pollatsek, Rayner, & Henderson, 1990), the authors showed that this facilitation was independent of intrasaccadic location changes. The latter finding has since been under dispute, however (Gajewski & Henderson, 2005; Germeys, De Graef, & Verfaillie, 2002; Henderson & Anes, 1994). 

Several of these studies have concluded that transsaccadic memory only contains a more sparse and abstract representation of the presaccadic stimulus, often described as similar to that of regular visual short-term memory (Carlson-Radvansky, 1999; Carlson-Radvanksy & Irwin, 1995; Irwin, 1991, 1992; Prime, Tsotsos, Keith, & Crawford, 2007). For instance, Henderson (1994) and Rayner, McConkie, and Zola (1980) both found that it is not necessary for letters to have the same case before and after a saccade in order to find the expected facilitation. In addition to this, Henderson (1997) observed that, when presenting subjects with line drawings partly obscured by vertical bars (a “picket fence” display), changing the visible part of the contour during a saccade does not slow down postsaccadic recognition. This led him to conclude that visual contour information is not retained transsaccadically. 

However, Melcher and Morrone (2003) argued more recently that many stimuli used in these studies have only a short “integrative window”—the time it takes to piece together enough visual information about a single stimulus to identify it. Transsaccadic integration of visual information can therefore not be observed, because this window does not extend into the next fixation. Measuring motion coherence thresholds of random dot motion displays, of which it is known that they have a long integrative window, they were able to prove that below-threshold pre- and postsaccadic signals could be integrated across a saccade into an above-threshold percept. The observation of preview benefits situated at a visual level may therefore be conditional on the exact manner of stimulus presentation. 

In sum, the established view that transsaccadic perception is mediated only by a sparse and abstract representation has recently been challenged by various studies. At the same time, the evidence that full sensory integration across saccades is not possible is still strong and has been strengthened further by findings of insensitivity of transsaccadic representations to low-level visual properties such as luminance or contrast. If visually detailed information is indeed retained across a saccade, it is therefore likely to be situated at a more intermediate level of visual processing, allowing the retention of more than coarse structure but abstracting basic shape-irrelevant properties of the image. However in identification studies, arguably the natural transsaccadic task for the visual system, integration of presaccadic visual information in postsaccadic perception has so far only been shown using amorphous motion patterns (Melcher & Morrone, 2003) or, when line drawings of objects were used, for general (not detailed) object structure (Pollatsek et al., 1984). The question then remains whether the possibility of transsaccadic retention of visual detail shown in change detection tasks has a functional benefit for the identification of visual shape. There are two reasons to suspect this might not be the case. First, change detection studies suggest that it takes an artificial postsaccadic blank to prevent detailed transsaccadic information about visual form from being masked away by the postsaccadic stimulus. This could imply that rather than using this representation to its benefit by integrating it with what is found postsaccadically, it could simply be discarded. Second, there might be good reason to indeed discard this information, since it could prove a daunting task for the visual system to integrate visual form at a more detailed level than general structures, given the positional uncertainty transsaccadic integration has to account for (Niemeier, Crawford, & Tweed, 2003) and the coarser spatial resolution of peripheral vision. The main goal of the present study was therefore to establish whether visual detail perceived presaccadically could be integrated with postsaccadic visual information in a postsaccadic identification task. 

We used a task in line with the literature on transsaccadic preview benefits. As illustrated in Figure 1, subjects fixated on a small cross until after a random delay a peripheral stimulus appeared, to which they saccaded immediately. By requiring an immediate saccade, we minimized the opportunity to consciously identify the preview stimulus before the saccade. In practice, the saccade latency, and thus the presaccadic stimulus duration, was around 200 ms. During a saccade, the stimulus identity could be altered without the subjects noticing the change. After each trial, subjects manually responded which of the two known stimuli they had perceived in the postsaccadic interval. To prevent floor and ceiling effects from concealing preview effects, we measured a psychometric curve for each participant individually in the first session and selected a stimulus pair that yielded a discrimination performance of 85% correct for use in the remainder of the experiment. 

Figure 1

 

The procedure. Stimuli visible to the subject are visualized in black or gray colors, the eye position is in red, and in blue we provide some additional information that was not visible to the subjects during trials. Subjects initiated each trial by pressing a button. After that, a random fixation delay occurred (1), followed by the presaccadic stimulus that was also the cue to start a saccade (2). If the eye movement was initiated in time, the presaccadic stimulus would be replaced by the postsaccadic stimulus during the saccade (3). After saccade landing, subjects were briefly presented with a postsaccadic test stimulus (4), followed by a mask (5). Finally, they were required to respond within 2000 ms (6).

Our primary aim was to investigate what benefit a presaccadic preview can provide to postsaccadic identification of visual form, using highly similar artificial shapes devoid of semantic load that do not differ in coarse structural properties. If the presence of visually detailed information in transsaccadic memory can indeed provide a functional advantage to postsaccadic recognition, then we would expect to observe preview effects using these stimuli. Figure 2 shows some examples from our stimulus set, as located on the average psychometric curve of all participants. 

Figure 2

 

The average psychometric curve of all participants is shown, along with some examples of stimuli on the continuum. The blue and red boxes are a typical stimulus pair S0 and S1 used in the actual experiment, whereas the dark gray boxes are an example of S1 stimuli that on average had a discrimination performance from S0 that was lower or higher than 85%, respectively.

Second, we wanted to disentangle the preview benefit gained from an identical preview (congruent condition) from a potential cost due to being presented with an incorrect preview stimulus (incongruent condition), compared to a no-preview baseline. This distinction has not always been made consistently in the literature. Henderson, Pollatsek, and Rayner (1987) did show that a foveal retinotopic preview could result in a naming speed benefit when congruent, but not in a cost when incongruent, but they did not apply the incongruent manipulation to peripheral, spatiotopic preview stimuli as we use and as are natural in everyday perception. 

Third and finally, we manipulated the contrast of the pre- and postsaccadic stimulus intervals independently to be set at high or low. Ever since the rejection of the integrative buffer model for transsaccadic perception, contrast has been considered as a low-level visual property that is irrelevant for integration. Indeed, Melcher (2005), while showing transsaccadic transfer of adaptation effects for rather detailed visual shapes, could not find the same sort of transfer for contrast information. Similarly, Henderson et al. (2008) observed that subjects were blind to intrasaccadic changes in overall contrast when exploring a scene. Both findings suggest that contrast has already been abstracted from that part of the presaccadic information that is retained until after saccade landing, and that is subsequently compared to or integrated with the postsaccadic information. If our data are to be consistent with these findings, we should observe no loss of preview effects as a result of an intrasaccadic contrast change. That is, preview effects should not be contrast-specific, and no statistical interaction should occur involving the contrast of both stimulus intervals. However, the contrast of either the presaccadic or the postsaccadic display can still, independently, affect the size of the preview effects. For instance, the contrast of the presaccadic stimulus information could be expected to influence postsaccadic preview effects, but it would have the same additive effect regardless of whether the postsaccadic contrast is identical or not. Similarly, and more interestingly, the quality of the postsaccadic stimulus, as defined by the contrast of the display, could determine to what degree presaccadic information is employed in forming a percept. 

Experiment 1

Methods

Subjects

Four subjects (three males, one female) between the ages of 25 and 50 participated in this experiment, among which were two authors. All had normal or corrected-to-normal vision. 

Apparatus

Stimuli were presented on an Iiyama Vision Master Pro 541 22-inch monitor, with a temporal resolution of 200 Hz and a spatial resolution of 800 by 600 pixels. The monitor was placed at a distance of 135 cm from the observer, subtending a total range of 17 by 13 visual degrees. Gamma correction was done on this monitor prior to the experiment. Eye movement data were obtained from each subject's left eye using a Dual Purkinje Image analog eye tracker and processed by internally developed software on a Windows XP platform. Stimulus presentation and analog-to-digital conversion of the analog eye movement signal was done using a Cambridge Research Systems Visage stimulus generator. 

Stimuli

As stimuli, we used 2D shapes defined by a sum of three radial frequency patterns (Wilkinson, Wilson, & Habak, 1998). Each radial frequency pattern consists of a sine wave converted to radial coordinates to create a closed figure with a diffuse Gaussian border. The three components that were summed had a frequency of 10, 5, and 5 cycles per 360°. The phase of the latter two components was fixed at π/2 and π, respectively, while the phase of the first component was used to manipulate the shape of the stimulus. It could vary between 0 and 4π/10. In the center of the figure, a dot was present that could change color from black to white as a cue to initiate the saccade. Figure 2 shows stimuli on this continuum. Each stimulus had an apparent diameter of 1.5 by 1.5 visual degrees. For the high contrast conditions, we used a Michelson contrast of 0.4; for the low contrast conditions, this was 0.2. 

Procedure

We used a three-way fully factorial design. There were three levels to the congruency factor (no preview, incongruent preview, congruent preview) and two levels to both the presaccadic contrast and postsaccadic contrast factors (high contrast and low contrast). 

The task used is known as a Yes/No task. Figure 1 illustrates the procedure. On each trial, subjects were asked to identify the postsaccadic stimulus as one of two known stimuli. On congruent trials, the same stimulus was shown before and after the saccade; on incongruent trials, the other of the two target stimuli was shown as the presaccadic stimulus; and finally, on no-preview trials, only a dot was present presaccadically. Before each block, the subject could freely study both stimuli and learn the association of the left and the right response button to their respective stimuli. At the start of a trial, the subject fixated on a fixation cross 2.1 visual degrees left of the center of the screen. Meanwhile, 2.1 visual degrees to the right of the center of the screen, a black dot was present. After a random delay of 500 to 1000 ms, this dot turned white and if a preview stimulus was provided, it appeared centered around this dot, with a random rotation. The dot's change in color was the subject's signal to initiate a saccade toward the dot. If the subject failed to do so within 150 to 400 ms, the trial was aborted. In practice, 90% of the trials had a saccadic latency smaller than 248 ms, with a median of 207 ms. Once the saccade was initiated, the postsaccadic stimulus appeared at the saccade target location in the same random rotation, replacing the presaccadic display. If the saccade did not reach its destination within 35 ms, the trial was again aborted. The stimulus stayed on the screen for 85 ms after saccade onset, followed by a 200-ms mask. This mask consisted of a grid of small squares, the size of the stimulus itself, that had a random luminance of either the highest or the lowest possible luminance in the stimulus (as defined by the stimulus contrast), or the intermediate background luminance. Within 2000 ms after the mask offset, the subject had to press either the right or the left response button to answer. Reaction times were also measured, relative to this mask offset. Aborted trials were recycled afterward; trials that were aborted again during recycling, however, were not repeated anymore. A total of 1.7% of all experimental trials (excluding the stimulus calibration phase) was aborted twice, and thus, no data are available for these trials. 

Prior to the main experiment, a pilot study was done to determine which stimuli to use for each subject, in order to avoid floor and ceiling effects. The same procedure as above was used but always in the congruent condition at high contrast both before and after the saccade. Discrimination performance was measured for six stimulus pairs of 100 trials each, in blocks of 50 trials and with decreasing stimulus difference. After having fitted a Weibull psychometric curve to the data, we inferred which pair would theoretically yield a performance of 85% correctly answered trials. This pair of stimuli was used for the main experiment. Since this pilot was done in presumably the easiest condition, this avoided a ceiling effect while still leaving enough room for performance-reducing effects to show in the data. 

After this initial session of 600 trials, two further sessions of equal length were done with the actual experimental trials. This resulted in 1200 experimental trials across 12 conditions—100 trials per condition. A total of 1800 trials was thus measured per subject, in 36 blocks of 50 trials. Subjects could rest for a few minutes after each block. 

Results

Figure 3 shows the proportion correct data. We performed a fully factorial three-way within-subjects analysis of variance (ANOVA), using the interactions with the subject variable as error term. A main effect for congruency ( F(2,6) = 12.16, p < 0.01) and for postsaccadic contrast ( F(1,6) = 31.89, p < 0.05), and an interaction between postsaccadic contrast and congruency ( F(2,6) = 34.60, p < 0.01) were observed. The main effect of presaccadic contrast ( F(1,6) = 1.46, p = 0.31) and all other interactions were not significant, including the interaction between presaccadic and postsaccadic contrasts ( F(1,3) < 1) and the three-way interaction between presaccadic contrast, postsaccadic contrast, and congruency ( F(2,6) < 1). 

Figure 3

 

Proportion correct results for all subjects individually and averaged over subjects. Whiskers show the 95% confidence intervals of the means. The X-axis contains the four possible combinations of stimulus contrast: low contrast before and after the saccade (LL), high contrast before and low contrast after the saccade (HL), low contrast before and high contrast after the saccade (LH), and high contrast before and after the saccade (HH). The three data points at each contrast condition denote the incongruent (left bar), the no-preview (middle point), and the congruent conditions (right bar), respectively. The red line lies at chance performance. (a) The conjoined data. (b) The individual data.

A Tukey HSD analysis on all means in the interaction between congruency and postsaccadic contrast revealed that in the absence of a preview, higher postsaccadic contrast resulted in higher recognition performance ( p < 0.01). In the high postsaccadic contrast conditions, a congruent preview provided a benefit relative to the no-preview baseline ( p < 0.01), and an incongruent preview resulted in a cost ( p < 0.01). The same was true for the low postsaccadic contrast conditions ( p < 0.05 and p < 0.01, respectively). However, both the benefit and the cost were significantly smaller when the postsaccadic contrast was higher (both p < 0.01). The benefit was reduced by a factor 2.06 and the cost by a factor 2.2 when increasing the Michelson contrast of the postsaccadic display from 0.2 to 0.4. Thus, doubling the postsaccadic contrast about halves both preview benefits and preview costs. 

Figure 3 also shows the details of the individual results. It can be seen that the overall effects of congruency were present in every subject. However, the presaccadic contrast manipulation seems to have had an inconsistent effect: the main effect of presaccadic contrast and its interaction with congruency were present in the data for subjects LB ( F(1,1174) = 5.77, p < 0.05 and F(2,1174) = 19.74, p < 0.01, respectively) and FG ( F(1,1179) = 10.08, p < 0.01 and F(2,1179) = 4.05, p < 0.05), but not at all for MD and PD (all F < 1). In subject LB, it can be seen that a cost rather than a benefit was observed in the congruent low-contrast preview conditions, similar in size to the cost of an incongruent preview. This suggests that this subject was simply unable to perceive the low-contrast previews properly. Subject FG, on the other hand, did show the expected benefit and cost pattern even in these conditions. The absence of effects of presaccadic contrast in the conjoined data of all subjects, although seemingly present upon visual inspection, is therefore mainly caused by a lack of consistency across subjects, increasing the error term in the overall within-subject ANOVA greatly. 

We then compared the proportion of correct answers to the reaction time data. A similar ANOVA design as above was applied to the logarithmically transformed reaction times to correctly responded trials. This analysis yielded only a main effect of congruency ( F(2,6) = 66.99, p < 0.01) and no effects of contrast. A Tukey HSD on the means of the congruency conditions revealed that reaction times in congruent conditions were significantly shorter than in incongruent conditions ( p < 0.05), which were in turn significantly faster than in no-preview conditions ( p < 0.01). Thus, we here observe a reaction time benefit even for an incongruent preview and an additional benefit for a congruent preview. Figure 4 illustrates these effects, along with their replication in every subject. 

Figure 4

 

Reaction time results for the significant main effect of congruency, for all subjects individually and averaged over subjects. Again, the three data points for each subject denote the incongruent, the no-preview, and the congruent conditions, respectively.

In short, these analyses show definitive preview effects of incongruent and congruent presaccadic stimuli on both dependent variables. The data also contain strong evidence concerning the effect of postsaccadic contrast on the proportion of correctly identified stimuli, increasing overall performance but reducing preview effects as the postsaccadic contrast is higher. 

Discussion

We have observed that postsaccadic, foveal identification of highly similar artificial shapes can be made more accurate and faster by presenting a presaccadic, peripheral preview of the same shape at the same location (i.e., a congruent preview). This shows that a preview of the detailed visual information required to solve the task can provide a functional advantage to postsaccadic recognition of shape. We also demonstrated that a similar but incongruent spatiotopic peripheral preview can decrease the accuracy of postsaccadic identification compared to a no-preview baseline. Interestingly, this incongruent preview still provides a benefit to the speed at which successful identification happens, albeit less so than a congruent preview. 

A main effect of postsaccadic contrast on the proportion of correct responses was consistently present. Because this is easily understood as a result of the increased visibility of the higher contrast postsaccadic stimulus, it is not particularly informative with regard to a better understanding of transsaccadic perception. Similarly, the decreased preview benefits observed for higher postsaccadic contrast may not be very informative, because the opportunity to observe benefits is simply more restricted when baseline performance is higher. The reduced preview costs in the higher contrast incongruent conditions, however, do point at a lesser weight of the presaccadic stimulus in the identification process as postsaccadic stimulus quality increases. This suggests that the visual system only allows the presaccadic stimulus to affect the postsaccadic percept in so far as postsaccadic perception is not reliable enough by itself. 

For the main effect of presaccadic contrast on the other hand, we must conclude that no consistent evidence was found that a better quality preview leads to larger preview effects in identification accuracy, although some subjects did individually show results in this direction. We suspect that this variability across subjects is due to individual differences in the amount of information that could be extracted from the low-contrast preview, combined with a ceiling on the impact any preview could have. Thus, the lack of a consistent presaccadic preview contrast effect could mean that preview effects are not linear across the entire range of subjective visibility. 

Our final observation on transsaccadic contrast effects is that there was no evidence of contrast specificity in transsaccadic preview effects. If this were the case, we would have expected an interaction between presaccadic contrast, postsaccadic contrast, and congruency, with congruency effects decreasing as presaccadic and postsaccadic contrasts differ more. We observed no such effects: An intrasaccadic change in contrast did not affect transsaccadic integration. This finding is in line with the previously cited observations in the literature that the transsaccadic representation makes abstraction of low-level contrast information. 

In our opinion, the most important conclusion to be drawn from these results is that they show preview benefits relying on quite small differences between highly similar stimuli in postsaccadic identification tasks. This goes beyond the preview benefits of general stimulus form, as reported by Pollatsek et al. (1984), and is consistent with what could be expected if the contents of a visually detailed transsaccadic store were indeed used for transsaccadic integration. In addition, our study reveals costs and benefits associated with an incorrect preview. Contrary to previous studies with retinotopic foveal previews, where an incorrect preview did not enhance nor impair postsaccadic recognition speed (Henderson et al., 1987), we found that an incongruent spatiotopic preview produced costs for identification accuracy and a preview benefit for identification speed. 

Looking closer at these reaction time data, they reveal two distinct results. First, there is a benefit of having a preview, be it congruent or incongruent, over having no preview. This finding can be seen as similar to what Pollatsek et al. (1984) found, namely that sharing a general visual shape, even if the visual details and the semantic load are different, already provides a preview benefit in reaction time. Second, there is an additional benefit of presenting a congruent preview when compared to an incongruent preview. Preview effects on reaction times seem not to be limited to global visual properties but can also be dependent on visual detail. The non-monotonic relationship between speed and accuracy argues against a simple trade-off and strengthens the idea that an incongruent preview stimulus does not merely serve as a forward mask to postsaccadic perception. Instead, it appears to be a part of the recognition process. Together with the below-chance performance we observe on three out of four subjects in the incongruent HL condition, this suggests an integration of both sources of information into a single percept that is inclined toward the incongruent preview. 

Evidently, more research is needed on the exact mechanisms that underlie these graded preview benefits on postsaccadic identification speed, with the largest benefit for identical previews and smaller benefits for similar but subtly different previews. In an account of transsaccadic scene perception, De Graef (2005) suggests that such mechanisms may operate along the lines indicated by Bar (2003, 2004). Bar proposes that fast activation of a select few templates using only coarse information is followed by slower recurrent processing to fill in the details and recognize the precise stimulus correctly. Applied to our present study, this could imply that both preview stimuli (congruent and incongruent) offered a processing speed gain because they both activated the two coarsely similar target templates relevant to the task at hand. Slightly slower identification times in the incongruent preview condition would then result from longer recurrent processing required to resolve the differences between the visual detail stored in transsaccadic memory and the visual detail encoded from the postsaccadic stimulus. Note that this could also be a useful model to describe the moderating effect of higher postsaccadic contrast on incongruent preview effects: If postsaccadic evidence is strong, it can quickly drown out the preview information before a conscious percept is reached. 

While this explanation of the reaction time data remains to be tested, we should point out that two alternative explanations can already be excluded. First, one could argue that the benefit of both preview conditions over the no-preview condition is induced by faster reaction times on the right button in those conditions. A large presaccadic stimulus could cause more motoric facilitation of the right-hand side than the small dot that was present presaccadically in the no-preview condition. The data, however, show that while right-hand answers were indeed on average about 15 ms faster, this did not vary across congruency conditions (the interaction is insignificant), and in any case the size of this effect would be too small to explain the effects of congruency on overall reaction time. Second, it is possible that the subject could, on a subset of the trials, identify the preview stimulus presaccadically and would start initiating a response based on this information. This strategy (although suboptimal with respect to response accuracy) would indeed lead to faster reaction times in both preview conditions and would also explain the larger proportion of incorrectly identified stimuli when the preview was incongruent. Henderson et al. (1987) rejected this possibility in their study, however, by showing that the expected bimodality of the reaction time distributions in conditions with a preview stimulus was absent. Figure 5 shows that this is also the case in our study: After removing the 5% slowest reaction times and offsetting the mean to 0 in each condition and for each subject, there is no difference in shape between preview and no-preview condition reaction time distributions. Thus, they differ only in their means and the data in the preview conditions are not an aggregate of different response distributions, or at least not more so than the data in the no-preview conditions. 

Figure 5

 

Reaction time distributions for preview and no-preview conditions. We plot the average proportion of trials situated in a specific reaction time bin, across subjects and across conditions, relative to the subject-specific mean of the condition that the trial belonged to. This way, we compare conditions with and without preview after correcting for differences in means between their constituent conditions and subjects. It can be seen that having a preview does not cause a greater spread in reaction times than not having one, suggesting that subjects do not sometimes respond directly to the preview stimulus when it is available.

In sum, we have established that transsaccadic preview effects based on visual detail rather than general structure can be observed. We believe that this is the result of a combination of pre- and postsaccadic stimulus information at the perceptual level, weighed according to the reliability of the postsaccadic display. However, even though the preview stimulus was presented only very briefly, it is possible that a quick presaccadic identification took place based on the then-present visual information. Our analysis of reaction time distributions suggested that the preview effects observed cannot be explained by direct responses to the presaccadic stimulus on a subset of trials. However, it is still a logical possibility that on every trial the observers did achieve presaccadic stimulus identification with a variable degree of confidence, and that this postperceptual judgment rather than a perceptual representation survives the saccade. This could in turn cause a bias on the postsaccadic response or on the interpretation of the uncertain contents of the (also very briefly presented) postsaccadic display and lead to preview effects similar to those observed in Experiment 1. 

To investigate this possibility, we conducted a control experiment in which we introduced an immediate postsaccadic mask preceding the postsaccadic stimulus. Such a mask would not affect any presaccadic processing, because it appears only after saccade initiation. At the same time, because it is a meaningless visual pattern, it should not affect the further postsaccadic retention of transsaccadically persistent, non-maskable information, such as a presaccadic categorization response. What it would affect is preview information stored in a visually maskable store, as well as any transsaccadic integration process that requires stimulus continuity. 

We therefore predicted that if indeed presaccadic identification and a subsequent response or interpretational bias were responsible for transsaccadic preview effects, then a brief immediate postsaccadic visual mask should not result in the attenuation of these effects. Alternatively, if preview effects are based on a perceptual combination of presaccadically and postsaccadically acquired stimulus information, then the postsaccadic mask should interrupt this process and result in smaller or absent preview effects. Our second experiment puts these predictions to the test. 

Experiment 2

Methods

Subjects

Four subjects, two authors and two subjects who did not participate in Experiment 1, participated in this experiment. All were male, in the age range of 25–50, and had normal or corrected-to-normal eyesight. 

Apparatus

Subject MD was run on the same setup as was used for Experiment 1. Due to a technical defect, however, the remaining three subjects were run in a different room, using a different Dual Purkinje Image eye tracker. As a consequence, the right eye was measured instead of the left eye. Other than this, the same equipment was used for analog-to-digital conversion and visual stimulation. 

Procedure

Again, subjects went through one session of stimulus calibration prior to the actual experimental trials. This session was identical to that of the first experiment. We set the stimulus contrast to those values that resulted in the largest preview effects in the previous experiment: a Michelson contrast of 0.4 for the presaccadic display and 0.2 for the postsaccadic display. Over the course of the next two sessions, behavior in four conditions in a fully crossed two-factor design was measured. The first factor, congruency, contained the levels “congruent” and “incongruent,” with similar meaning as in the first experiment. The second factor, immediate postsaccadic masking, was set to either “no masking” or “masking.” If stimulus presentation was not interrupted by a mask, it was exactly identical to that of the previous experiment. If it was interrupted, an object mask was presented for 100 ms as soon as saccade start was detected, followed by a blank display lasting 200 ms to prevent strong forward masking of the postsaccadic stimulus. The interrupting mask consisted of a single-component radial frequency pattern equal to the highest frequency component of the regular stimuli. After this, the postsaccadic stimulus and its mask were presented in the same way as in the “no masking” conditions. Notice that this means that the subject had no way of knowing before the saccade to which condition the current trial belonged, so he could not adapt his strategy accordingly until after the saccade had started. If presaccadic identification plays a prominent role in generating preview effects, it should already have taken place by then and thus occur equally for interrupted and non-interrupted trials. 

In total, 1600 trials were measured, of which 600 calibration trials and 1000 experimental trials. This results in 250 trials for each condition. In total, 6.9% of the trials was aborted twice and excluded from the analysis. 

Results

Figure 6 displays the results for each subject. The data show a nuanced pattern: In three subjects, a large decrease in preview effects can be seen when stimulus presentation was interrupted by a mask, whereas subject PD's performance was unaffected. For this reason, we decided not to do an overall ANOVA but analyze each subject separately. 

Figure 6

 

Proportion correct results for Experiment 2, per subject. The first two (gray) bars are, respectively, the congruent and incongruent conditions without interruption, the last two (white) bars illustrate the data of those trials where stimulus presentation was interrupted with a mask. Each pair of bars is plotted with respect to their average. There was no neutral condition in this experiment; unlike in the previous figures, the point at which they meet has no separate empirical meaning but only denotes the global average performance for that subject.

The main effect of congruency was present in all subjects: BS ( F(1,929) = 204.03, p < 0.01), GH ( F(1,852) = 83.40, p < 0.01), MD ( F(1,970) = 22.76, p < 0.01), and PD ( F(1,958) = 1620.09, p < 0.01). The main effect of postsaccadic masking was never significant, nor is it relevant for our purposes. The interaction between congruency and masking, however, was present for subjects BS ( F(1,929) = 114.34, p < 0.01), GH ( F(1,852) = 51.98, p < 0.01), and MD ( F(1,970) = 18.07, p < 0.01) but not for PD ( F(1,958) = 1.66, p = 0.2). As can be seen in Figure 6, when this interaction was significant it was caused by the combination of a large congruency effect in conditions without interruption ( p < 0.01), and the absence of a significant difference between congruent and incongruent means when stimulus presentation was interrupted ( p > 0.05 for all these three subjects). Given the tendencies present in the data, we do however not exclude that more statistical power could show a minor effect of congruency even here. 

Further analysis on the data of subject PD reveals an anomalous reaction time pattern in the interrupted conditions: 48.2% of his responses to these trials were done within 200 ms after the offset of visual stimulation, compared to on average of 3.7% for the other three subjects. This observation is especially remarkable given that the same subject had the slowest reaction times in the previous experiment and did not display an especially fast average RT in conditions without interruption in this experiment, either (356 ms for MD, 405 ms for GH, 479 ms for PD, 531 ms for BS). Moreover, PD's average overall performance of 52% correct did not differ significantly from chance ( t(961) = 1.23, p = 0.22), whereas it did for subject BS ( t(932) = 6.11, p < 0.01), GH ( t(855) = 8.39, p < 0.01), and MD ( t(973) = 16.43, p < 0.01). In sum, subject PD shows a pattern of results that is qualitatively different from the other three participants. 

Discussion

Three out of four participants showed a similar pattern of results: When a visual mask precedes the postsaccadic stimulus, preview effects on postsaccadic identification disappear or are severely reduced. The data of these participants suggest that the preview effects observed in Experiment 1 were not reliant on presaccadic identification of the preview stimulus and subsequent transsaccadic retention of this abstract identity information, either to guide the interpretation of the very briefly presented and therefore uncertain postsaccadic information, or to bias the response. Rather, the results suggest an interpretation of the preview effects as a consequence of transsaccadic integration of detailed shape information. The final participant (PD) was unaffected by the interrupting mask. However, PD's average recognition performance of the postsaccadic stimulus did not differ significantly from chance either with or without an interrupting mask. This indicates that this participant responded exclusively on the basis of the presaccadic stimulus. In addition, and in accordance with this explanation, his reaction times on interrupted trials were abnormally fast when compared to the other subjects. 

It could be suspected that an immediate postsaccadic mask would, even in the absence of transsaccadic integration, exercise an influence on preview effects through its modulation of the visibility of the postsaccadic stimulus. We attempted to avoid a sharply decreased visibility by introducing a brief blank after the presentation of the interrupting mask. Subject PD still appeared to be unable to perceive the postsaccadic stimulus, resulting in his reliance on presaccadic information, but the similarity in performance between interrupted and non-interrupted trials would indicate that this was the case regardless of postsaccadic masking. The other three subjects were able to use the postsaccadic stimulus information, as evidenced by their above-threshold average performance. Second, and more relevant to the interpretation of the results of these subjects, the masking manipulation and specifically its ensuing blank could have increased postsaccadic stimulus visibility. By manipulating stimulus contrast, we have shown in Experiment 1 that reduced postsaccadic uncertainty can lead to reduced preview effects. If the blank period following the immediate postsaccadic mask would overcompensate for possible forward masking of this stimulus, and decrease the uncertainty of the postsaccadic stimulus display, we could therefore observe a reduction of preview effects regardless of any interrupting effects the postsaccadic mask could have. However, the average performance on interrupted trials, while above-threshold, was still rather low and similar to that of uninterrupted trials. This makes it unlikely that the drastic disappearance of preview effects was caused by a more reliable perception of the postsaccadic stimulus display. 

Thus, we believe that the data of the three participants who did succeed in an above-chance recognition of the postsaccadic stimulus reflect an interruption of transsaccadic integration of detailed shape information by the visual mask. Specifically, two plausible reasons for this disruption can be advanced. 

First, the attenuation of preview effects could reflect backward masking of a transsaccadic store containing visual information about the preview stimulus. In apparent contradiction to this hypothesis, the data of participant PD show that he was able to identify the presaccadic stimulus despite the immediate postsaccadic mask. However, the memory store from which a successful identification process can complete is not necessarily the same store that supports the integration of its contents with a postsaccadic stimulus. A second possibility is that perceptual integration might be conditional on the likelihood that pre- and postsaccadic information originate from one and the same object, and therefore requires stimulus continuity. This does not exclude the first hypothesis of a maskable visual store but is not restricted to it either. The integration process itself should be considered to go beyond the pattern-level integration conceived in the now rejected visual buffer model (Bridgeman et al., 1975) and will possibly involve higher levels of abstraction, such as visual form descriptors or top-down interactions. Nevertheless, the fact that an interrupting visual mask reduces transsaccadic preview effects does indicate that transsaccadic integration is an inherently perceptual phenomenon and cannot be reduced to a decisional bias. One could argue that a decisional bias may not be generated until after the saccade based on ongoing processing of the presaccadic stimulus itself, which continues to develop during the saccade. In that case, our postsaccadic mask could be taken to disrupt processing of the presaccadic stimulus, thus preventing a response bias from emerging and effectively eliminating bias-based preview effects on the response to the postsaccadic stimulus. Note, however, that if this were indeed the case, we should never observe transsaccadic preview effects. Indeed, on all trials where a postsaccadic stimulus is present upon saccade landing instead of a postsaccadic mask, the same backward masking of this processing of the presaccadic stimulus should occur, response biases should never emerge, and preview effects should remain absent just as they do in the masking conditions of our Experiment 2. 

In conclusion, our main goal in this study was to test whether significant preview effects of visual detail in meaningless stimuli could be observed. Experiment 1 documented that this is indeed the case, and Experiment 2 found that these preview effects cannot be reduced to presaccadic categorization of the preview stimulus and subsequent biasing of the postsaccadic response or perceptual interpretation. Instead, they require a visually maskable store, a visually oriented integration process, or both. 

We view transsaccadic memory as a heterogeneous collection of memory traces and active processes that survive the saccade. The question of what part of transsaccadic memory, and more specifically which visual memory structure, could support transsaccadic integration of visual shape is still wide open, however. As mentioned in the Introduction section, the most often cited locus of transsaccadic integration is VSTM. This memory system is traditionally seen as not visually maskable, postcategorical, and containing only coarse and abstract information. However, our results imply that visually detailed and possibly precategorical and maskable information plays a role in transsaccadic information integration. Indeed, an intermediate transsaccadic store in this vein, called the transsaccadic visual analog, was already suggested by De Graef and Verfaillie (2002; Germeys et al., submitted for publication) and could explain the possible susceptibility of preview effects to masking by a visual object, the possibility of retention of visual detail, and the fact that the lowest level visual properties such as contrast and luminance (Henderson et al., 2008; Melcher, 2005) are abstracted from transsaccadic memory. We should note, however, that the dichotomous distinction between precategorical, maskable memory and postcategorical, non-maskable VSTM is not necessarily clear cut. For instance, Keysers, Xiao, Földiak, and Perrett (2005) propose that, like the visual system itself, abstraction and thus categorization in visual memory is gradual rather than dichotomous. Sligte, Scholte, and Lamme (2008) provide evidence for the existence of a more visually detailed variant of VSTM, that is maskable by another object but not by a light flash. Even certain models of VSTM itself could explain the results obtained. Zhang and Luck (2008) recently proposed an intrafixation model that could in principle account for visual integration in VSTM, called the “slots + averaging” model. According to the model, several instances of the same object are stored to fill up available VSTM slots and are subsequently averaged to attain a more reliable percept. If, in the absence of an interrupting mask, pre- and postsaccadic stimulus information is regarded as belonging to the same object, this could prove to be an elegant model of transsaccadic integration in VSTM. 

Our results are still at odds with previous work that did not support preview effects of visual detail, though. We present four possible reasons. First, regarding preview effects of incongruent stimuli, it has often been proposed (e.g., by Irwin et al., 1990) that integration of visual detail is not meaningful when the postsaccadic stimulus is too dissimilar from the presaccadic stimulus, and that too large a discrepancy might lead the visual system to cancel visual integration and resort to high-level abstract transsaccadic representations. In our study, on the other hand, pre- and postsaccadic stimuli were highly similar and presented at a low contrast to further minimize the subjective discrepancy. Second, Melcher and Morrone (2003) showed that the temporal proximity of both stimuli relative to the integration window of the presaccadic stimulus could be of critical importance. Presaccadic visual information would then only be used postsaccadically when the visual system is unable to collect enough information about the stimulus before the saccade. As consistent as possible with that constraint, in our study the onset of both perisaccadic stimuli was placed in close temporal proximity by the task instruction to immediately saccade toward the presaccadic stimulus as soon as it appeared. Third, we observed that higher postsaccadic contrast reduced the effect of a presaccadic preview. This implies that using too clear a postsaccadic stimulus will make the effects of visual information transfer across a saccade hard to measure. Fourth, reaction time (or naming latency) was often the dependent variable used in past studies. Even though this has the merit that it enables researchers to more closely mimic real-life perception by using meaningful, categorically different stimuli, we can see in our results that the benefit of using a congruent over an incongruent preview is much larger and easier to show in the accuracy of response than in the speed of response. Naturally, this is only the case when the difficulty of the task is calibrated individually toward intermediate levels to avoid floor and ceiling effects. This typically requires discrimination between highly similar shapes, which has been shown to be relevant for within-category object identification (e.g., Gillebert, Op de Beeck, Panis, & Wagemans, 2009; Op de Beeck, Béatse, Wagemans, Sunaert, & Van Hecke, 2000; Panis, Vangeneugden, Op de Beeck, & Wagemans, 2008). However, we believe our findings can be generalized beyond within-category identification as well. Retaining detailed visual information can aid the speed of perception even in between-category identification and, in combination with higher level scene-consistent templates (Bar, 2003; De Graef, 2007), reduce the immediate perceptual uncertainty following the saccadic gap. 

A view on transsaccadic perception thus arises in which detailed shape information is carried across a saccade and can cause preview effects in postsaccadic identification tasks. For highly similar shapes such as those used in this study, presenting a preview always provides a benefit to postsaccadic recognition speed—but at the same time, the additional benefit for using a congruent benefit shows that this enhancement in processing speed is not only governed by the general shape structure but also by the visual detail of the shape. The postsaccadic percept itself, on the other hand, can be made less veridical by preceding it with an incongruent preview and more veridical when the preview is congruent. The fact that these effects are interruptible by a visual mask shows that they are not merely the result of a response bias following presaccadic identification of the preview. We further confirm the finding that the transsaccadic representation mediating these effects is independent of stimulus contrast, and we observe that its functional importance for postsaccadic perception can depend on the quality of the postsaccadic information available. 

Acknowledgments

This research was supported by the Concerted Research Effort Convention GOA of the Research Fund K.U. Leuven (GOA/2005/03-TBA) granted to Johan Wagemans and Karl Verfaillie and by the European Community through GazeCom project IST-C-033816 granted to Karl Verfaillie and Peter De Graef. In the final stages of manuscript preparation, financial support to Maarten Demeyer was provided by means of a Methusalem grant to Johan Wagemans (METH/08/02). 

Commercial relationships: none. 

Corresponding author: Maarten Demeyer. 

Email: [email protected]. 

Address: Laboratory of Experimental Psychology, Tiensestraat 102, B-3000 Leuven, Belgium.