Convergent flash localization near saccades without equivalent “compression” of perceived separation

Abstract

Visual space is sometimes said to be “compressed” before saccadic eye movements. The most central evidence for this hypothesis is a converging pattern of localization errors on single flashes presented close to saccade time under certain conditions. An intuitive version of the compression hypothesis predicts that the reported distance between simultaneous, spatially separated presaccadic flashes should contract in the same way as their individual locations. In our experiment we tested this prediction by having subjects perform one of two tasks on stimuli made up of two bars simultaneously flashed near saccade time: either localizing one of the bars or judging the separation between the two. Localization judgments showed the previously observed converging pattern over the 50–100 ms before saccades. Contractions in perceived separation between the two bars were not accurately predicted by this pattern: they occurred mainly during saccades and were much weaker than convergence in localization. Different forms of spatial information about flashed stimuli can be differentially modulated before, during, and after saccades. Structural alterations in the perceptual field around saccades may explain these different effects, but alternative hypotheses based on decision making under uncertainty and on the influence of other perisaccadic mechanisms are also consistent with this and other evidence.

Introduction

Several decades of research have shown overwhelmingly that perceivers systematically misjudge the location of very brief flashes presented near saccade time (Bischof & Kramer, 1968; Bock, 1986; Bockisch & Miller, 1999; Honda, 1985; MacKay, 1970, 1973; Mateef, 1972, 1973; Matin, 1976, 1986; Matin, Matin, & Pola, 1970; Matin & Pearce, 1965). This fact has often been ascribed to mechanisms that combine visual signals with eye position signals to maintain “perceptual stability.” For decades, the observation in a number of studies (Bischof & Kramer, 1968; Honda, 1993; O'Regan, 1984) that these errors are typically anisotropic and show different time courses in different parts of the visual field had little effect on this consensus. 

About a decade ago, findings of one particular anisotropic pattern of localization, a converging pattern of perisaccadic localization errors near saccade time was reported, which has been replicated many times since, and its occurrence under certain conditions is well established (Awater, Burr, Lappe, Morrone, & Goldberg, 2005; Awater & Lappe, 2004, 2006; Kaiser & Lappe, 2004; Lappe, Awater, & Krekelberg, 2000; Lappe, Kuhlmann, Oerke, & Kaiser, 2006; Matsumiya & Uchikawa, 2001; Michels & Lappe, 2004; Morrone, Ross, & Burr, 1997; Ross, Morrone, & Burr, 1997). 

This line of evidence bears on judgments about the locations of briefly flashed (<15 ms) individual items in the visual environment. It has been presented, along with a series of other findings, in support of a hypothesis about the perceived structure of the environment: a “compression of visual space” over a period centered roughly around the onset of saccades (Morrone et al., 1997; Ross et al., 1997). In what follows, we use the terms “convergent localization” and “convergence” to distinguish this type of finding from that hypothesis. 

Besides the convergent localization of individual stimuli flashed in this period, several accessory lines of evidence have also been presented in support of this hypothesis of a transient change in the appearance of the visual world as a whole near saccades. These include a miscounting of bars flashed near saccade time, thought to be induced by an extreme reduction in subjective inter-bar distances (Matsumiya & Uchikawa, 2001; Morrone et al., 1997; Ross et al., 1997); the misjudgment of the collective width of a set of perisaccadically flashed bars (though not of a single object of comparable spatial extent) (Matsumiya & Uchikawa, 2001); errors in vernier alignment judgments on half-bars presented with a 75-ms asynchrony (Morrone et al., 1997); and errors on individual localization judgments for simultaneous perisaccadic flashes (Morrone et al., 1997). 

Curiously though, no published study that we are aware of has examined one of the most straightforward apparent consequences of the view that spatial judgments on flashes reflect a structural compression of “visual space” before saccades: namely, that the judged separation between two stimuli simultaneously flashed before a saccade should be diminished in the same measure as the separations between individually localized points in an equivalent stimulus. With this study we replicated the type of convergence in localization judgments found in previous studies and compared it with separation judgments on the same stimuli. 

Methods

Equipment

Stimuli were presented on a Sony GDM-F520 CRT monitor, with illuminated dimensions of 39 by 29 cm (about 66° by 52° at the viewing distance of 30 cm), driven at a resolution of 1024 × 768 and a refresh rate of 85 Hz by a Macintosh computer. Its contours were visible to subjects. Eye movements were tracked at 250 Hz using an Eyelink II (SR Research, Osgoode, Ontario, Canada) video-based pupil tracker. Coordination of stimulus presentation and eye movement recordings was done through Matlab using the Psychophysics Toolbox (Brainard, 1997) and the Eyelink Toolbox (Cornelissen, Peters, & Palmer, 2002). Ordinary fluorescent room lighting was turned on. 

Subjects

Two of the subjects were naive, one (an author) was unaware of the stimulus parameters and design of the study, the other (the first author) had designed and programmed the experiment. Subjects' heads were stabilized by a chin rest during the experiment. All had normal or corrected-to-normal vision. 

Stimuli

The screen background was a uniform gray (luminance: 17.5 cd/m ²). A black ruler (luminance: 1.0 cd/m ²) stretched horizontally across the midline of the screen, marked at intervals of 5° of visual angle with vertical tick marks labeled with pale gray text (luminance: 20.5 cd/m ²). Test stimuli consisted of two vertical pale gray bars, each 1° wide and stretching over the entire vertical extent of the screen (luminance: 20.5 cd/m ²). These pairs of bars could be centered on one of three locations, measured from the screen's midline: 0°, 5°, or 10°. On location trials (see “ Procedure” below for details on trial types), the pairs of bars were separated by 5° on every trial. On separation trials they were separated by a distance randomly chosen between 3°, 4°, 5°, 6°, or 7°. The identical red fixation and saccade target cues (luminance: 9.2 cd/m ²) occupied 1° by 0.5° of visual angle: they were presented at 5° to the left and the right of the screen's midpoint, respectively. 

Several distinctive auditory stimuli were used during the experiment. One of three possible sounds presented at the beginning of each trial indicated the task (separation or location, and in the latter case whether the bar to judge the position of was the left or the right bar in the pair). Distinctive sounds were also used to trigger saccades and to tell subjects when they made premature saccades or failed to reach and fixate on the saccade target. 

Stimuli on each trial were chosen in the following way. First, one of the three possible pair locations (centered on 0°, 5°, or 10°) was randomly chosen. On separation trials (50% of all trials), this choice determined the stimulus to be judged. On location trials (50% of trials), one out of the left and right bar within the chosen pair was picked randomly as the target for the localization judgment. 

Procedure

On each trial subjects began by fixating on the fixation target, located 5° to the left of the center of the screen at the vertical midline. After fixation was detected and confirmed came a delay period with a random duration of between 500 and 1000 ms. After the delay period the fixation target disappeared and a combination of auditory and visual signals indicated to subjects to make a 10° rightward saccade. The visual signal, presented for two video frames, was identical to the fixation target but was located at 5° to the right of the center of the screen. After a variable delay with respect to the target presentation (within 250 ms), the test stimulus was presented. Then, depending on the type of trial, one of two things occurred:

1.  
   Location judgment trials: 500 ms after the test stimulus, the mouse pointer appeared at a random location on the screen. Subjects used it to indicate the location of the relevant bar as indicated by the auditory signal at the start of the trial. If they had not seen two bars, they indicated this instead by clicking at the far left of the screen.
2.  
   Separation judgment trials: A vertical bar of dimensions identical to those of the test stimuli appeared at the saccade target location after a 500-ms delay. Subjects could use the arrow keys to split it into two and to adjust the separation between the two bars to match the perceived separation of the test stimulus. During the adjustment process the bars remained centered on the saccade target: only their separation was adjusted by the subjects.

Subjects were given means to report if they only saw one bar, but out of all subjects, only one did so on more than 4 out of 1100–1300 trials: DM, who did so on 26 trials out of 1198, with scattered timing. These trials were discarded from the analysis. 

The procedure and the sequence of events on a typical trial are illustrated in Figure 1. 

Figure 1

 

(A) Procedure. (a) Subjects begin each trial by fixating on a red rectangle located at 5° to the left of the vertical midline of the screen. An auditory signal indicating the task for the trial is presented at the beginning of this fixation period. (b) After a variable delay simultaneous auditory and visual signals indicate to subjects to make a 10° rightward saccade. (c and d) Sometime before, during, or after the saccade, a pair of white vertical bars spanning the vertical size of the screen is flashed for one video frame (∼12 ms), centered on one of three possible locations: 0° (midline), 5° (saccade target), or 10° (far side of target). On location trials their center points are separated by 5° of visual angle. On separation trials they may be separated by 3°, 4°, 5°, 6°, or 7°. (e) After the saccade, successful gaze direction to the target location is verified. Subjects are then free to look anywhere on the screen as they carry out the task. (f) Separation trials (50%): the task begins with a single bar displayed at the saccade target location 500 ms after test stimulus presentation. (No gloved hands or other indications were actually present on the screen.) Subjects used the arrow keys to adjust the bars' separation to match the perceived separation of the bars in the test stimulus. The pair of comparison bars remained centered on the saccade target. (g) Location trials (50%): the mouse pointer appears at a random location on the screen and subjects use it to report the location of the left-side or the right-side bar in the stimulus. (B) Sequence of events on a trial, with sample eye trace. After 500–1000 ms of confirmed fixation at −5°, the fixation target is extinguished and a saccade target cue at +5° is presented for two video frames (along with an auditory cue). Some variable time later (here, just before saccade onset) the test stimulus is presented for one video frame. 500 ms later, the response cue is presented and remains present until the subject responds. On location trials, the response cue is a mouse pointer at a random location on the screen; on separation trials, a vertical bar at the saccade target location.

Results

Localization judgments

Figure 2 is a plot of location judgments. A “compressive” pattern of convergence in the judged locations of bars presented at various points on the screen close to saccade onset, similar to those seen in previous studies, is visible. Strong overall distortions in reported bar location farther from saccade time are also apparent. Distortions in perceived location far from saccade time have also occurred in some other studies finding perisaccadic localization convergence (for an example, see Figure 1 of Lappe, Awater, & Krekelberg, 2000). Mislocalization of flashed stimuli toward the fovea has also been found under conditions of steady fixation (Mateeff & Gourevich, 1983; O'Regan, 1984; Osaka, 1977; Rauk & Luuk, 1980; Rose & Halpern, 1992). We leave detailed discussion of this and other aspects of the results to the Discussion section. 

Figure 2

 

Localization around saccade time. Each panel shows data for one subject. Lines show a blocked average of location judgments calculated in bins of 25 ms (this holds for all figures unless otherwise noted). Correspondingly colored dots show individual judgments. Colored squares indicate the physical positions of the corresponding stimuli. Gray rectangle shows approximate period of saccades.

Figure 2 makes the convergence of the bars presented at each location visible, but it does not distinguish between cases where the bars in the central stimuli were presented as either the leftmost or the rightmost bar in a pair. Figure 3 gives blocked means of location judgments separately for each bar according to which of the three possible pairs it was presented in. 

Figure 3

 

Judged locations according to pair membership. The bottom curve in each colored pair is the left of the two bars in that pair; the upper curve is the bar on the right. (Only one bar was localized on any given trial.) Upper panels give individual subject data; lower panel gives mean for all subjects. Note the convergence between the central red and blue bars (the two central bars, at −2.5° and 2.5°, presented as members of the two peripheral pairs) during the saccade and the lesser convergence between green bars (the same bars presented in one pair centered on the saccade target).

Figure 3 makes clear that over a period around saccade onset, corresponding to the time of greatest “compression” of locations in space, the two inner bars (at 2.5° and 7.5°) are localized differently according to which bar pair they were part of. For a period near the saccade onset which varies from subject to subject, the two central bars were localized more toward the center of the range of locations when they were part of one of the two pairs we will call “peripheral”—the ones centered at 0° and 10°—than when they were part of the pair centered on the saccade target itself, at 5°. Note the mutual approach of the blue and red curves during the saccade. The space between the curves from the two members of each pair, on the other hand, appears to remain relatively constant throughout the presaccadic period, which suggests that simultaneously presented bars were jointly mislocalized, rather than independently converging toward the saccade target, during this period. During the saccade, distance between the reported locations of simultaneously presented bars contracts somewhat but to a considerably lesser extent than the distance from bars to saccade target. See Figure 8 for a comparison of the time course and amplitude of changes in location and in separation. 

Separation judgments

Figure 4 shows data on errors in the reproduction of separation between paired bars. Presaccadically, no strong overall developing pattern of compression is obvious in this figure, though some occurs in the most peripheral pair shortly before saccade onset. Degrees of downward change in estimated separation from trials where stimuli were presented between 100 and 125 ms before the saccade to those where they were presented in the last 25 ms before the saccade for the three stimulus locations (0°, 5°, and 10°) were 0.4%, 1.5%, and 7%, respectively. A dip in estimated separation occurs during the saccade of somewhere on the order of 20% at all locations. After the saccade, a tendency to overestimate separation seems to occur, particularly for pairs of bars centered at 0°, the midpoint of the saccade and the only stimulus location whose center is “behind” the saccade target along the saccade path in this period. 

Figure 4

 

(A) Separation judgments around saccade time. Thick lines show a blocked average of the judged separation between bars in epochs of 15 ms. Results are given as ratio of judged to actual bar separation: each flat horizontal line marked “1” corresponds to accurate judgments, 0.5 below to 50% underestimation, and 0.5 above to 50% overestimation of separation. Over the presaccadic period, two subjects (FC and DM) show constant underestimation, one (KOR) shows roughly constant overestimation, and one (PR) is accurate. (B) Separation judgments by location and time, means of individual subject curves. The proportional drop in reported separation from between 125 and 100 ms before the saccade to the final 25 ms before the saccade is 0.004, 0.015, and 0.07 for pairs centered at 0°, 5°, and 10° from the midpoint of the screen, respectively. Intrasaccadic minima are 0.19, 0.16, and 0.19 respectively. Error bars show ±0.5 standard deviation.

Effect of stimulus width on intrasaccadic separation distortions

On separation trials, five different stimulus widths were used. Figure 5 gives curves of judged separation separately for each stimulus width. From this figure it is clear that degrees of separation distortion depended on width. The widest stimuli were the most underestimated during saccades, both absolutely and proportionally. The widest stimuli (6° and 7° of visual angle) also seem to show a degree of underestimation just before the saccade (0.06 and 0.1, respectively). 

Figure 5

 

Distortions in separation judgments by stimulus width (error bars give ±0.5 standard deviations). Each point gives mean judged separation for a given stimulus width over a 25-ms bin. Upper left panels show individual subject data; upper right panel gives means of individual subject curves. Bottom panel gives ratio of intrasaccadic minimum separation to mean judged separation over the 125 ms before saccades for each width: reduction in judged separation during saccades is not only absolutely but also proportionally greater for greater stimulus widths.

Measures of “compression”

“Compression” of separation

A way to directly compare the degrees of saccade-related “compression” in the separation between adjacent, simultaneously presented bars on the different tasks we employed is to take the distance between mean localization curves for adjacent bars from location trials and compare it to the mean of separation judgments at the same moment in time. We will use the term “perceptual separation” to refer to these two measures, although on location trials subjects only reported the position of one bar on any given trial and therefore did not directly judge separation. 

Figure 6 shows the time courses of these indices of judged separation for each subject on trials with bars separated by 5° of visual angle. Notable aspects of the results include an overall difference in separation between the two types of trials, a flat pattern in the presaccadic period (with the exception of subject PR), and an intrasaccadic drop in separation in both trial types for all subjects (again with the exception of PR, whose judged separation remains lowered at the beginning of the saccade). After saccades, separation judgments are somewhat higher than presaccadic judgments for all four subjects. It should be kept in mind that the retinal locations of presaccadic and postsaccadic stimulation differ by roughly 10°, corresponding to the amplitude of the saccade. 

Figure 6

 

Perceptual separation of adjacent bars as a proportion of real separation by time. For location trials, “perceptual separation” is defined as the separation between the mean reported locations of the adjacent bars in particular stimuli and thus does not refer to direct judgments of separation. Upper panels give individual means; lower panel gives overall mean. On separation trials, curves are derived from trials with stimuli at a real separation of 5°, as on these trials stimuli are physically identical to those used on location trials. Overall, judged separation is greater on separation than on location trials; relatively little change in judged separation occurs before saccades for three of four subjects (subject PR shows a clear drop, particularly in location trials), but a sharp drop occurs during the saccade in both trial types for all subjects. Judged separation is higher after saccades than before them in both trial types.

It appears from this measure that separation is subject to similar perisaccadic effects in location and separation trials, but that there is a constant difference in judged separation between the two trial types. Separation is judged to be higher on separation trials than on location trials. 

“Compression” of location

The measures used in the preceding section illustrate the time course of change in judged separation around saccades. For location trials, this shows the evolution of separation between judged locations of adjacent bars (separations of same-colored curves in Figure 3) but does not include changes in apparent location in space (placement of the pairs of curves in space in Figure 3). One way to estimate this aspect of the data is to take the “centers” of the pairs of bars whose location was judged on location trials and chart their evolution. Figure 7 gives the results of this analysis. 

Figure 7

 

Convergence of pair centers on location judgments. This index was calculated by taking the mean of the two judged locations for each pair and taking the mean of the two separations between the centers of the adjacent pairs at each moment in relation to the saccade. Recall that the correct separation is 5°. Unlike separation between members of simultaneously flashed pairs, distance between pair center locations diminishes progressively with decreasing interval to saccade onset for all subjects.

Figure 7 captures an aspect of the data that was eliminated by the measure of “compression” in judged separation. Here, rather than flatness before the saccade and a sudden intrasaccadic plunge, the spread in space of the judged locations of pairs progressively changes with decreasing interval to the saccade onset, reaching a mean of 23% “compression” before saccade onset and a mean minimum of about 50% during the saccade. 

Figure 8 directly illustrates the central contrast in our results, between the effects of saccades on perceived separation and those on spread in reported locations. As mentioned, pair locations converge over the 100 ms before saccade onset and reach a minimum spread shortly after saccade onset, whereas separations remain essentially constant over the period just before saccades and drop during them to a level similar to the presaccadic minimum level of location convergence. 

Figure 8

 

Discrepancy between convergence in localization and “compression” of separation. “Spread index” is calculated as the mean of the two separation indices from Figure 6, after normalization to the value for each measure to its value between 125 and 100 ms before saccade onset so as to remove constant differences in effects on the different tasks. For location trials, the index is the measure of convergence from Figure 7 and is equivalently normalized. Mean of individual subject curves is shown.

Auxiliary experiment

Introduction

Studies on perisaccadic localization convergence have generally been conducted under conditions described as “dim lighting” or in darkness apart from screen lighting. The first experiment described here was performed under ordinary fluorescent room lighting. In order to verify that this fact was not responsible for the pattern of findings presented above, we had two subjects (an author and a naive subject) perform the exact same experiment as above, but in dim lighting (the screen, whose borders were visible, being the only source of luminance in the room). Apart from this fact, all methods were precisely as described above. 

Results

As can be seen in Figures 9– 11, the results of the auxiliary experiment suggest that room lighting was not responsible for the pattern of results seen in the first experiment, in particular the contrast between patterns of “compression” in separation and in location in space. Subject CS shows a quite large constant difference between the two trial types in judged separation but a very small dip in judged separation during the saccade (less than any of the subjects in the first experiment) in both conditions. For pair location, in contrast, he shows a similar convergent localization to the subjects in the first experiment. Subject PR shows virtually identical patterns to his results in the first experiment, once again including a much greater degree of convergence between locations in space than in separation between pairs of simultaneously flashed bars. 

Figure 9

 

Judged locations according to pair membership, auxiliary experiment. Calculated as in Figure 3. The bottom curve in each colored pair is the left of the two bars in that pair; the upper curve is the bar on the right. (Only one bar was localized on any given trial.) Subject CS, like the inexperienced subjects of the first experiment (DM and FC), shows a gap in data in the period of 50 to 100 ms before the saccade.

Figure 10

 

Judged separation of adjacent bars as a proportion of real separation, auxiliary experiment. Calculated in the same way as in Figure 6. Subject PR shows a pattern virtually identical to his pattern in the first experiment (plotted here in grey).

Figure 11

 

‘Compression index’ for pair locations: separation between centres of pairs for Location judgments, auxiliary experiment. Calculated as in Figure 7. Data from first experiment from subject PR are also included for comparison (plotted in grey).

Discussion

Contrasting effects of saccades on localization and separation

What do these findings tell us about the idea of a visual “compression” near saccades? Although perceptual separation underwent distortion during saccades (and possibly just before, under some conditions), distortions in the reported locations in space of pairs of items ¹ simultaneously flashed near the time of a saccade did not predict perceived structure within those pairs ( Figure 8). Notably, the progressive convergence in the localization of individual flashes with decreasing interval to saccade onset, seen here as in previous studies, did not apply to the judged separation between simultaneously flashed bars under the conditions of our experiment. Nor did the magnitude of intrasaccadic effects on the two match. 

Separations between simultaneously flashed stimuli thus undergo perceptual effects around saccades with a different time course and strength from those that apply to the judged positions of flashed objects in space. What is the relation between these different types of effects, and what do they tell us about the hypothesis of an underlying visual “compression” before saccades—or about other hypotheses that might account for part or all of the findings that are associated with this hypothesis? 

To begin, we must distinguish between at least two distinct uses of the term “compression” in this context. The first use might be called “descriptive” and defined simply as a property of psychophysical curves wherein points in the environment to which subjects attribute flashed items, or intraitem distances that subjects attribute to them, is less than the physical distance between them. Our results manifest such an effect on both separation of distinct flashed items and their locations in space, albeit with markedly different time courses and strengths. 

A second, stronger, “explanatory” concept of compression—initially identified by the phrase “compression of visual space” (Ross et al., 1997)—is required to offer a unifying account of the effects which have been associated with this term. Bar miscounting, for example, does not involve subjects attributing any overtly spatial property to flashed stimuli and thus does not meet the criterion for the first, descriptive concept of compression. Instead, structural distortions in the perceptual field are hypothesized and invoked to justify the view that miscounting is a manifestation of one and the same underlying phenomenon as changes in localization of single flashed items (Morrone et al., 1997), changes in the reported spatial extent of multiple-bar stimuli (Matsumiya & Uchikawa, 2001), etc. 

It may be noted that our results fail to manifest the equivalence between perisaccadic effects on perceived location and on perceived intraitem distance that is assumed in this account. 

The issue of explanatory unification is not only raised by non-spatial judgments such as counting. Discrepancies between the spatial and temporal profiles of different instances of the descriptive concept, such as those seen in our results, also raise it, as do other discrepancies between conditions where it applies and relevantly similar ones where it does not, as in the results of Matsumiya and Uchikawa (2001) on the reported spatial extent of spatially continuous or uniform and discontinuous or non-uniform stimuli. 

Here we survey the issues raised by our results and look at strategies either for reconciling them with others through the unifying concept of a structural “compression” of the perceptual field, or of accounting for some or all of these results in other ways. 

Perceptual grouping, separation, and convergence

Matsumiya and Uchikawa (2001) suggested that only inter-object distances are subject to structural compression, and not the spatial extent of single visual objects. It might be supposed, following this suggestion, that the two bars in our stimuli were grouped together into a single object, thus avoiding or reducing compression between them. Increases in “compression” effects on separation with increasing inter-item distance (Figure 5) could be attributed to a diminishing tendency to group items into a single visual object with increasing separation. 

Matsumiya and Uchikawa (2001) suppose that the spatial separation between bars leads to their identification as distinct visual objects and take this as the precondition for compression. The adjacent bars in their stimuli, however, were separated by distances (1.3–5.1° of visual angle) up to the middle of the range in our study (3–7° of visual angle). If, as suggested, separation between items in itself determined grouping into objects and thus the occurrence or absence of “location compression,” then we should have found a substantial degree of separation underestimation before saccades, which is precisely what we did not find. 

An odd consequence also follows from the proposed reconciliation of these different lines of findings based on the separateness of object and location processing proposed above. Suppose that:

1.  
   Compression applies to the locations of objects but not (or less) to their internal spatial structure.
2.  
   The binding together of distinct items into objects depends on the (retinal) distance between them, with closer items more likely to be bound together, and their spatial relations thus likely to be less or not “compressed.”
3.  
   Counting errors in multiple-bar stimuli reflect compression, subject to the constraints noted above: the bars are “compressed” together only if they are far enough apart to be treated as separate objects by the visual system.

In this case, it seems that the condition required for bars to be so drastically “compressed” together as to be impossible to count as separate objects is that they be far enough apart to be … treated as separate objects! Such an apparent paradox is not inconceivable: we could hypothesize a form of multiple-stage processing in which items are non-consciously identified as separate objects, with compression occurring on the way to a further stage wherein explicit counting judgments are performed on items become rendered indistinguishable by structural compression. It is not clear, on the other hand, that this type of account stands as a uniquely plausible way to understand these effects. 

Other accounts might invoke, for example, a failure to register some in a set of several widely spread, very briefly flashed peripheral items near saccade time, leading to judgments based only on the subset of items that are in fact registered (Reeve, Clark, & O'Regan, 2004). Alternatively, extremely high degrees of both spatial (Binda, Bruno, Burr, & Morrone, 2007) and temporal (Kumar & Stevenson, 2007) uncertainty about perisaccadic flashed stimuli in particular could, for example, lead to the mistaking of multiple bars for the pre-, intra-, and postsaccadic sensory consequences of the same object. The use of low-contrast stimuli, associated with a low signal-to-noise ratio and thus high uncertainty, could add to these effects. This account would also offer an alternative interpretation of the finding of Lappe et al. (2006) of a lesser tendency to miscount colors than objects near saccade onset and the ascription of the colors of different items to a single counted object. Color detection need not be subject to the same spatiotemporal uncertainty as the detection and counting of the visual items the colors are to be ascribed to. 

These alternative hypotheses do not require the hypothesis of a structural compression of the perceptual field near saccades and therefore offer a possible way to reconcile non-compressed separation perception with miscounting. Though the compressed image of a scene is simple to imagine, this should not lead us to exclude less pictorial alternatives. 

Are pairs of flashed stimuli near saccades “compressed” together?

In our study, judged location of flashes in space and judged separation between flashes in the same conditions yielded a stark dissociation. Although both location and separation judgments showed some distortions, convergence in localization did not predict either the much briefer, mostly intrasaccadic time course, or the lesser magnitude of changes in judged separation. 

In other studies, uncompressed judgments of spatial relations between separate extremely brief flashes near saccade time have been repeatedly found, both in darkness (Brenner, Meijer, & Cornelissen, 2005; Matin, 1976; Sogo & Osaka, 2002) when localization convergence is not typically found (Lappe et al., 2000; Sogo & Osaka, 2002) and under conditions including illuminated postflash visual context (Brenner et al., 2005; Morrone et al., 1997) where convergence has been found to occur (Lappe et al., 2000; Morrone, Ma-Wyatt, & Ross, 2005; Morrone et al., 1997). Other studies have found large deviations from retinal separation near saccades (Morrone et al., 1997; Sogo & Osaka, 2005) in the direction of “compressive” errors in localization. 

What is behind these contrasts between the findings of different studies? A number of differences between studies could be relevant. 

One of these is the spacing between flashed items. Studies finding nearly or entirely undistorted judged separation before saccades have used smaller retinal separations between flashed items, ranging from 1.34° (Brenner et al., 2005) to 6.5° (Sogo & Osaka, 2002). In the present results no clear presaccadic distortion is apparent with less than 7° of separation. Groups finding “compressive” distortions in separation, on the other hand, have generally used larger separations: Morrone and colleagues (Morrone et al., 1997; Ross et al., 1997) found strong distortion of intraflash distance for pairs of “isoluminant” peripheral stimuli separated by 20° of visual angle (though not for such stimuli symmetrically arranged around the fovea). Sogo and Osaka (2005) found strong intrasaccadic separation distortions (presaccadic effects were not reported in this connection) relative to continuous stimuli at an intra-item separation of 17°, much weaker at 7°, and none at 3.67° or less.² 

In our findings as well ( Figure 5), increasing spatial spread is associated with increasing distortion, at least during the saccade. These findings appears compatible with the hypothesis that the degree of correspondence between retinal separation and judged separation found with flashes presented sometime near saccade onset depends on retinal distance, with greater distance associated with greater distortions. As noted, our results suggest that such an effect is distinct in both spatial and temporal respects from changes in perceived flash location around the saccade. 

A second issue is retinal eccentricity. The very large retinal eccentricities (up to 30°) used by Morrone et al. (1997), Sogo and Osaka (2005), and Matsumiya and Uchikawa (2001) could be crucial in creating the separation effects that they found. Studies finding accurate separation judgments have generally used lesser eccentricities: the two clearest cases being at a maximum eccentricity of 4° (Sogo & Osaka, 2002) and just over 6° (Brenner et al., 2005). 

Our maximum eccentricity was intermediate between these two groups at about 19°: the distortion we found was almost or wholly exclusive to the intrasaccadic period for all but one subject. The effects that we found, however, had a similar strength for stimuli presented centered at eccentricities of 5° and 15°. 

Finally, it may be noted that distortions in separation have been found in studies using quite large (20°) saccades (Matsumiya & Uchikawa, 2001; Morrone et al., 1997; Sogo & Osaka, 2005).³ Studies finding little or no distortions have used smaller saccades: 5° (Brenner et al., 2005), 8° (Sogo & Osaka, 2002), and, in the present study, 10°. Because the eccentricity of stimuli is generally determined by saccade target location, these two factors are interrelated. This factor could nonetheless have its own impact, if the effect of eye position signals on the relevant processes varies in some non-linear fashion with saccade amplitude (as suggested by Pola, 2004, with regard to effects on localization). 

It should be observed that larger eccentricities, greater stimulus spacing, and greater saccade sizes have been associated with each other. Experiments aimed at characterizing the interaction of these factors are not difficult to conceive. 

Decisions and uncertainty around saccades

How do we account for the discrepancy between the patterns in localization in space and in spatial relations between simultaneously presented stimuli that we find? How are these different aspects of the same stimuli related in perception? The answers to these questions will depend on hypotheses about how the relevant judgments are performed. 

One model of perisaccadic flash localization comes from Awater and Lappe (2006). They presented a two-stage account of localization seeking to account for convergence effects and their dependence on visual stimulation after the saccade (Awater & Lappe, 2006; Lappe et al., 2000). In their model, a signal of the “retinal” relationship between a bar flashed before a saccade and the saccade target is compressed, and then postsaccadic stimulation determines the placement of these items in space. Depending on whether or not visual references are present, either these or an extraretinal signal is consulted to determine the location of the saccade target, leading either to compression or to uniform forward mislocalization. They suggest several reasons for a presaccadic distance compression, related to a presaccadic attention focus on the saccade target (Deubel & Schneider, 1996; Hoffman & Subramaniam, 1995; Kowler, Anderson, Dosher, & Blaser, 1995), “reentrant” oculomotor signals (Hamker, Zirnsak, & Lappe, 2004), or a “mismatch between egocentric and exocentric localization mechanisms.” Others (Morrone et al., 2005, 1997) have also suggested that presaccadic changes in the spatial profile of responsivity of cells in certain brain areas near the time of saccades (Duhamel, Colby, & Goldberg, 1992; Kusunoki & Goldberg, 2003) might be involved. 

How might our findings be related to this account ascribing separation to distortions in bar-target distance introduced by presaccadic processes? At first glance, they appear not to fit, as in our findings perceptual separation is not much affected by temporal proximity to saccade onset until the eye actually begins to move. Perhaps, however, the temporal interval between flashes is crucial: simultaneously presented bars might not be subject to this effect with respect to each other, but only with respect to the (earlier) saccade target. 

It may be, however, that the presaccadic period is the wrong time to focus upon. As Awater and Lappe (2006) point out, judgments on immediately presaccadic flashes must be performed long enough later that the eye has already moved when they occur. And as they emphasize, this apparently leads to an influence from later events—including visual events—on judgments about this earlier stimulus. One possibility that certainly cannot presently be ruled out is that postsaccadic events are conditions on the very occurrence of “compression” and not only a requirement to make it manifest. 

How might convergent localization be understood, if not as the result of a transient compression of the entire visual world? Recent evidence suggests that temporal uncertainty about the relationship between brief, abrupt visual events and saccades as well as “simulated saccades” can be extremely large (on the order of 150–200+ ms standard deviation) (Kumar & Stevenson, 2007). Large increases in spatial uncertainty about location of stimuli when they are flashed near the time of saccades have also been found (Binda et al., 2007). We might suppose, then, that convergence is a consequence of high levels of uncertainty about the particular forms of information that are used to judge spatial location near saccades. 

Brenner, van Beers, Rotman, and Smeets (2006), in turn, found that spatiotemporal uncertainty about moving stimuli triggered by brief sensory events (flashes or beeps) may lead to reliance on a prior expectation of closeness to the fovea. Such foveal biases have been manifested in flash localization in other contexts as well (Mateeff & Gourevich, 1983; Müsseler, van der Heijden, Mahmud, Deubel, & Ertsey, 1999; O'Regan, 1984; Osaka, 1977; Rauk & Luuk, 1980; Rose & Halpern, 1992). (These biases are likely also reflected in the constant underestimation of spatial spread in our location trials.) 

A straightforward hypothesis could tie these facts to convergent localization. If

1.  
   the decision process involved in flash-triggered localization includes a delay long enough for decisions on immediately presaccadic stimuli to be taken after the saccade has ended (Awater & Lappe, 2006; Brenner et al., 2006; Murakami, 2001);
2.  
   information on both the relative timing of flash and saccade (Deubel, Irwin, & Schneider, 1999; Kumar & Stevenson, 2007; Volkmann & Moore, 1978) and on flash location (Binda et al., 2007) is highly uncertain; and
3.  
   location is subject to a prior expectation of proximity to the fovea under spatiotemporal uncertainty (Brenner, Mamassian, & Smeets, 2008; Brenner et al., 2006).

Then highly uncertain flash locations, evaluated when the eye has reached its postsaccadic fixation, would be judged as closer to that fixation point due to the application of the relevant prior expectations. On this alternative hypothesis, convergent localization emerges in psychophysical data as the outcome of a decision process about a particular aspect of the scene—location—and not as a local sample from a globally distorted representation of the scene. 

Why should convergence effects depend on postsaccadic visual stimulation? Potential reasons are not hard to find. This stimulation could, for example, enhance uncertainty about earlier positions through automatic, stability-related postsaccadic visual processing triggered by the presence of stimulation carrying local spatial information after the saccade (Deubel, 2004). Richly documented enhancements of visual responses immediately after saccades in many visual areas could also conceivably be involved (Ibbotson, Price, Crowder, Ono, & Mustari, 2007; Lal & Friedlander, 1989; Lee & Malpeli, 1998; Leopold & Logothetis, 1998; Park & Lee, 2000; Rajkai et al., 2008; Ramcharan, Gnadt, & Sherman, 2001; Reppas, Uppas, & Reid, 2002; Royal, Sáry, Schall, & Casagrande, 2006; Takemura & Kawano, 2006). 

Furthermore, it is well established that when flashes trigger judgments when relevant signals are somehow changing, those judgments can be influenced by events in those changing signals over a period on the order of 80–100 ms or more. Well-known examples include the “flash-lag effect” (Eagleman & Sejnowski, 2000; Khurana & Nijhawan, 1995; Krekelberg & Lappe, 2000; MacKay, 1958; Mateeff & Hohnsbein, 1988) and flash localization during smooth pursuit eye movements (Blohm, Missal, & Lefèvre, 2003; Hazelhoff & Wiersma, 1924, 1925; Mateeff & Hohnsbein, 1988; Mitrani, Dimitrov, Yakimoff, & Mateeff, 1979). 

What about perceptual separation, on this view? A priori, it seems that the spatial properties of separate items could be subject to anything between total independence (effects on single items apply separately to each) to total unity (effects apply jointly to all bars, preserving spatial information about relations). Results from our presaccadic trials appear to suggest the latter: convergence in location before saccades seems to apply not to each bar independently, but to the pairs of bars flashed together. Insofar as this convergence in location reflects the effects of uncertainty, this would suggest that under our conditions and before saccades, uncertainty applies not to the location of each bar independently but to their placement in space as a pair. Otherwise we would expect the judged locations of the two bars presented simultaneously on either side of the side of the saccade target to be distorted in the same way as the same two bars when presented as part of the two peripheral pairs. 

Morrone et al. (1997) found, in contrast, that pairs of bars flashed near the saccade were separately mislocalized toward the postsaccadic direction of gaze. Conceivably this difference may reflect differences in uncertainty about retinal separation itself. Their conditions of greater eccentricity (Hess & Field, 1993), greater stimulus spacing (Whitaker & Latham, 1997), and lower contrast (Stocker & Simoncelli, 2006) might all be expected to increase visual uncertainty on the strength of evidence from other contexts. The apparent unity of effects on separation and on location in their results thus may be an artifact of their use of stimuli subject to exceptionally high degrees of sensory uncertainty. 

Our contrasting results on judgments about different aspects of stimuli flashed near saccades, combined with others in this small research literature, suggest that existing results may not be best explained by the hypothesis of a single, cohesive deformation of the visual world near saccades. 

Contrast, two-bar stimuli and convergence

It may be observed that the convergence seen here appears to be weaker than what was found for an equivalent contrast by Michels and Lappe (2004). A plausible reason for this fact relates to the fact that the convergence we find seems to apply to pairs of bars and not to each bar separately. The results in our Figures 3 and 8 strongly suggest that the limit on the convergence of pairs of bars toward the saccade target is dictated by the bar that appears closer to the saccade target. Note how close together the inner blue and red curves corresponding to these bars come during saccades for all subjects in our study, in most cases touching or overlapping, in contrast with the green curves which remain separated. If this is right, then the degree of convergence occurring for the outer bars is already at or close to the maximum possible level with these stimuli. Furthermore, under these conditions the central pair can show no presaccadic location convergence at all because of the mutually constraining influence that the two bars exercise on each other. In this case, then the convergence measure we used is bound to understate the power of convergence under our conditions, as the only clear indication comes from the two bars that will not at all have been prevented from approaching the saccade target more closely by the constraining influence of another bar. The “compression” of these two bars is very high, with the distance between their judged locations reduced by 90% or more at the point of greatest convergence. This in turn would imply that the dissociation seen between the magnitude of effects on location and on separation in our results considerably understates what might have been found should location convergence have been tested using single-bar stimuli. 

Postsaccadic expansion

From our Figure 4 it appears that the separation of pairs presented immediately after saccades was overestimated. This finding bears a suggestive relation to the findings of Cho and Lee (2003) on a “postsaccadic expansion of visual space.” The increased spread they found for the localization of postsaccadic stimuli is concentrated in the part of space located along the saccade path (see their Figure 9), just as the stimuli centered at 0°, the midpoint of the saccade along the saccade path, appear in our results to be the most consistently overestimated as a result of the saccade. Location trials show a similar tendency to show a greater judged separation after saccades than relatively long before them, a tendency that in this case too is strongest for stimuli presented along the saccade path (pairs centered at 0°: see Figures 3 and 8). 

Conclusion

MacKay (1970) argued that perisaccadic errors may be the outcome of an “integrative process,” relating flashes to a background of “evidence” originating with contextual events over an extended temporal window. Though MacKay focused on the role of visual–visual interactions, as manifested in effects with simulated saccades, he did not rule out the possibility that oculomotor activity could also play a role. The recent models of Pola (2004, 2007) illustrate one way that integrative processes incorporating oculomotor signals could account for the patterns of errors found in some perisaccadic flash studies. Awater and Lappe (2006) suggest ways that postsaccadic context could be involved in convergent localization, but other possibilities about the role of postsaccadic events in these perceptual phenomena remain open, any of which could be tied to specialized and generally enhanced visual processing occurring just after saccades. Further work may permit the elucidation of just what aspects of sensory signaling related to flashed stimuli are modulated close to saccades; how this modulation contributes to other types of errors such as convergence and, under apparently somewhat distinct conditions, separation distortion; just what temporal window of events contributes to this modulation; how these facts relate to the more general susceptibility of flash-triggered judgments to modulation by events over an extended temporal window; and how all these issues relate to the time scale of perceptual decision making more generally. It may be that visual perception of the world as a whole goes through a strange, massive, and usually invisible distortion several times per second; perhaps too, the time has come, decades later, to explore MacKay's idea more fully. 

Acknowledgments

JJC acknowledges the financial support of NSERC and the IRIS NationalCentre of Excellence in Robotics and Intelligent Systems. 

Commercial relationships: none. 

Corresponding author: Paul T. Reeve. 

Email: [email protected]. 

Address: Laboratoire Psychologie de la Perception, CNRS-Université Paris Descartes, UFR Biomédicale des Saints Pères, 45 rue des Sts Pères, 75270 Paris cedex, France. 

Footnotes

Footnotes

¹  We will use this term to refer to spatiotemporal regions of stimulation taken as spatially and temporally discrete and contiguous for psychophysical purposes: contrast-defined bars flashed on CRT monitors, etc. The question of their perceptual “objecthood” is treated as distinct, for reasons discussed in the section on Perceptual grouping, separation, and convergence.

Footnotes

²  At this physical separation they still found underestimation of spatial extent, as they did with continuous forms, but it is unclear whether this effect is due to timing in relation to the saccade, and to what extent it may be a consequence of presenting stimuli when the eye is already in saccadic motion.

Footnotes

³  Morrone et al. (1997) found convergent localization with smaller saccade sizes but did not test the impact of this factor on separation.