Seeing and ballistic pointing at perisaccadic targets

Abstract

We studied the effects of visual references and the level of illumination on the localization of stimuli flashed briefly near the start of saccades. A translucent shutter made it possible to remove visual references, but admit light, at different times after saccadic onset. The results show that post-saccadic visual references are not necessary for compression: a consistent compression of verbally reported relative stimulus distances is found at all shutter latencies and at all post-shutter levels of illumination. They also show that positions indicated by blind pointing show no compression except when visual references remain in view for a substantial time after saccades. These results confirm that the visual system uses multiple representations of space and suggest that it weights them differently for different tasks and different viewing conditions. No single map is used exclusively for conscious perception or for motor action, and conscious perception is always subject to compression at the time of saccades.

Introduction

Conscious perception of the positions of stimuli flashed near the start of saccades is error prone. Stimuli are subject to saccadic suppression (Volkman, 1986), are sometimes not seen at all, and are often seen in erroneous positions, sometimes far from their true positions (Bischof & Kramer, 1968; Cai, Pouget, Schlag-Rey, & Schlag, 1997; Honda, 1995; Mateeff, 1978; Matin, 1965; Matin, Matin, & Pearce, 1970; for a review, see Ross, Morrone, Goldberg, & Burr, 2001). There are two components to the mislocalization of stimuli from their true positions: (1) a uniform translation in the direction of saccades, independent of true stimulus position; and (2) a bidirectional compressive shift towards the saccadic target, against the direction of saccades for stimuli beyond the saccadic target and in the direction of saccades for stimuli on the nearer side (Morrone, Ross, & Burr, 1997; Ross, Morrone, & Burr, 1997). 

It has been suggested that vision may have separate representations of space, one for conscious perception and one for action (Goodale & Milner, 1992; Trevarthen, 1968). Many investigations of this suggestion have employed visual illusions, on the assumption that while conscious perception may be susceptible to illusions, action will not. Some evidence supports this suggestion (Goodale & Milner, 1992; Goodale & Westwood, 2004; Westwood & Goodale, 2003); but some does not (Franz, Gegenfurtner, Bulthoff, & Fahle, 2000; for a review, see Goodale & Westwood, 2004). Although there is overwhelming evidence for perceptual mislocalizations during saccades, there is also good evidence that subjects maintain a veridical spatial map, and that this can be used to guide motor activity such as secondary saccades and hammering (Hallet & Lightstone, 1976a, 1976b; Hansen & Skavenski, 1977, 1985). Other studies (Bridgeman, Hendry, & Stark, 1975; Goodale, Pelisson, & Prablanc, 1986) also reported that subjects can point accurately to targets that were displaced perisaccadically, although the subject did not perceive the change in target position. These results are consistent with the suggestion of the existence of two visual representations. 

While the early evidence seemed clear, some recent experiments on pointing and secondary saccades have failed to replicate the original dissociation between motor accuracy and perceptual error during saccades, reporting localization errors for both tasks (Bockisch & Miller, 1999; Dassonville, Schlag, & Schlag-Rey, 1992; Dassonville, Schlag, & Schlag-Rey, 1995; Honda, 1991; Miller, 1996; Schlag & Schlag-Rey, 1995). Recently, Burr, Morrone, and Ross (2001) found evidence supporting the dissociation in an investigation of susceptibility of pointing to saccadic compression. They found that under some conditions when stimuli are seen in false positions, observers can nevertheless point blindly at them with high accuracy. But their results could not be explained simply by a distinction between conscious perception and action. They found that under normal lighting conditions neither verbal report of position or pointing is accurate: compression is found in both cases. They suggested vision has two maps available to it, one subject to distortion and the other not, and that neither viewing conditions nor task alone determines how vision uses these two maps. Both contribute to determining the weight given to each map. 

Lappe, Awater, and Krekelberg (2000) reported that the shift component of perisaccadic errors of location was found in the dark with or without visual references after a saccade, but that the compression component was found only when visual references were present. The availability or unavailability of post-saccadic references is usually associated with the presence or absence of light. When they are available there is usually ambient light; when they are unavailable there is not. It is therefore possible that effects attributed to visual references may be due, at least in part, to ambient light. This distinction is important—if post-saccadic visual references are critical, it would suggest that the construction of space may be a dynamic process that takes time to incorporate new information about references in space. On the other hand, if it is due merely to the presence or absence of light, it might be merely a weighting of two maps that are more reliable under certain circumstances. In the presence of light, a map based on immediate visual perception could be treated as more reliable (Niemeier, Crawford, & Tweed, 2003; Pouget, Deneve, & Duhamel, 2002). More weight may then be given to this visual map, which is subject to distortion. 

There are two main difficulties in interpreting Lappe et al.'s work. The first arises from the fact that subjects had to point by moving a mouse to the location of the target. The subject could therefore see the mouse arrow when making the point, generating a hybrid condition of pointing with partial visual feedback. This makes it difficult to assess if these localizations were driven by a motor or a perceptual localization map. In addition, recent work from Awater, Krekelberg, and Lappe (2000; see also Awater, 2002) introduced another factor to be taken into account. They investigated the perceived position of the target of saccades as well as the perceived position of briefly flashed bars. In the dark, they found errors in the perception of target position as well as errors in the perception of the position of bars. The target was attracted toward the bar despite the fact that observers knew where the target was and saw it for longer than they saw the bars. Awater et al. suggested that saccadic compression ‘should be interpreted as a misperception of the relative position of the flashed bar and the saccadic target’. It is possible that compression of space occurs under all lighting conditions, and regardless of visual references, contradicting the initial interpretations of the results proposed in Lappe et al. (2000). If this is the case, then it would seem that compression of space is triggered primarily by the corollary discharge associated with an eye movement, in agreement with earlier work from our group showing that simulated saccades never produce compression of space (Morrone et al., 1997). 

The experiments we report here are designed to disentangle the effects of general illumination from the effects of visual references. Both may be important in dissociating errors in verbal report from errors in pointing. In all conditions, the availability of visual references is limited by the time at which a shutter closes. In one condition, darkness ensues when the shutter closes: The availability of light is thus the same as the availability of visual references. In two other conditions, the shutter is flooded with diffuse light after it closed; thus visual references are removed but light remains available. After making saccades, observers either point with an unseen hand to indicate perceived positions of stimuli displayed before the shutter closes or make verbal reports of position with reference to a remembered ruler. 

Methods

Visual stimuli

Stimuli were displayed on a NEC Multisync V921 color monitor with a Touch-Screen of display area 33 × 23 cm, subtending 47° × 33° at the viewing distance of 40 cm. The display was surrounded by the dark brown frame of the monitor. Stimuli were delivered at a frame rate of 300 Hz (beyond standard specifications) by a visual stimulus generator (Cambridge Research Systems VSG2) housed in a personal computer (PC) programmed in MatLab. All stimuli were presented on a red background [Commission Internationale de l'Eclairage (CIE) coordinates: x = 0.563; y = 0.398; luminance, 12 cd/m²]. The fixation spot and saccade target were black dots 1° in diameter. The test stimulus was a clearly visible vertical green-blue bar (x = 0.541; y = 0.044), 2° wide, and usually 33° high, presented for a single frame (3.3 ms) at an intensity nearly equiluminant (11 cd/m²) with that of the red background. We used near equiluminant stimuli because they are not subject to saccadic suppression (Burr, Morrone, & Ross, 1994). A black ruler with ticks from −12 to 12 (1 unit corresponding to 1.9°) was present on the screen 3° above the level of the fixation point after the end of each trial. 

Eye movement measurement

Eye movements were monitored by an infrared limbus eye tracker (HVS SP150). The horizontal resolution was 0.01°, and accuracy was 0.1° (manufacturer's specifications). The infrared sensor was mounted below the right eye on transparent wrap-around plastic goggles through which observers viewed the display screen binocularly. The sensor was mounted very close to the eye, so the observers were unaware of its presence. The PC sampled eye position at 1000 Hz and stored the trace in digital form after suitable linearization. Eye traces of observers, together with target and stimulus presentation times and observer responses were stored in digital form for later off-line analysis. Average latencies for each observer were estimated after the first few trials, and this information was used, when necessary, to display the bar at a given time with respect to saccadic onset. However, actual delays were calculated off-line. 

Procedure

Observations were made in a totally dark room. The observers sat comfortably and their hand was always hidden from view. They observed the monitor through a liquid crystal shutter positioned very close to the nose. Trials began with a dark fixation spot, which appeared 8.5° to the left of the center of the screen and stayed on thereafter. After an observer achieved steady fixation, a target appeared at 8.5° right of center; the observer immediately saccaded to it, producing a 17° horizontal saccade. A PC, housed in a separate room, recorded the trace on line and detected the onset of the saccade with an approximate delay of about two to three frames (7–10 ms, the main reason to drive monitor at a fast rate, corrected during the off-line analysis). At a fixed delay from saccadic onset, the liquid crystal shutter closed and the monitor became black (0.2 cd/m²). No residual light was available to the observer. On each trial, the bar was briefly presented at a random position and observers were required to report the positions of bars with reference to the ruler or to touch ballistically the screen with the index finger of their right hand. The trial was annulled if the subjects did not see the bar. This happened when the bar was presented after the closure of the shutter or when it was displayed during the saccade, particularly for bars in the far periphery on the left side of the monitor. However, the overall number of aborted trials due to the invisibility of the bar was less than 3% of the total data in the latency between −25 and 10 ms from saccadic onset. For some experiments, the observer also had to report verbally the apparent position of the saccadic target, either alone or together with the position of the bar. It should be noted that the saccadic target remained in view while the shutter was open. It was used during training, and also as a reminder at the end of the saccade trials at the subject's request. The ruler was never visible during trials. 

There were three types of stimulus condition (see Movie ). In Experiment 1 a transient dark was produced simultaneously with the closure of the shutter; by changing the display on the monitor from red to black. This gave a transient dark once the shutter was closed. In Experiment 2 a second monitor (Joyce Electronics) was triggered in synchrony with the shutter, delivering a mean luminance background of about 15 cd/m². In this experiment, the observer viewed both monitors through a half-silvered mirror positioned at 45° between the two orthogonal monitors. The mirror was positioned between the monitor on which the stimuli were presented and the shutter. The luminances of the red background and of the bar were attenuated to 2.8 cd/m² by the mirror. Great care was taken to align the two monitors both to deliver a homogeneous and large light when the shutter was closed, and to avoid showing the frame of the Joyce monitor when the shutter was open. In Experiment 3, a bright photographic flash was delivered synchronously with the shutter closing. The shutter acted as a diffuser, ensuring a uniform bright light. The flash generated a strong bleaching of the retina. After the observers recovered normal vision (usually 30 s), the next trial began. The shutter was opened again 500 ms after the response of the subject and the initial background lighting restored to start a new trial. 

Data for these studies were collected over a period of 3 years in several sections, and results remained stable during this time. All authors served as observers. In addition, two observers who were naïve to the aims of the experiment served as observers for the major conditions but complete data were collected on only one. All data were analyzed off-line. The saccade trace was convolved with a Gaussian (time constant of two samples) and the average gradient of the saccade was evaluated. The saccadic onset and offset were evaluated by linear approximation as the intersection between the two lines that best fitted the pre-saccadic or post-saccadic fixation trace and the saccade itself and the exact latency of the shutter closure calculated. If, due to low-frequency noise on the trace, the program was not able to correctly mark the saccadic onset, the operator could manually shift the cursor to the starting point as evaluated by inspection. However, this happened only on very rare occasions. If the eye trace was unstable due to incorrect fixation or blinks, the trial was eliminated. Data were analyzed in various temporal windows to check the dependence of performance on delay from saccadic onset, reproducing the well-known pattern of results for similar conditions with maximum mislocalization at saccadic onset (Morrone et al., 1997). Only trials in which the bar was presented between −25 and 10 ms from saccadic onset were used for the final analysis. We also verified that extending the time interval from 0 to 10 ms did not alter the pattern of the results, but it increased the sample in some conditions by up to 10%. More than 50% of the trials did not meet the temporal condition and were therefore eliminated. To check the visibility of bars in Experiment 3, observers were required to report in a 2AFC the vertical position of half-length bars as well as their horizontal position. The second judgment was performed only if the bar was visible. Results of horizontal reports are reported only for bar positions for which verbal reports are accurate to better than 70% in range between −25 and 10 ms from saccadic onset. Overall accuracy was high. Accuracy was lowest for presentations midway during saccades, and these data were not included in the analysis. 

Results

In all the experiments reported here, the stimuli were vertical bars displayed for 4 ms, running the full height of the display screen or half that height. Observers reported position either verbally by reference to numbered positions on a remembered ruler or by jabbing at a touch screen (pointing). Results are reported only for bars displayed in the interval from 25 ms before to 10 ms after the start of saccades, as determined by an off-line analysis of eye movement traces. Previous studies that monitored the time course of compression have clearly indicated that this is the optimal time to observe maximal mislocalization (see Morrone et al., 1997; Ross et al., 1997). 

Experiment 1: darkness follows the shutter

Figure 1 shows the effect of varying shutter latency, with darkness following the shutter, on both verbal reports and pointing for a naïve observer, PM. There is a clear dissociation between verbal report and pointing: Verbal reports of position are compressed at all three latencies while pointing responses show no compression except at the longest shutter latency (340 ms). Compression is most evident for the range of real bar positions from 0° (the center of the screen) to 20° (to the right of center). Over this range of bar positions, the range of positions PM reports verbally at all three shutter latencies is less than 20°, so the slopes of his response functions in this range are well below unity (0.26 ± 0.03, 0.33 ± 0.03 and 0.07 ± 0.07 for shutter latency of 25, 55 and 340 ms, respectively). In contrast, the pointing functions are much steeper at the two shorter latencies (0.76 ± 0.05 and 0.96 ± 0.08 for shutter latency of 25 and 55 ms, respectively). There is thus a clear dissociation between verbal report and pointing at the two shorter latencies. But there is no dissociation at the longest shutter latency, 340 ms (0.13 ± 0.04). At this latency, when visual references remain in view longer, there is compression for pointing as well as for verbal report. This is consistent with what Burr et al. (2001) found when observers were allowed to view the display screen for an unlimited time after making saccades: There was compression for pointing as well as for verbal report. Two other observers (MCM and AMW) gave a pattern of results similar to PM's. During the data collection, all subjects reported two interesting qualitative effects: They reported that they based their perceptual localization on the relative distance from the position of the saccadic target, and that during ballistic pointing they often had the sensation that the hand moved towards a different position from where the bar was perceived. These reports are in agreement with the quantitative data of dissociation between the two tasks. 

Figure 1

 

Locating a bar by verbal report (panels A, C, and E ) and by pointing with an unseen hand (panels B, D, and F) for observer PM (a naïve observer). The shutter closed 25 ms (A and B), 55 ms (C and D), or 340 ms (E and F) from saccadic onset. As the shutter closed, the display screen went dark and no other light was present in the room. Each data point is the mean of three to eight trials with bars presented in the interval from 25 ms before saccadic onset to 10 ms after. The horizontal lines in each panel show the location of fixation (−8.5°) and saccadic target (8.5°); the diagonal lines show the locus of veridical responses. The verbal reports of PM show response compression at all latencies. His pointing responses show little or no compression at the shorter shutter latencies (25 and 55 ms) but substantial compression at the longest latency (340 ms). Compression is most evident for bar positions from 0° (the center of the screen) to 20° (beyond the saccadic target). Error bars show standard error of the mean.

Figure 2 shows the results for one of the other observers, JR who explicitly stated that he was localizing the bar without relying on relative distance from the saccadic target. Here there is an interesting divergence from the pattern of results for PM. At the intermediate shutter latency of 55 ms JR, like PM, shows a clear dissociation: There is compression for verbal report, but not for pointing. At 340 ms, the longest latency, he shows strong compression both for verbal report and for pointing, again like PM. But at 25 ms, the shortest shutter latency, JR shows very little if any compression for verbal report (red symbols). 

Figure 2

 

Locating a bar by verbal report (panels A, C, and E) and by pointing with an unseen hand (panels B, D, and F) for observer JR (an experienced observer). JR's responses are very like those of PM except for verbal report at the shortest shutter latency (25 ms) where PM shows some response compression and JR shows none.

The apparent discrepancy between the data of Figures 1A and 2A can be reconciled by considering whether the two subjects based their report on absolute or relative distances. Awater (2002) has shown that, in total darkness, some observers substantially misjudge the position of the saccadic target, but others do not. JR and another two observers, MCM and AMW, were therefore asked to make two verbal reports, one of target position and one of the distance between target position and bar position, with a shutter latency of 25 ms followed by dark. Figure 3 shows that at a shutter latency of 25 ms JR substantially misjudges target position, whereas MCM (another experienced observer) makes only small errors. The misjudgment of target position for verbal report is quite surprising given that the saccadic target is present on average for more than 200 ms. Nevertheless, perceptually the target moves towards the bar, generating a compression of the relative distance. 

Figure 3

 

Locating both saccadic target (panels A and C) and bars (panels B and D) by verbal report at a shutter latency of 25 ms, followed by dark. Observer JR makes large errors in absolute target location (panel A); MCM (also an experienced observer) does not (panel C). Both observers show considerable response compression in relative bar location, that is, distance between bar position and target position corrected for real target position (panels B and D). Because her absolute reports of target position are accurate, MCM like PM also shows compression in her absolute verbal reports of bar position (data not shown), but JR, whose reports of target position are inaccurate, does not (see also Figure 2, panel A).

Figure 3 also shows the reported relative distance of bar from target position. These relative positions are illustrated as real target position plus the distance between target and bar position. There is substantial compression in the relative bar positions reported by both observers, although JR showed no compression in his reports of absolute bar position shown in Figure 2 (panel A). Once the perceived location of the target is taken into consideration, it is evident that for observer JR space is compressed at the time of saccades for a shutter latency of 25 ms resolving the apparent contradiction with the data of Figure 1A. Altogether, these results show that compression is evident for all observers, at all shutter latencies when darkness follows the onset of the saccade obliterating all post-saccadic visual reference. 

We found it impossible to measure the relative distance between bar and target with a ballistic pointing action. However, in separate runs of the same session, we asked three subjects (JR, MCM, and AMW) to point to the bar or the saccadic target. The results showed that observers never made errors in any condition in pointing at the saccadic target (see example for JR in Figure 4). 

Figure 4

 

Pointing to the saccadic target for observer JR as function of bar position, for the same stimulus condition of Figure 3.

Experiment 2: light follows the shutter

Conditions in Experiment 2 were the same as in Experiment 1 except that as soon as the shutter closed it was flooded with a homogeneous light background at a luminance of 15 cd/m⁻². The uniform bright light, presented after the shutter closed, ensured that light was available but visual references were not. Observers made verbal reports target position and distances between target and bar, or pointed at the bar. 

Figure 5 shows the relative verbal reports of bar position (apparent separation) and the pointing responses of MCM and JR. There is substantial compression in their verbal reports, but none in their pointing responses. Similar results were obtained for a third observer, AMW. At a shutter latency of 25 ms, there is a strong dissociation between verbal report and pointing, as there is when darkness follows closure of the shutter. 

Figure 5

 

Locating a bar by verbal report (panels A and C) and by pointing (panels B and D) with an unseen hand at a shutter latency of 25 ms when shutter closure was followed by a homogeneous light background (15 cd/m²). Verbal reports are relative separation between bar and target apparent position corrected for real target position; pointing responses are absolute positions. Both MCM and JR show strong compression in their verbal reports (panels A and C) and none in their pointing responses (panels B and C). Because in control trials both observers were always veridical in pointing to the target (results not shown), absolute and relative pointing responses can be regarded as identical.

The perceived relative distance between bar and target is subject to similar compression independently of whether dark or light follows the shutter (compare Figure 3D with Figure 5C for JR). We therefore conclude that compression for verbal report does not depend on the mean luminance of the post-saccadic display. Furthermore, our results also show that the dissociation between compression for verbal report and pointing does not depend on the mean luminance of the post-saccadic display. Compression for pointing is sensitive to shutter latency and therefore the delay between the presentation of the bar and the pointing response rather than the mean luminance of the post-saccadic luminance display. 

Experiment 3: a blinding flash

In Experiment 3, the shutter closed at 25 ms, as in Experiment 2, and triggered an immediate photographic flash, obliterating any visual references. Care was taken to diffuse the light from the flash so that it did not provide a visual reference point. The difference between Experiments 2 and 3 is the level of light (higher in Experiment 3) and the duration of the light (very short in Experiment 3). However, in this experiment the subject reported a feeling of spatial disorientation usually common during abrupt retinal bleach. 

There is again dissociation between verbal report and pointing, as illustrated by the results of observer PM (Figure 6). He shows strong compression for verbal report of relative bar position and little or none for pointing. Given that no visual references were provided by the flash, the observer had very little or no post-saccadic visual information for spatial orientation. Nevertheless, he showed strong compression, indicating that post-saccadic visual references are not necessary for compression. More surprisingly, despite the subjective feeling of spatial disorientation subjects could point quite accurately to the bar, demonstrating that the motor map is robust and reliable. 

Figure 6

 

Relative verbal reports and absolute pointing responses for PM at a shutter latency of 25 ms when shutter closure was followed by a blinding flash. Verbal reports show strong compression, pointing responses do not.

There is the possibility that the flash might mask the stimulus bar, compromising the perceptual localization results of Experiment 3. All four observers were therefore asked to perform a dual task with half-length vertical bars, to report their horizontal position with reference to a remembered ruler and their vertical position (top or bottom of the screen). Figure 7 shows results for one observer, MCM, which are typical of all observers. Her reports of relative horizontal position (reported only for positions at which reports of vertical position are better than 70% correct) are strongly compressed, particularly for positions to the right of the original fixation point (panel A) where the reports of vertical position are almost always correct (panel B). 

Figure 7

 

Relative verbal reports by MCM of the horizontal position of a short bar and of its vertical location (above or below the target). Shutter latency was 25 ms, and shutter closure was followed by a blinding flash. Reports of vertical location are 100% correct over most of the visual field, where reports of horizontal position show strong compression. Results for reports of horizontal position are included only for positions for which reports are accurate to 70%.

To summarize, Experiments 2 and 3 show that when verbal reports of bar position are corrected for errors in the perception of target position, there is always a dissociation between verbal report and pointing, except at a shutter latency of 340 ms when visual references remain in view for an appreciable time after a saccade has ended. Figure 8 brings out the pattern clearly by plotting indices of compression for verbal report against indices of compression for pointing under the corresponding conditions. The index of compression is the slope of the linear fit to responses for real bar positions from 0° to 20°. The figure shows that slopes of verbal reports are low and in many cases flat with values scattered around zero, indicating strong compression for all observers in all experimental conditions. In contrast, slopes are high for pointing, all greater than 0.8 and scattered around 1, in all conditions except where the shutter closes 340 ms after saccades begin and visual references are therefore available for about 300 ms (veridical performance would yield unity slope). In Figure 8, we have included verbal localization data for JR once it had been corrected for perceived target location (Figure 3D). JR was the only subject to show a prominent effect of saccadic target mislocalization at a short shutter latency. For all the other subjects, the correction of relative distance did not substantially change the estimate of the slope, but these subjects also explicitly stated they used relative distance base to locate the bar. 

Figure 8

 

Slopes of linear fits to verbal report and pointing responses in five conditions to bar positions within the range 0–20° (screen center to right periphery). The color of symbols indicates the experimental condition; each shape represents a different observer (stars AMW, squares JR, circles PM, and triangles MCM). The slope for verbal reports is low in all conditions, indicating response compression. It is high for pointing in all conditions except at the longest shutter latency, 340 ms. Error bars indicate standard error in the fit of the slope.

Discussion

By using a uniformly bright field (Experiment 2) and a photographic flash (Experiment 3), we were able to dissociate the effects of light and visual references. Our results, as summarized in Figure 8, show that when corrected for errors in the perception of target position, verbal reports of the position of stimuli flashed briefly just before or early in saccades always show compression. They also show that compression is not seen in the pointing response except at a shutter latency of 340 ms, when visual references remain in view for almost 300 ms after a saccade has ended. It should also be noted that pointing responses do not show a shift component for shutter latencies below 100 ms. 

Some previous studies (Cai et al., 1997; Honda, 1991; Lappe et al., 2000) failed to show perisaccadic compression of space in perceptual localization. These studies were conducted in the dark and the lack of compression was attributed to the absence of post-saccadic visual references. The present results reject this explanation and clearly show that post-saccadic references are not necessary for compression. The compression that we find was probably obscured in those previous studies by the simultaneous mislocalization of the saccadic target, as happened here for one subject and for many subjects in the Awater (2002) study (conducted in total darkness). Taking into account the fact that compression is never observed when a saccadic-like retinal motion is generated during fixation, the present results support the view that compression is mediated primarily by the corollary discharge associated with an eye movement. The compression effects reported in this paper are somewhat different in size and range from those previously reported by us (Awater, Burr, Lappe, Morrone, & Goldberg, 2004; Burr et al., 2001; Morrone et al., 1997; Ross et al., 1997) and by others (Awater & Lappe, 2004; Kaiser & Lappe, 2004; Lappe et al., 2000; Matsumiya & Uchikawa, 2001; Matsumiya & Uchikawa, 2003). Nevertheless, compression is found over a substantial range of bar positions and the divergence between positions as reported verbally and as indicated by pointing can be as large as 20°. This study used a faster frame rate than previous studies, a smaller display screen and a lower luminance level. These differences may explain differences from previous studies in the size and spread of effects. Another feature of the studies reported here is that when observers made verbal reports they had to remember a ruler that was visible in most previously published experiments. However, it is unlikely that memory effects influenced our results. Compression is found only within the same narrow perisaccadic window that it occupied in previous experiments and verbal reports in this experiment. In the intervals just outside that window, from −30 to −60 ms (before a saccade) and from 30 to 60 ms (after its start), reports are highly veridical. 

The present data also help to clarify some contradictory results in the literature concerning the dissociation between motor accuracy and perceptual error during saccades. 

Our pointing results at short shutter latency closely agree with those of Bockisch and Miller (1999), Hansen and Skavenski (1977, 1985), and Miller (1996), showing that at saccadic onset, gaze and hand pointing to the test stimulus are accurate. However, Miller and Bockisch found large errors after the end of the saccade. Dassonville et al. (1992, 1995) found large localization errors for secondary saccades, which started at least 200 ms before saccadic onset. Interestingly, the error depended on eccentricity, with smaller errors observed for secondary saccadic targets closer to the target for the first saccade, particularly for the no-gap paradigm where some visual references remain available. The decrease of error with eccentricity, previously interpreted as an increase in accuracy (Dassonville et al., 1992, 1995; Honda, 1995), is actually a signature for spatial compression. Thus, our pointing results at long shutter times are in agreement with Dassonville et al.'s data. 

As discussed by Bockisch and Miller (1999), the discrepancy between the various studies can depend on dark adaptation and stimulus brightness. These two variables affect the processing delay of the visual stimulus and hence the temporal synchronization between the internal corollary discharge signal, the visual response, and the amount of visual references available post-saccadically. At very low levels of adaptation and stimulus luminance, as in Dassonville et al.'s (1992, 1995) experiments, a stimulus presented at saccadic onset may interact with a later phase of the corollary discharge signal and, if at this time visual references are available (highly possible given that subjects were fully dark adapted), a compressive mislocalization should be found, in agreement with the present data at longer shutter latencies. At medium (Bockisch & Miller, 1999; Miller, 1996) or high levels of light adaptation (Hansen & Skavenski, 1977, 1985), the visual responses are faster and the motor map can prevail to mediate an accurate localization, especially when vision becomes unreliable with transient darkness and the obliteration of visual references. The conditions at short shutter latencies reported here would mimic these later experimental conditions and in fact do produce a similar pattern of results. 

We conclude that vision must have access to different representations of space, one of which is subject to compression and another of which is not. Pointing favors the use of the undistorted representation, but does not make it mandatory. What are these different representations of space and what determines their use? And why do saccades cause the distortion of one of them? 

Recently, there has been much interest in the question of whether there are separate pathways for vision and action (Goodale & Milner, 1992). To investigate this question, it has been popular to examine the effects of context on both visual and pointing responses (Aglioti, DeSouza, & Goodale, 1995; Bridgeman, 1981; Bridgeman, Peery, & Anand, 1997). The results have not been clear—some report dissociation while others do not. A compelling case has also been made that when the task is equivalent for the two systems, responses are comparable (e.g., Franz et al., 2000; Ma-Wyatt & McGraw, 2003; Yamagishi, Anderson, & Ashida, 2001). Interestingly the dissociation, when present, can be annulled by forcing the sensorimotor system to use the memory representation of the target (Bridgeman et al., 1997; Dassonville & Bala, 2004; Hu & Goodale, 2000) or by allowing vision of the stimulus during the initial preparation of the motor command (Westwood & Goodale, 2003). These results are consistent with the present results of erroneous localization by pointing in presence of post-saccadic visual references. 

There is ample neurophysiological evidence that vision establishes different frames of reference, gaze centered, head centered, body centered, and possibly others, combining them probabilistically for different tasks. It has been argued on computational grounds that to perform different tasks it must have access to multiple representations of space and the capacity to combine them (Andersen & Buneo, 2002; Andersen, Snyder, Li, & Stricanne, 1993; Bremmer, Pouget, & Hoffmann, 1998; Boussaoud & Bremmer, 1999; Buneo, Jarvis, Batista, & Andersen, 2002; Cohen & Andersen, 2002). It is generally agreed that a weighted combination of proprioceptive and visual information is used to make a goal directed point and to correct a trajectory online (e.g., Saunders & Knill, 2004; Sober & Sabes, 2003). Visual information is weighted more heavily than proprioceptive information to localize the target in space, most likely in a viewer centered framework (e.g., van Beers, Baraduc, & Wolpert, 2002). This visual information then undergoes a transformation to be used to guide the hand. There is evidence to suggest that this final representation may be hand centered rather than gaze/viewer centered (e.g., McIntyre, Strata, & Lacquaniti, 1998) and that these maps evolve over time (e.g., Ilg, Schumann, & Thier, 2004). These models emphasize the necessity to weight the different sources of sensory feedback used in this position estimate, and accordingly to calculate uncertainty for a given position. We suggest that conscious perception, the basis for verbal report, always favors a gaze-centered representation (not to be confused with merely retinotopic excitation). When a saccade is being prepared, receptive fields shift in anticipation of a shift in position of the direction of gaze (Duhamel, Colby, & Goldberg, 1992). These shifts cause changes in the apparent directions of stimuli flashed, and so they are seen erroneously, shortly before and early in saccades, when the eye has not yet made the anticipated movement. 

We also suggest that pointing relies on a combination of a gaze-centered representation with others centered elsewhere, such as on the hand. Combination of different representations may be achieved by means of gain fields (Cohen & Andersen, 2002). Weights given to different maps will depend on how reliable these maps are on viewing conditions (Niemeier et al., 2003; Pouget et al., 2002) and on task demands. When a saccade is imminent, we suggest it is the preexisting hand-centered map that provides the dominant input to the combination used for pointing, avoiding the effects of anticipatory receptive-field shifts. Pointing to targets is therefore accurate for targets flashed soon before or very early in saccades, when the eye has turned little or not at all. Errors in pointing by hand should increase as the eye shifts during saccades, while it relies on a pre-saccadic map, because as the eye position shifts, the pre-saccadic map being used is increasingly out of registration with the location of objects. Our results indicate that there is little or no error at saccadic onset, and the results reported by Bockisch and Miller (1999) indicate that errors are greatest toward the end of saccades and small at their start. Perception, on the other hand, is subject to errors at the start of saccades, and even 70 ms before, because at this time receptive fields mediating the perceptual localization are shifting, and because the reference point for its representation of space is shifted, presumably by an extra-retinal signal (Morrone et al., 1997). 

From electrophysiological recordings from a number of visual areas, it is now well established that only in a few visual areas (probably only V1 and V2) receptive fields do not change perisaccadically (Duhamel et al., 1992; Kusunoki & Goldberg, 2003; Nakamura & Colby, 2002; Tolias et al., 2001; Umeno & Goldberg, 1997; Walker, Fitzgibbon, & Goldberg, 1995). Many neurons start to respond to stimuli positioned at locations different from their direct retinal afference before an eye movement. These effects are particularly evident in LIP but also very common in V3, V3A and V4. For some of these neurons, the size of RF can shrink, such as cells in V4 (e.g., Tolias et al., 2001), or enlarge extending to position in opposite direction of the saccade (e.g., Kusunoki & Goldberg, 2003). 

But what causes the compression of distances? At the time of saccades, receptive field shifts are rapid, probably to maintain visual stability. We suggest that compression is a by-product of these rapidly shifting receptive fields. A retinocentric RF that undergoes retinal remapping before the saccade, like those in LIP or V3A, becomes de facto transiently craniotopic from the moment of the initial remapping until the next plan to move the eye. This activity compensates for the retinal shift induced by the eye movement. This is an aspect of remapping often not stressed, but one that may correspond to the dynamic mapping needed to explain compression of space. The physiological estimate of the remapping is fast: A complete predictive shift of the RF takes place in less than 100 ms for stimuli of 100 ms or 50 ms duration (Kusunoki & Goldberg, 2003; Nakamura & Colby, 2002) for a range of saccadic size from 10° to 20°. Recently, there have been reported cases of dynamic remapping of craniocentric neurons of VIP (Duhamel, Bremmer, BenHamed, & Graf, 1997; Galletti, Battaglini, & Fattori, 1995; Kubischik, 2002). Some of these cells shift their RFs in the direction opposite to the saccade so their RFs remain anchored to the external position independently of the saccade. They reach maximum activity 20 ms after the start of the remapping and complete it well before saccadic onset (Figure 3.16B of Kubischik, 2002). If these fields shift at speeds that approach the limit for the neural information transfer, then, as argued by Morrone, Ross, and Burr (2005), relative distances of transient stimuli signaled by them will be compressed, following the Lorentz transform. However, visual stimulation present before the remapping and persistent after the remapping will be sensed as constant and stable. 

Conclusion

We conclude that vision must have access to different representations of space, one of which is subject to compression and another of which is not. The perceptual map goes astray during saccades leading to errors in verbal report and to compression of relative distances. The results show that the compression is not mediated by post-saccadic visual references, as previously believed. Instead, the results indicate that compression is mediated by an internal corollary discharge signal. The positions indicated by blind pointing are veridical and robust under many impaired visual conditions, indicating that vision is kept grounded during saccades. The motor map shows no compression except when visual references remain in view for a substantial time after saccades, providing an explanation for the contradictory results present in the literature about the dissociation between motor and perceptual maps. We suggest that these two maps can be combined with various weightings, depending on conditions of viewing and the task to be performed, to perceive and report or to point. 

Supplementary Materials

Movie - Movie File 

Acknowledgments

This work was supported by an NHMRC Project Grant (JR), a Human Frontiers Science Foundation Grant (MCM and JR), a PRIN 2003 from the Italian MIUR, and the Rachel C. Atkinson Fellowship (AMW). 

Commercial relationships: none. 

Corresponding author: Anna Ma-Wyatt. 

Email: [e-mail]. 

Address: The Smith-Kettlewell Eye Research Institute, 2318 Fillmore Street, San Francisco, CA 94115, USA.