Spatiotemporal dynamics of perisaccadic remapping in humans revealed by classification images

Abstract

We actively scan our environment with fast ballistic movements called saccades, which create large and rapid displacements of the image on the retina. At the time of saccades, vision becomes transiently distorted in many ways: Briefly flashed stimuli are displaced in space and in time, and spatial and temporal intervals appear compressed. Here we apply the psychophysical technique of classification images to study the spatiotemporal dynamics of visual mechanisms during saccades. We show that saccades cause gross distortions of the classification images. Before the onset of saccadic eye movements, the positive lobes of the images become enlarged in both space and in time and also shifted in a systematic manner toward the pre-saccadic fixation (in space) and anticipated in time by about 50 ms. The transient reorganization creates a spatiotemporal organization oriented in the direction of saccadic-induced motion at the time of saccades, providing a potential mechanism for integrating stimuli across saccades, facilitating stable and continuous vision in the face of constant eye movements.

Introduction

One of the main mysteries of spatial vision is how the perception of the world remains stable in the face of continual movements of the eyes. Although saccades are fundamental to active perception, the continual displacement of the retinal image each time the eye moves poses many challenges to the visual system. It is well established that each eye movement is accompanied by an active internal signal (the corollary discharge), which has clear physiological and psychophysical effects. For example, RFs of many neurons in the lateral intraparietal area (LIP) change drastically at the time of saccades, shifting in the saccadic direction, before the eyes have moved (Duhamel, Colby, & Goldberg 1992). Anticipatory shifts have also been observed in other areas, including superior colliculus (Walker, Fitzgibbon, & Goldberg, 1995), frontal eye fields (Sommer & Wurtz, 2006), area V3 (Nakamura & Colby, 2002), and to a lesser extent, V1 (Wurtz, Joiner, & Berman, 2011). 

Evidence for spatial updating and remapping have also been reported in humans in several studies, demonstrating a role of the posterior parietal cortex (Bellebaum, Hoffmann, & Daum, 2005; Chang & Ro, 2007; Khan, Pisella, Rossetti, Vighetto, & Crawford, 2005; Medendorp, Goltz, Vilis, & Crawford, 2003; Merriam, Genovese, & Colby, 2003; Pisella & Mattingley, 2004; Prime, Vesia, & Crawford, 2011; Rushworth & Taylor, 2006) and of extrastriate visual areas (Han, Xian, & Moore, 2009; Khan et al., 2005; Merriam, Genovese, & Colby, 2007; Tolias et al., 2001) in maintaining visual stability during saccades. 

Saccades also cause gross perceptual errors in localization of briefly flashed stimuli, displacing them toward the saccadic landing point (Honda, 1989; Ross, Morrone, Goldberg, & Burr, 2001). They also affect the perceived timing of stimuli (Binda, Cicchini, Burr, & Morrone, 2009; Morrone, Ross, & Burr, 2005). The effects of saccades on space and time are strongly correlated (Burr, Ross, Binda, & Morrone, 2010), suggesting common neural mechanisms characterized by the perisaccadic deformation of neuronal receptive fields (RFs) in many visual areas. The potential role of saccades on visual perception has been investigated by a large number of studies (for a review, see Cavanagh, Hunt, Afraz, & Rolfs, 2010; Melcher & Colby, 2008; Prime et al., 2011). However, a direct link between neuronal RFs and perceptual distortions is still lacking. 

In this study, we apply the Classification Images Analysis technique to probe the spatiotemporal structure of perceptive fields during perisaccadic remapping. Classification Images Analysis was introduced to vision research by Ahumada and Lovell (1971; see also Abbey & Eckstein, 2002; Ahumada, 2002; Murray, 2011; Neri & Levi, 2006; Shimozaki, Chen, Abbey, & Eckstein, 2007), following the seminal ideas of Volterra (1930) and Wiener (1958) who showed that the first-order kernel (also called the impulse-response function) of a stable system with limited memory can be obtained by cross-correlating the noise input with the system output. The technique has been used in many neurophysiological studies, both at the single-cell level (Bredfeldt & Ringach, 2002; Ringach, 2004) and also with gross EEG signals (Schyns, Petro, & Smith, 2007; Smith, Gosselin, & Schyns, 2007). For psychophysical and behavioral responses, the output of the system is sparse and discrete (usually a “yes” or “no” response), but it has nevertheless been successfully applied in many perceptual studies, including vernier and grating acuity (Ahumada, 1996), motion (Ghose, 2006; Neri & Levi, 2008), and stereoscopic vision (Neri, Parker, & Blakemore, 1999). Indeed, there are several examples where this technique corresponds more closely to physiological results than does standard psychophysics, such as revealing a reversed depth with pixel-inverted stereo pairs (Neri et al., 1999). Neri and Levi (2006) provide an interesting discussion of the relationship to what they term “perceptive fields” (the psychophysical equivalent of receptive fields) and physiologically defined receptive fields. 

In this study, we apply the “agnostic ”classification image technique to investigate perception in humans at the time of saccades. The results reinforce and extend previous psychophysical studies, which necessarily rely on subjective reports. 

Methods

All the experiments were performed in a quiet dark room. Five subjects (one author and four naive to the goals of the study, with normal or corrected-to-normal vision, aged 25.5 ± 2.5) participated in the study. MP, DM, and LLV were observers in Experiment 1, collecting 10,000 trials each, and MP, PG, and MA collected 6000 trials in Experiment 2. Subjects sat 30 cm from a monitor screen (70° by 50°) with eyes at the height of the screen center. Stimuli were generated by a VSG graphics card (Cambridge Research System, VSG2/5 framestore) and displayed on a CRT monitor (Barco Calibrator) at a resolution of 800 × 600 pixels, refresh rate of 140 Hz, and mean luminance of 20 cd/m². Eye movements were recorded by an infrared limbus eye tracker (ASL 310) at a sample rate of 1000 Hz with sensor mounted below the right eye recording the eye position during binocular viewing. 

Stimuli and task

In the Experiment 1 (illustrated in Figure 1), each trial started with subjects fixating a black fixation point (f ₀), 9° left of screen center. After a warning cue (the fixation point briefly turned white), the fixation point disappeared and a saccadic target (f ₁) appeared 9° right of screen center, to which subjects saccaded as rapidly as possible. At a random interval (between 50 and 150 ms) after display of saccadic target, the 170-ms stimulus sequence started. The sequence comprised eight 21.3-ms frames, each containing independent strips (1.5 × 24°) of one-dimensional, luminance-modulated white noise centered 1.5° above and below fixation. Each matrix comprised sixteen 1.5° × 1.5° pixels of whose luminance was chosen at random from a discrete uniform distribution (between 15 and 25 cd/m²). 

Figure 1

 

Stimuli and task for Experiment 1. Trials comprised eight 21-ms frames, each comprising two arrays of unidimensional, luminance-modulated white noise (in the upper and lower screens). Each noise array covered a region of 1.5° × 24°. The signal was a luminance increment 4.5° wide, added to the upper or lower noise sequence (at random) during the 5th frame (from 85 to 106 ms after the beginning of the noise sequence), 4.5° left of center (spanning three noise pixels: dashed circles). f ₀ and f ₁ represent the saccadic fixation and target point, respectively. The white arrow represents the saccadic direction.

The stimulus to be detected was a bright bar in the fifth frame of the sequence (85 ms after sequence onset), added to the 6th, 7th, and 8th squares (spanning −4.5 to 0°) of either the upper or lower noise strips. Subjects were required to report, in two-alternative forced choice (2AFC), whether the bar appeared in the upper or lower strip. Every 20 trials, a stimulus was presented in the two possible positions to remind the subjects of the stimulus locations, luminance, and size. 

Because of the random onset, and natural variability in saccadic latency, the bar stimulus was presented at variable times relative to saccadic onset. Figure 2 shows the total number of trials, as a function of time from saccadic onset. The stimulus contrast ranged from 0.16 to 0.25, adjusted throughout the experiment to maintain accuracy near 75% correct for each subject. In practice, performance varied little with latency from saccadic onset. The upper red curve of Figure 2 shows average percent correct, as a function of time from saccadic onset (always around 75% correct). 

Figure 2

 

Total number of trials for each subject and average percent correct, as a function of time from saccadic onset, for Experiment 1. The black vertical dashed lines delimit the time frame for which the CIs were analyzed. The upper red curve shows average percent correct across subjects, as a function of time from saccadic onset, and the red dashed line shows average performance across temporal conditions.

Experiment 2 (Figure 7) used only 3 frames of noise, triggered by the saccadic onset. The bar stimulus was presented in the first frame between −1.5 and −4.5°, again adjusted to yield 75% correct responses (varying contrast from 0.10 to 0.21 between subjects). In the saccadic condition, each trial started with a cue (small white square, positioned 7.5° left of screen center), followed by the black saccadic target 7.5° right of fixation, to which subjects saccaded. Again, the fixation spot disappeared on the appearance of the saccadic target. In the fixation condition, both the white cue and the black fixation point were in the center of the screen. No feedback was provided in either experiment. Dynamic examples of a pre- and post-saccadic trial can be downloaded online (Supplementary material). 

Saccadic onset was detected by offline analysis of each individual eye trace by linear fitting of the space–time trajectory. Trials with invalid eye movements were discarded from further analyses. These were trials where the saccadic latency was less than 100 ms (anticipation), or the saccade did not reach goal (more than 10% inaccuracy) or there was a corrective saccade. In practice, less than 5% of trials were discarded. Valid trials were binned on the basis of saccadic latency (τ), usually into bins of 21-ms width, sometimes larger (e.g., for Figure 3). After binning for saccade latency, the images were aligned with stimulus presentation. 

Figure 3

 

Classification images averaged over three subjects, for stimulus presentation at different times (τ) relative to saccadic onset. The left panels show the eye movement trace (in blue) with the red bar indicating the target presentation. The upper panels (a–d) show the classification images for all data (Equation 1) averaged across subjects, and the lower panels (e–h) show the averages of only the false alarm trials (Equation 2). The black polygon represents the target position in space and time, and the thin white arrows and dotted lines represent the average retinal projection of the fixation (f ₀) and saccadic target (f ₁) points, respectively. The thick white arrow represents the fovea. Response is plotted in Z scores (−3 to 3) with color-coding defining negative response with blue shades, non-significant response (−1 to 1) in gray, and positive response in red–yellow in retinal (left) and external spatial coordinates (right).

Data analysis

As described above, the experimental design was a 2AFC discrimination task. Following Neri and Heeger (2002), we convert the 2AFC responses into standard signal detection classes, separately for the two noise strips. Correct responses are classified as a “hit” in the noise strip where the stimulus was presented and a “correct rejection” in the other. Errors are classified as a “miss” in the noise strip where it was presented and a “false alarm” in the other. In both cases, each trial yields two noise images for analysis. This technique has several advantages over a simple yes/no detection, including maintaining criteria constant, as well as providing two classification images for each trial, halving the number of required trials (Neri, 2011). 

For each temporal bin (τ), we averaged separately the noise images that led to the four response classes. The full classification image (μ) for a particular temporal bin τ _i is then defined as the averaged sum of the images yielding hits and false alarms minus those yielding misses or correct rejections: 

$\mu \left({\tau }_{\mathrm{i}}\right)=({\mu }_{\mathrm{H}}+{\mu }_{\mathrm{F}\mathrm{A}})-({\mu }_{\mathrm{M}}+{\mu }_{\mathrm{C}\mathrm{R})},$

(1)

where μ _H, μ _FA, μ _M, and μ _CR, refer, respectively, to the average of the noise images yielding hits, false alarms, misses, and correct rejections. In some cases (lower Figures 3 and 5), only false alarms were considered: 

$\mu \left({\tau }_{\mathrm{i}}\right)={\mu }_{\mathrm{F}\mathrm{A}}.$

(2)

To obtain the CIs for each interval, μ(τ _i) was converted to Z scores by dividing by the standard deviation calculated over all images used in the experiment, then convolved with a spatiotemporal Gaussian filter (σ _s = 0.75°, σ _t = 10.6 ms) to smooth pixel blocking. Color codes for the CIs are indicated by bar plot of Figure 3: red–yellow, positive images (>1 SD) and blue–cyan, negative (<1 SD). Except for Figure 6, which shows individual plots, the classification images were pooled across observers (as the results were very consistent). 

Results

Classification images are usually interpreted as spatiotemporal maps in which each element represents the probability that a particular noise sample is mistaken for the target (Abbey & Eckstein, 2002, 2006; Neri & Heeger, 2002). Positive peaks in the classification image are points where the visual system infers with high probability the existence of the stimulus, while negative peaks show lower than chance (anti-correlated) inference of the stimulus. 

Figure 3 shows classification images obtained for stimuli presented at various times relative to saccadic onset, averaged over all observers into broad temporal bins (42–84 ms). The large (and variable) bins were used to make the effects clear—the following figures (i.e., Figures 4–6) show data with smaller and constant bins. 

Figure 4

 

Classification images for different times (τ) from saccadic onset, averaged across observers, on a finer scale. Conventions as for Figure 3. An animated representation of these results in space and time can be downloaded from the online Supplementary material.

The upper figures show results considering all data (Equation 1), and the lower figures show only false alarms (Equation 2). Importantly, the false alarm CIs, when observers report a bar that was not there, were very similar to those considering all the data, suggesting that the classification images do not result from an interaction between the noise and the signal bar but reflect the structure of the mechanisms used for the detection task. Data are plotted both in retinal coordinates (center plots) and in screen coordinates (right plots). 

Figures 3a and 3e show CIs obtained when the stimuli were presented well before saccadic onset (64–85 ms), when vision is not subject to distortions. The positive lobe in the image (red–yellow, reflecting noise brighter than mean luminance) correspond reasonably well to the position where the stimulus was (or should have been) displayed. There is also flanking negative activity (blue colors, reflecting noise darker than mean luminance), particularly to the right of the bar. Flanking, negatively correlated lobes are commonly observed with this technique, usually interpreted as reflecting a center–surround organization of the presumed underlying receptive field (Neri & Heeger, 2002). Both the positive and negative lobes persist about 64 ms over time, with a gradual temporal evolution as has been previously observed (Burr & Morrone, 1996). Figures 3d and 3h show results for stimuli presented well after saccadic onset, after the eyes have landed. The pattern of results is similar to those of Figures 3a and 3e: In both cases, the position of the positive lobe of the CI corresponds well to the actual position of the stimulus. 

The more interesting results are the changes of the classification images at the time of saccades. Figures 3b and 3f show CIs for bars presented to the stationary eye, just before saccadic onset. To make the effect clearer, we binned together all the data where the stimulus was presented from 64 to 21 ms before the saccadic onset. These CIs are quite different than those for earlier presentations of bars: The positive lobe broadens in both space and in time, and its center of gravity becomes anticipated in time and shifted in space away from saccadic target, toward fixation. The negative lobe also broadens and is anticipated. Although the stimulus itself (when present) was actually highly localized in retinal space and in time, as the eye was still stationary during this interval, the noise that led to a positive response (either hit or false alarm) was spread out in space toward fixation and earlier in time than the actual stimulus. 

For stimuli presented during the saccade (from −21 to 64 ms; Figures 3c and 3g), the stronger positive lobe splits into two regions, one late in time, representing the real position of the target, and the other earlier in time, anchored to the external position of the original fixation (f ₁ at −9°), even though the fixation spot was extinguished 200 ms earlier. In the temporal bin that encompasses the actual eye movement, the spatial position of the stimulus varies over a retinal eccentricity from −12 to 8°: but the foci of the CI are localized around initial and final fixation. It is also important to note that these two foci have the same space–time direction of the saccade-induced motion, indicating an integration of the noise along this trajectory. The latter and broader bin was created to summarize the peri- and post-saccadic averaged behavior of the CI. 

Figures 4 and 5 plot the data on a finer time scale, for all responses (Figure 4) and for only false alarms (Figure 5). The anticipation in time and the spread over space of the positive lobe is very clear for targets presented in the two perisaccadic intervals shown in Figures 4c, 4d, 5c, and 5d even though the eyes are still during these intervals. The response breaks into two separate peaks in the intervals during the saccade (Figures 4e, 4f, 5e, and 5f), with the pre-saccadic lobe becoming progressively weaker over time. Even 64 ms after saccadic onset (Figures 4g and 5g), the response remains delayed, although the overall lobe begins to match the spatiotemporal characteristics of the target (upper yellow focus); 85 ms after saccadic onset, the positive lobe returns to match the spatiotemporal configuration of the signal (Figures 4h and 5h). 

Figure 5

 

Classification images for false alarm trials only, for different times (τ) from saccadic onset, averaged across observers. Conventions as for Figure 3.

The pattern of the results is robust and repeatable over subjects. Figure 6 shows classification images separately for the three subjects tested in this experiment: There is very little intersubject variability, indicating also that individual strategies of searching for a white bar embedded in noise do not affect the results. 

Figure 6

 

Classification images for different times (τ) from saccadic onset, for each observer tested, plotted in spatial coordinates. Conventions for Figure 3.

To provide more evidence about the effect of saccades on classification images, we measured classification images with only three frames of noise and the bar triggered by the saccade (Figure 7a). These stimuli gave rise to similar results as the previous task. During fixation (Figure 7c), the images are very similar to those observed for stimuli displayed well before or after the saccade in the previous study; however, for stimuli presented at saccadic onset the response peaks are shifted toward saccadic targets and delayed in time (Figure 7d), although the delay is somewhat less than in the previous experiment. The earlier positive peak at pre-saccadic fixation cannot be demonstrated in this condition because no noise was presented at that time. 

Figure 7

 

Stimuli, task, and results for Experiment 2 (triggered experiment). (a) Task schematization for the saccadic condition. Trials comprised 3 frames (21 ms each) of two arrays of unidimensional, luminance-modulated white noise (in the upper and lower screens). The signal was a luminance increment 3° wide, added to the upper or lower noise sequence (at random) during the 1st frame (from 0 to 21 ms after the beginning of the stimulation), 4.5° left of center (spanning two noise pixels: dashed circles). f ₀ and f ₁ represent the saccadic fixation and target point, respectively. The white arrow represents the saccadic direction. (b) The stimulation was triggered to the saccadic onset according to an automatic procedure. The blue line represents the trace of the eye movement, with the red bar indicating the frame when the target was presented. (c) Mean classification images obtained for three subjects in the fixation condition (with the fovea centered at eccentricity zero). (d) Mean classification images for the saccadic condition for each subject plotted in external spatial coordinates. The white arrow represents the average position of the fovea. The black box represents the spatiotemporal position of the target. As for Figure 3, positive and negative responses (Z scores from −3 to 3) are plotted in red and blue, respectively, with the non-significant responses in gray. Saccadic amplitude was 15°.

All the results show that classification images become distorted around the time of saccades. To highlight the changes in space and time, we plot separately the location of the local maxima and minima after integrating the CI over space or over time. Figure 8 plots time of the space-averaged CIs (relative to saccadic onset) against physical time (from saccadic onset). The dotted 45° diagonal is the identity line, expected if the maximal responses followed physical time. However, the positive lobes fall clearly below this line in the pre-saccadic intervals and above it in the post-saccadic intervals. The heavy black lines in Figure 8 pass through the maxima of space-averaged CIs. Note that there are two intervals where this curve is relatively flat, before and after the saccade, indicating that the noise stimulus predominantly contributing to the detection tended to be either just before or just after the saccade: at the end of fixation or on saccadic landing. Seldom did the response correspond to the time of the actual saccade. Closer inspection also reveals a strong undershoot after the first flat region and an overshoot at the beginning of the second region. The negative regions of the CI follow a peculiar form, occurring around the central interval at all saccadic latencies. It is not clear why the negative lobe has this form. 

Figure 8

 

Temporal dynamics averaged over space, plotting the time of space-averaged classification images against physical time for the averaged (upper left) and single subjects. The white area represents the time of the saccade, and the diagonal represents the equality line. Positive and negative responses are represented according to the color-coding of Figure 3. The black thick line connects the response maxima.

Figure 9 plots perceived spatial position (averaged over time) in screen coordinates as a function of presentation time relative to saccadic onset (vertical black lines define the physical position of the target). For stimuli presented well before or after the saccade, the average position of the positive lobe generally corresponds to the physical position of the bar, extending between −4.5° and 0° (shown by the dark line). For perisaccadic presentations, however, maximum response is shifted by about 6°—a third of the saccadic amplitude—toward the first fixation target. On average, the negative pattern flanked the positive lobe on the right before the saccade and on the left after the saccade. 

Figure 9

 

Spatial dynamics of classification images, plotting time-averaged spatial images against time from saccadic onset for the averaged and single subjects. Color-coding is the same as Figure 3. The black box represents the signal, and the thick black line connects the response maxima. The fixation points and the fovea are represented by the white dotted lines and the white arrow, respectively.

Discussion

In this study, we used an “agnostic” psychophysical paradigm to study the spatiotemporal dynamics of perceptual mechanisms responding to stimuli presented briefly around the time of a saccadic eye movement. The technique measures the probability that a particular noise element at a certain time and spatial position contributes toward the detection of a light bar. One of the advantages of this technique over more standard psychophysical techniques—such as indicating the apparent position of a briefly flashed bar—is that it does not rely on a perceptual judgment to be made after the saccade is completed, necessarily confounding memory, visual references, and many other complications. For stimuli presented well before or after the saccade, the relationship between the noise contributing to the classification images and the subjective percept is quite clear: All noise contributing to the positive lobe is perceptually associated with the physical position of the white bar (in both space and time). There is some uncertainty about position, producing classification images broader than the stimulus, probably reflecting the spread of activity in the neuronal network associated with the perception of space. Importantly, similar results were obtained when the bar was present or absent, indicating that the noise used for detection can be selected by an a priori knowledge about the target. 

The most conservative approach to interpreting classification images is to consider that they represent the noise that is most probably mistaken as a bar, and this leads to several interesting conclusions (at least for the conditions of this experiment). First, the results show that the spatial range of integration broadens perisaccadically to extend well over 14° of visual field, about two thirds of the saccade amplitude. The spatial smearing cannot be a consequence of the fast retinal motion, as in many instances the bar preceded the motion by 40 ms or so. It is most probably the consequence of the decrease in localization reliability (Binda, Bruno, Burr, & Morrone, 2007), either at a late “decision stage” or at an earlier stage of visual analysis. These effects are in broad agreement with the psychophysical observations of compression of visual space (Lappe, Awater, & Krekelberg, 2000; Morrone, Ross, & Burr, 1997; Ross, Morrone, & Burr, 1997): Larger receptive fields imply that stimuli over a wide area will all be localized in the same space. As well as the increase in the spatial spread, there is also a perisaccadic shift of the center of gravity of the classification images toward the pre-saccadic fixation position, a shift about half the size of the saccade. At first glance, it may appear that the current results run contrary to previous results showing a shift toward saccadic target (Honda, 1989; Morrone et al., 1997; Ross et al., 1997, 2001) rather than backward to fixation. However, it must be recalled that the CIs reflect the distribution of noise that is confused with the stimulus. Thus, noise around fixation is confused with the centrally positioned target, a shift in the direction of the saccade. The magnitude of the shift seems to be less than that often reported with classical psychophysics methods, but there are many factors that can vary the magnitude, including individual differences, size of screen, and luminance. 

Similar deformations occurred in time: There was an increase in the spread over time and also systematic delays and accelerations. The temporal width of the classification images around the time of the saccade doubled, consistently with the two-fold compression in time observed psychophysically (Morrone et al., 2005). In this time window, the classification images showed two strong temporal peaks, one around −43 ms and one at 43 ms from the saccadic onset, which could also contribute to the compression. These separate peaks—preceding and succeeding the saccade—cannot be attributed simply to “saccadic suppression” or to physical blurring by the actual eye movement, for at least two reasons. First, as Figure 2 shows, detection performance of all subjects remained constant around 75% for all presentation times relative to saccadic onset (probably because both stimulus and noise were equally attenuated by saccadic suppression, leaving the ratio unaltered). Second, during the saccade, we found reliable measures of inverse correlation, implying that dark noise pixels contributed to the perception of a white bar. Therefore, it is highly probable that the double-peaked classification image reflects the spatiotemporal profile of functional mechanisms. 

Another interesting aspect of the temporal distortions of the classification images is that the peak responses tended to occur either before the saccade or postponed until after it. Close inspection of the response maxima reveals clear undershoots and overshoots in the function of perceived versus physical time. That is, perceived time does not increase monotonically with physical time but, at key moments, goes backward! However, this very strange pattern is consistent with previously described temporal order judgments (Binda et al., 2009; Morrone et al., 2005), where observers perceive the temporal order of visual stimuli inverted. The point of inversion is well predicted by the non-monotonicity of the plots of perceived against real time. 

Before the eye moves, the receptive fields of neurons in many cortical areas—including LIP, V3, superior colliculus, and FEF—undergo profound changes, often termed “remapping.” The most prominent of these changes is a shift of the receptive fields, in the direction of the saccade (Krekelberg, Kubischik, Hoffmann, & Bremmer, 2003; Kusunoki & Goldberg, 2003; Sommer & Wurtz, 2002, 2008; Umeno & Goldberg, 1997; Wang, Zhang, & Goldberg, 2002). While it is tempting to suggest that our results reflect the action of these cells, the shift is actually in the opposite direction to what we observe. Our receptive fields extend backward (for example, hotspot at fovea before the saccade in Figure 3b), against the direction of the saccade. While consistent with the psychophysical observations of a perceptual shift in the saccade direction, this is inconsistent with many electrophysiological studies. However, the neural effects of remapping are complicated. For example, Zirnsak, Xu, Noudoost, and Moore (2011) show that the perisaccadic shift in frontal eye field neurons is toward saccadic endpoint, not parallel to the saccade vector (correlating well with the psychophysical effect of spatial compression). Interestingly, we have recently shown that saccadic compression is not always toward the saccade endpoint but can be altered by the presence of other stimuli presented around the time of the saccade, demonstrating the high plasticity and variability of saccadic remapping phenomena. 

While we do not fully understand the exact role of all the hotspots in the classification image maps, what is important is that the shape of the receptive field (on the retina) across saccades establishes a transient craniotopy, at the position of the target. To maintain continuity between pre- and post-saccadic signal processing requires that these two signals be combined, by integration of signals along the saccadic trajectory—effectively annulling the displacement. In previous publications, we have argued that integration of signals across saccades could be a fundamental process for transient spatiotopy at the time of saccades (Burr & Morrone, 2010, 2011). The key to cross-saccadic integration is that receptive fields become transiently oriented in space–time, thereby creating a transient spatiotopy effectively compensating for the saccadic retinal displacement (similar strategies have been proposed for the analysis of the form of objects in motion; Burr & Ross, 1986; Burr, Ross, & Morrone, 1986; Burr & Thompson, 2011). The classification images of Figures 3–5 are the first direct demonstration for the transient spatiotemporal tilt in presumed receptive fields in human vision, thereby providing the first direct support for the existence of transiently spatiotopic mechanisms of this type. 

Supplementary Materials

Supplementary Movie - Supplementary Movie 

Movie 1 

Supplementary Movie - Supplementary Movie 

Movie 2 

Supplementary Movie - Supplementary Movie 

Movie 3 

Acknowledgments

This study was funded by the EC Project STANIB (FP7-ERC, No. 229445) and PRIN 2009. 

Commercial relationships: none. 

Corresponding author: Stefano Baldassi. 

Email: [e-mail]. 

Address: Department of Psychology, University of Florence, Via di San Salvi 12 Pad. 26, 50135 Florence, Italy.