The reference frame of the motion aftereffect is retinotopic

Abstract

Although eye-, head- and body-movements can produce large-scale translations of the visual input on the retina, perception is notable for its spatiotemporal continuity. The visual system might achieve this by the creation of a detailed map in world coordinates—a spatiotopic representation. We tested the coordinate system of the motion aftereffect by adapting observers to translational motion and then tested (1) at the same retinal and spatial location (full aftereffect condition), (2) at the same retinal location, but at a different spatial location (retinotopic condition), (3) at the same spatial, but at a different retinal location (spatiotopic condition), or (4) at a different spatial and retinal location (general transfer condition). We used large stimuli moving at high speed to maximize the likelihood of motion integration across space. In a second experiment, we added a contrast-decrement detection task to the motion stimulus to ensure attention was directed at the adapting location. Strong motion aftereffects were found when observers were tested in the full and retinotopic aftereffect conditions. We also found a smaller aftereffect at the spatiotopic location but it did not differ from that at the location that was neither spatiotopic nor retinotopic. This pattern of results did not change when attention was explicitly directed at the adapting stimulus. We conclude that motion adaptation took place at retinotopic levels of visual cortex and that no spatiotopic interaction of motion adaptation and test occurred across saccades.

Introduction

In the presence of the large-scale translations in retinal input that eye-, head- and body-movements regularly produce, visual perception is notable for its spatiotemporal continuity. It has been suggested that spatiotopic processing relies on the interplay of retinotopic coding and knowledge of changes in gaze and body position, knowledge that could be embodied as proprioceptive information and the motor command of impending saccades, referred to as corollary discharge or efference copy (Sperry, 1950; Wurtz, 2008). Indeed, neurons in parietal and frontal areas of the monkey brain show predictive coding of stimuli that will fall into the neuron's receptive field only after the saccade (Duhamel, Colby, & Goldberg, 1992) and this shift in the retinotopic locations that activate the cell around the time of a saccade (called ‘remapping’) appears to be guided by a corollary discharge signal bringing the pre-saccadic and post-saccadic positions of an object into alignment (Wurtz, 2008). Imaging studies in humans and electrophysiological studies in monkeys yielded converging evidence for remapping and showed that it occurs even in lower visual areas (Merriam, Genovese, & Colby, 2007; Nakamura & Colby, 2002). However, it remains unclear whether it is information about the outside world that is being remapped or whether increases in BOLD-response or firing rate are due to perisaccadic shifts of attention. 

Psychophysics offers a way to examine what information is remapped across saccades by investigating the reference frame of aftereffects, changes in visual responses that occur during prolonged inspection. As opposed to retinotopic aftereffects, which are fixed in a retinal frame of reference and thus move when the eye moves, spatiotopic aftereffects would reveal themselves at the same spatial location when adaptation and test are separated by a change of gaze position. There is evidence for the existence of spatiotopic form aftereffects (Melcher, 2005), indicating that some adaptation is remapped around the time of a saccade (Melcher, 2007). 

Motion aftereffects (MAE) are mediated to some extent by MT (Kohn & Movshon, 2003, 2004), an area specialized in motion analysis (Born & Bradley, 2005) and one group has claimed that a portion of the human extrastriate visual cortex encodes motion in spatiotopic (d'Avossa et al., 2007) rather than in purely retinotopic coordinates. This finding has recently been contested by Gardner, Merriam, Movshon, and Heeger (2008) who found only retinotopic representations in the human analog of MT. 

These physiological data have been paralleled by recent psychophysical studies into the reference frame of adaptation of motion processing, which leads to the MAE. A study by Wenderoth and Wiese (2008) concluded that spatiotopic motion direction aftereffects (DAE) were largely explained by spatially nonspecific DAE. However, the strength of the MAE is known to be influenced by gaze position (Mayhew, 1973; Nishida, Motoyoshi, Andersen, & Shimojo, 2003) and Ezzati, Golzar, and Afraz (2008) did claim to find a spatiotopic MAE, using random-dot stimuli with clearly defined edges. Here, we evaluate the spatiotopic MAE using a global motion stimulus and dynamic test stimuli (Mather, Verstraten, & Anstis, 1998). The use of the large global motion stimulus and the dynamic test may tap higher levels of motion coding and increase the chances of finding a spatiotopic response. 

Methods

Participants (2 authors, 5 naive as to the purpose of the experiment, age 24–31, 1 female, 5 participants in each experiment) were seated in a silent and dimly lit room with the head positioned on a chin rest, 63 cm in front of a computer screen. Stimuli were presented on a gamma-linearized 22″ Formac ProNitron 22800 screen with a spatial resolution of 1440 by 1050 pixels and a vertical refresh rate of 100 Hz. Gaze position of the right eye was measured using an EyeLink 1000 Desktop Mount (SR Research, Osgoode, Ontario, Canada) with an average spatial resolution of 0.25 to 0.5 degrees of visual angle, sampling at 1 kHz. Saccades were detected using a two-dimensional velocity space algorithm developed by Engbert and Mergenthaler (2006). The experiment was controlled by an Apple MacPro Xeon computer running custom software. Manual responses were recorded via a standard keyboard. 

Experimental conditions

The adapting stimulus was displayed at the center of the screen, and the fixation mark was projected either above or below of the adaptation stimulus. Throughout Figure 1 examples are shown with fixation below adaptation, but all observers participated in sessions with both adaptation fixation positions. Four different test conditions were used, illustrated in Figure 1A. We were interested in the magnitude of the MAE at locations that correspond either retinotopically or spatiotopically to the location of the adapting stimulus. This requires fixation to be changed to a location opposite that of the adapting stimulus. After this change of fixation position, we tested for spatiotopic MAEs at the same location on the screen. Note that due to the novel fixation location during testing, this test takes place at a retinotopic location that has not been adapted. Conversely, retinotopic MAEs were measured at the retinal location that was exposed to the adapting stimulus, and this means that we show the test stimulus at a different position on the screen. These retinotopic and spatiotopic MAEs should be viewed in relation to the magnitude of the MAE that is obtained at the location that corresponds to the adaptation location in both frames of reference (labeled Full inFigure 1A). Furthermore, an important control is constituted by the magnitude of the MAE that can be obtained at a location that has not been adapted in retinotopic nor spatiotopic reference frames (labeled Nonspecific in Figure 1A). This nonspecific MAE is tested at the same retinal location as our spatiotopic MAE test, and thus can be used to correct the spatiotopic MAE for any spreading of adaptation across the visual field in a retinotopic reference frame (Snowden & Milne, 1997; Weisstein, Maguire, & Berbaum, 1977). 

Figure 1

 

A. Experimental conditions—After adaptation with fixation above or below the adapting stimulus, four different conditions could be tested. With fixation at the same location as during adaptation, the combination of the retinotopic and spatiotopic MAEs could be measured (labeled Full), or a nonspecific MAE could be measured at a location that was neither retinotopically nor spatiotopically adapted. With fixation at the opposite side of the adaptation stimulus' location, a purely retinotopic MAE and a spatiotopic MAE could be probed separately. B. Temporal order of a trial—After adaptation to horizontal motion, observers fixated a dot placed next to the center of the screen at the same distance as the initial fixation mark. After this intermediate fixation, which served to equate the number of saccades in all conditions, they fixated the dot that would be the fixation mark during the test phase, either above or below the center of the screen. Finally, the test stimulus would be presented for 250 ms after which the observer responded as to the perceived horizontal direction of motion. C. Stimuli—Example of stimulus image, arrows indicate the size of the stimulus and its distance to fixation. (Right) The velocity of each dot in the stimulus was drawn from a 2D isotropic Gaussian distribution, of which the mean was displaced in the horizontal direction to create a net motion signal, and noise could be controlled by adjusting the standard deviation of the distribution.

Trial sequence

The sequence of phases in one trial are depicted in Figure 1B. The experiment started with EyeLink calibration routines, followed by 30 s of adaptation to motion. Trials after the first started with 3 seconds of top-up adaptation, except for every 25th trial, in which adaptation again lasted 30 s. Adaptation, which was always in the same direction during a single session, was followed by 500 ms of time to fixate a fixation mark shown at a position left or right of the center of the screen. The laterality of this fixation mark changed from trial to trial. Then, a fixation mark was shown for 750 ms at the location where the observer was to be fixating during presentation of the test stimulus, followed by the test presentation that lasted 250 ms. Thus, in every trial two saccades were made, independent of test condition or fixation location during test stimulus presentation. If a minor portion of the MAE is ‘remapped’ with each saccade, we may have sacrificed sensitivity to this remapped MAE with our procedure of two intermittent saccades, diminishing this remapped MAE. The two saccades are the smallest number that can equate eye movements across all conditions (retinotopic, spatiotopic, and non-specific) so we found it a necessary part of the procedure despite the possible sensitivity loss. If there is proportionate loss of the spatiotopic MAE with each saccade, we can recover the theoretical maximum strength of spatiotopic aftereffect following a single saccade (the necessary minimum) from its strength following two saccades by taking the square root of the percent decrease that we see with two. Furthermore, Wenderoth and Wiese showed that the effect of making two saccades on the magnitude of the DAE is very limited. The color of the fixation mark indicated the phase of the trial to the observer. The fixation mark was red during adaptation and refix/test fix phases, green during test stimulus presentation, and blue when the observers was to answer. 

Stimuli

Adaptation and test stimuli consisted of 2000 random dots, 50% black and 50% white, 3′ in size, within a Gaussian contrast envelope, a still example of which is shown in Figure 1C. The total stimulus area, 8 standard deviations in width and height, subtended 5 degrees of visual angle, resulting in a dot density of 80 dots per square degree of visual angle. The visual extent of the stimuli can be seen in Figure 1C. By employing stimuli with blurred edges, we avoid solely retinotopic effects that are known to occur for stimuli with strong motion-defined edges (Ezzati et al., 2008). Motion content of adaptation and test stimuli was varied as follows: dot velocities were drawn from a two-dimensional isotropic Gaussian distribution of which the mean and the standard deviation were varied in order to control motion signal and noise, respectively. For adaptation stimuli, the mean of the Gaussian distribution would be shifted in the leftward or rightward direction to produce a mean motion vector of 2.5 deg/s with a standard deviation of 0.3 deg/s. For test stimuli, a relatively high standard deviation of 1.25 deg/s made the test stimulus noisy (producing a dynamic test stimulus) and the aftereffect elicited by this noisy input was nulled by adjusting the mean of the Gaussian velocity distribution in the horizontal direction as dictated by a staircase procedure. Using large dynamic random dot stimuli with Gaussian edges and dynamic tests should ensure that motion adaptation occurred at higher neural levels (Mather, Pavan, Campana, & Casco, 2008), and a dynamic test stimulus of this kind (Hiris & Blake, 1992) specifically probed the amount of adaptation in a single direction instead of the direction aftereffect (Wenderoth & Wiese, 2008). Dot stimuli were implemented in a GLSL shader program and were drawn using OpenGL vertex array extensions. The fixation stimulus consisted of a black circle, 14′ across, within which a 6′ colored disk was drawn. The color of the central disk indicated the phase of the trial to the observer as described above. In a second experiment, we increased the size of the stimuli to 6.7 degrees of visual angle, with a dot density of approximately 200 dots per square degree of visual angle. The speed of the adapting stimulus was increased to 33 degrees per second, and the test stimulus noise standard deviation was increased to 8.3 degrees per second. As attention is thought to play a large role in motion aftereffects (Culham, Verstraten, Ashida, & Cavanagh, 2000), we also added a secondary task intended to ensure that attention was directed at the adapting stimulus. In 50% of trials, the adapting stimulus would undergo a Gaussian-enveloped contrast decrement (contrast: 25, 50 or 75%, standard deviation: 100 ms), which observers had to detect and report by pressing the space bar. All other parameters of the experiment were identical to the first. 

Data acquisition

Observers participated in 4 sessions, factoring the two fixation locations during adaptation and the two horizontal motion directions of adaptation in a 2 × 2 design. In each session, eight randomly interleaved one-up-one-down staircases with decreasing step size were run, two for each of the experimental test stimulus conditions. Of these two staircases, one had a positive and the other a negative starting value. Staircases were terminated after eight reversals, and the last four reversal values were averaged to deliver the PSE (point of subjective equality) estimate of that staircase. In total, each of the four experimental conditions was probed using eight separate staircase runs. In the second experiment each observer participated in two sessions, in which the two different motion directions were adapted. Fixation position during adaptation (above or below the adapting stimulus) was counterbalanced across subjects. For both experiments, there was ample rest in between subsequent sessions to allow for residual adaptation of earlier sessions to wear off. 

Results

If an observer were to look to the opposite side of the screen from the fixation mark during the test, it would be possible to mistake Retinotopic with Spatiotopic conditions, and Full with Nonspecific. Gaze recordings show this never happened. Brief glimpses at the test stimulus could not have altered the ongoing peripheral topped-up adaptation systematically, nor are saccade errors likely to have had a strong impact in our experiments. Nevertheless, we investigated their occurrence. Recordings show that observers were able to make the two required saccades in >98% of the trials. Recordings also showed that during test presentation gaze position was only directed at the test stimulus rather than at the fixation dot in <0.5% of the trials (defined as trials containing any sample within a circular area, 3 degrees wide, centered on the test stimulus center). Furthermore, there was no correlation between the conditions tested and the frequency of misfixations. We conclude that observers' fixation strategies did not affect our results. In the second experiment, observers detected the contrast-decrement in the secondary task within 1.5 s of its occurrence with 90% accuracy. Percentage correct did not differ significantly between contrast conditions, repeated measures ANOVA F(2,8) = 1.14, p = 0.37. 

Individual psychophysical data for each of the five subjects are shown in Figure 2. First, the four experimental conditions (Retinotopic, Spatiotopic, Full, and Nonspecific MAEs) are plotted as gray bars in Figure 2A. Values represent the shift of the mean of the test stimulus velocity distribution needed to reach the PSE for horizontal motion direction, i.e., necessary to null the MAE. We ran a repeated measures ANOVA with the relative position of adaptation and test stimulus in spatiotopic (same vs. different) and retinotopic (same vs. different) coordinates as independent variables (Full: same, same; Spatiotopic: same, different; Retino: different, same; Non-specific: different, different). We found a strong main effect of retinotopic alignment of test and adaptor; F(1,4) = 228.5, p = 0.0001. However, there was neither a significant effect of spatioptopic alignment ( F(1,4) = 1.210, p = 0.33), nor an interaction of the two factors ( F(1,4) = 0.15, p = 0.72), both of which could have revealed the presence of an influence of the spatiotopic location on the MAE in this experiment. 

Figure 2

 

A. Experiment 1—MAE magnitude, defined as the mean horizontal velocity needed to null the MAE, for five individual subjects. Four experimental conditions were tested; bars depict data from these conditions. In all subjects, the Retinotopic and Full conditions are very alike, whereas the Spatiotopic condition shows a much smaller MAE. The Nonspecific MAE is very similar to the spatiotopic MAE for all observers, indicating that the spatiotopic MAE may be due to retinotopic spread of adaptation. We thus subtract the Nonspecific MAE from all other MAEs, and average across subjects (inset). The Retinotopic and Full MAEs are indistinguishable, whereas there is no remaining spatiotopic MAE. B. Experiment 2—Bars represent data in the same conditions in Experiment 2, format identical to figure A. Also with larger stimuli, faster motion and secondary attention task, no spatiotopic MAE is present. Error bars are 95% CIs.

These statistical results are easily confirmed visually, as Full MAE and retinotopic MAE are highly similar for all subjects. Also, there is no residual spatiotopic MAE when the Nonspecific MAE is subtracted from all other MAEs. Our data indicate the motion adaptation occurs in a retinotopic frame of reference, and that adaptation in a spatiotopic frame of reference plays no role in the generation of the MAEs we probed. Despite the higher speeds, greater stimuli and guided attention we find a similar pattern of results in our second experiment ( Figure 2B, red bars). As in the first experiment, the repeated measures ANOVA revealed a highly significant retinotopic MAE; F(1,4) = 320.25, p = 0.006. And as before, there was no evidence for a spatiotopic MAE, neither in a potential main effect of spatiotopic alignment ( F(1,4) = 2.14, p = 0.22), nor in the interaction of spatiotopic and retinotopic alignment ( F(1,4) = 0.002, p = 0.97). 

Discussion

We have conducted two experiments to determine the reference frame in which the MAE takes place. When compared to the Full MAE (both retinotopic and spatiotopic), we find equally large retinotopic MAEs, small magnitude spatiotopic MAE and equally small spatially general MAEs. Importantly, any spatiotopic MAEs that we find can be fully explained by the spatially general MAE that is due to a spreading of motion adaptation, dropping off from the adapted location in distance across the retinotopic visual field. If any fraction of the Full MAE is remapped after a saccade, it does not survive the 2 saccades necessary to equate the eye movements across conditions. 

Our large, global motion stimuli and dynamic tests were chosen to favor higher level motion detectors that might be more likely to exhibit spatiotopic properties. Yet, our MAE results are even more strictly retinotopic than those obtained for the direction aftereffect (Wenderoth & Wiese, 2008). Ezzati et al. (2008) do report an MAE, following a saccade, at a location consistent with a spatiotopic aftereffect. However, is important to note that the magnitude of their MAE at the spatiotopic location is comparable to the magnitude of the MAE we measured in non-retinotopic tests (both Spatiotopic and Nonspecific). Unfortunately, these authors had no measure of the non-specific spread of the MAE to a non-spatiotopic, non-retinotopic location comparable to the location where they tested the spatiotopic MAE. Nevertheless, if non-specific results in their procedure were like ours (and our results do match across our two procedures for the strengths of the MAE at the spatiotopic location) we can assume they would find a non-specific effect of approximately the same strength as the spatiotopic and so no net advantage for the spatiotopic location. 

Another difference between recent literature and our results can be found in the fact that we do not find a gaze-dependent modulation of the MAE, whereas Nishida et al. (2003) did. We believe this difference to be due to two factors: (1) the size of the difference in gaze direction between adaptation and test, and; (2) the type of stimulus used. The gaze direction difference needed to fixate the two separate fixation marks for adaptation and test stimulus presentation in the present study was 7.2 degrees. In the Nishida et al. study, this difference was 62.2 degrees, a difference between studies of an order of magnitude. With such a large gaze eccentricity, ocular muscles provide salient proprioceptive information, and it is more easily envisioned that the gaze difference plays a role. The stimulus used in the previous study (Nishida et al., 2003) was one in which two Gabor patterns (one above, one beneath fixation) drifted in opposite directions, causing adaptation to a motion difference, and the duration of the resulting MAE was tested. In our study, we used random dot patterns resulting in a global motion stimulus eliciting more motion integration compared to local-motion Gabor patterns, and we tested the MAE using dynamic test presentations instead of duration measurements. This may have caused more adaptation outside the area of original stimulation. Although in both studies the stimuli were of comparable size, the modest difference in gaze direction between, and higher level of motion integration in our stimuli is likely to cause the difference in gaze modulation of the MAE. 

A recent series of functional imaging papers constitute a controversy regarding the frame of reference of the human MT complex, with d'Avossa et al. (2007) finding a spatiotopic organization principle, whereas Gardner et al. (2008) subsequently provided evidence for a solely retinotopic organization of MT. If we assume that the adaptable units that contribute to the MAE we measure are also involved in the activations found with functional imaging, our results strongly support the recent findings of Gardner et al. and run counter to the earlier results of d'Avossa et al. 

To conclude, we have found no MAE in a spatiotopic frame of reference, and this result bears on current models of trans-saccadic perception. It has been argued that the occurrence of spatiotopic aftereffects points to the existence of feature remapping (Melcher, 2005, 2008; Melcher & Colby, 2008), in which detailed information about the features of an item in the visual field is transferred from pre-saccadic to post-saccadic retinotopic location of the item. Adaptation to a specific stimulus feature would be repositioned retinotopically around the time of the saccade to appear at the same spatiotopic location after the saccade. Despite this theoretical possibility, our results show that for motion adaptation no such process acts to stabilize perception across saccades, and clearly speak against this type of feature remapping. A more abstract remapping, in which not stimulus features but only attentional pointers to items are remapped perisaccadically is more likely given our results (Wurtz, 2008). 

Acknowledgments

This research was supported by a Chaire d'Excellence grant to PC. We thank Jeroen van Boxtel for his comments on an earlier version of the manuscript. 

Commercial relationships: none. 

Corresponding author: Tomas Knapen. 

Email: [e-mail]. 

Address: 45, Rue des sts‐Pères, 75006, Paris, France.