Motion and tilt aftereffects occur largely in retinal, not in object, coordinates in the Ternus–Pikler display

Abstract

Recent studies have shown that a variety of aftereffects occurs in a non-retinotopic frame of reference. These findings have been taken as strong evidence that remapping of visual information occurs in a hierarchic manner in the human cortex with an increasing magnitude from early to higher levels. Other studies, however, failed to find non-retinotopic aftereffects. These experiments all relied on paradigms involving eye movements. Recently, we have developed a new paradigm, based on the Ternus–Pikler display, which tests retinotopic vs. non-retinotopic processing without the involvement of eye movements. Using this paradigm, we found strong evidence that attention, form, and motion processing can occur in a non-retinotopic frame of reference. Here, we show that motion and tilt aftereffects are largely retinotopic.

Introduction

The early visual system is organized retinotopically (e.g., Tootell, Hadjikhani, Mendola, Marrett, & Dale, 1998). When we move our body, head, or eyes, the retinal images of objects undergo dramatic changes. Remarkably, we are not aware of these changes. We perceive the world stable necessitating the use of non-retinotopic representations. Hence, a fundamental question in vision science is to determine which processes are carried out retinotopically and which processes are carried out non-retinotopically. 

Non-retinotopic processing is mainly investigated with the saccadic stimulus presentation paradigm (SSPP). In SSPP, observers make a saccadic eye movement from one fixation point to a second fixation point. Two stimuli, one before and one after the saccade, are presented briefly at the same position on the screen. Interestingly, visual processing at the retinal location where the first stimulus is presented influences the processing at the location where the second stimulus will be presented after the saccade (Irwin, 1996; McRae, Butler, & Popiel, 1987). These findings are in good agreement with neurophysiological findings in which neurons start responding to a stimulus even before the saccade, which will bring the stimulus into the receptive fields of these neurons (Duhamel, Colby, & Goldberg, 1992; Nakamura & Colby, 2002; Umeno & Goldberg, 1997). Using SSPP, it was shown that attention (Golomb, Chun, & Mazer, 2008; Mathot & Theeuwes, 2010), color (Wittenberg, Bremmer, & Wachtler, 2008), motion (Melcher & Morrone, 2003; Ong, Hooshvar, Zhang, & Bisley, 2009), position (Prime, Niemeier, & Crawford, 2006) and time processing (Burr, Tozzi, & Morrone, 2007) can occur in a non-retinotopic frame of reference. Aftereffects are often used to selectively target a mechanism in the visual processing hierarchy. Based on this rationale, Melcher (2005) chose contrast, tilt, form, and face aftereffects to probe visual processing at gradually increasing levels of visual hierarchy. Results from this and related studies (Ezzati, Golzar, & Afraz, 2008; Melcher, 2005, 2007, 2008) showed evidence for non-retinotopic aftereffects. 

These findings, however, have been challenged. Using the same paradigm as Melcher (2005, 2007), both motion (Knapen, Rolfs, & Cavanagh, 2009) and tilt aftereffects (Knapen, Rolfs, Wexler, & Cavanagh, 2010) were found to be based on retinotopic coordinates. Moreover, using a gaze direction modulation paradigm, Wenderoth and Wiese (2008) failed to show evidence that the motion direction aftereffect is non-retinotopic. 

SSPP and gaze modulation paradigms are based on eye movements. Here, we used a recently developed paradigm, based on the Ternus–Pikler display, which tests retinotopic vs. non-retinotopic processing without the involvement of eye movements. In the first frame, three squares are presented followed by an inter-stimulus interval (ISI). In the second frame, the same three squares are presented, shifted rightward by one location. After the second frame, another ISI is presented and the sequence starts over (Figure 1). With ISIs longer than 40 ms, the three disks are perceived to move back and forth in “group motion” (Pantle & Petersik, 1980). 

Figure 1

 

Motion adaptation in the Ternus–Pikler display. Schematic of the stimuli used in Experiments 1A and 1B comprising an adapting sequence and a testing frame. The solid black arrows between the squares indicate the non-retinotopic motion correspondences. The colored arrows depict the direction of drift for each Gabor patch (arrows were not present in the actual stimulus display). (A) Incoherent motion condition: Continuous upward motion is perceived in the central square even though retinotopically and spatiotopically “incoherent” up-and-down motion is presented framewise (Supplementary Video 1). (B) Coherent motion condition: Gabors are perceived to drift up and down alternately from frame to frame in the central square (Supplementary Video 2). Coherent motion is presented at each frame retinotopically and spatiotopically.

Retinotopically, the central square in the first frame occurs at the same position as the leftmost square in the second frame (Figure 1). Non-retinotopically, the central square in the first frame is matched with the central square in the second frame (solid arrows in Figure 1), even though they are presented at two different locations. This correspondence is repeated as the stimulus cycles. By placing a stimulus in each of the squares, we can test if a given mechanism operates in retinotopic or non-retinotopic coordinates. Using this paradigm, we found attention and processing of form and motion to be carried out in a non-retinotopic frame of reference (Boi, Ogmen, Krummenacher, Otto, & Herzog, 2009; Ogmen, Otto, & Herzog, 2006). On the other hand, we found evidence that the motion aftereffect occurs in a retinotopic frame of reference (Boi et al., 2009). 

Here, we extend previous results on the motion aftereffect and also show that the tilt aftereffect is based on a retinotopic frame of reference. 

General materials and methods

Observers viewed the stimuli binocularly from a distance of 66 cm on a PHILIPS 201B4 CRT monitor driven by a standard accelerated graphics card. Screen resolution was set to 1280 by 1024 pixels at 75-Hz refresh rate. 

To monitor eye movements, an iViewX-HiSpeed eye tracker from SensoMotoric Instruments (SMI) was used, set up for binocular mode at 500-Hz sampling frequency. Signals of both eyes were averaged. 

A total of eleven observers took part in two experiments. Observers had normal or corrected-to-normal vision at least for one eye, as assessed with the Freiburg Visual Acuity Test (Bach, 1996). All but one observer were naive to the purpose of the study. Naive observers were paid CHF 20/h. Observers were explained the general aim of the experiment and signed informed consent. All experiments were approved by the local ethics commission. Observers were told they could quit the experiment at any time. 

Experiment 1A

Methods

Stimuli

Three gray squares moved back and forth on a black background. An adapting sequence preceded a testing frame. 

The adapting sequence comprised ten frames presented for 306.7 ms each interleaved by an Inter-Stimulus Interval (ISI) of 200 ms, adding up to a total duration of 4867 ms. In each frame and square, an 80% Michelson contrast Gabor (sinusoidal luminance modulation constrained by a Gaussian window) drifted smoothly either upward or downward at a speed of 1.41 arcdeg/s (Figure 1). 

The last frame of the stimulus was the testing frame, which consisted of three squares. This configuration was also presented for 306.7 ms. The testing frame was separated from the adapting sequence by a 200 ms ISI. One 40% Michelson contrast Gabor was presented in the central square only. To measure the magnitude of the MAE, the Gabor drifted either upward or downward. The velocity at which the drift nulls the motion aftereffect, estimated by the method of constant stimuli, was used as an estimate of the magnitude of the MAE. The Gabor drifted smoothly either upward or downward for 306.7 ms with one of five different velocities (−15.62, −7.81, 0, 7.81, 15.62 arcmin/s). These conditions are very similar to the ones used in a recent study (Boi et al., 2009; the spatiotemporal layout of the testing stimulus differed slightly and we recorded eye movements here). 

Square and background luminance values were 40 cd/m² and 4 cd/m², respectively. The central square was 2.78 arcdeg side, and lateral squares were 3.57 arcdeg side. The squares were 4 arcdeg apart from each other (center-to-center distance). We used different square sizes to further enhance the group motion percept (Pantle & Petersik, 1980; Wallace & Scott-Samuel, 2007). Gabors had a spatial frequency of 1.6 cycles/arcdeg and a σ of 0.5 arcdeg and were centered at the center of the squares. 

Procedure

Five paid observers and one of the authors (MB) took part in this experiment. 

Each trial began with the presentation of a 2.9-arcmin fixation dot. The dot stayed on for a minimum of 1 s. The subject had to keep fixation for 500 ms within the borders of a 1-arcdeg side fixation window. In case of failure to fixate, as determined by eye monitoring, a trial was not started. 

Subjects were required to keep fixation at the center of the screen for the entire duration of the stimulus. Subjects were instructed to attend to the central square and to report the perceived drift direction of the Gabor in the testing frame (upward or downward) by pressing one of two buttons. 

There are three different coordinate systems that have to be distinguished. First, the stimulus can be described within a spatiotopic frame of reference, i.e., with respect to the screen. Second, the stimulus can be described with respect to a retinotopic frame of reference. Third, because of the group motion percept, the stimulus can be described with respect to this grouping-based frame of reference. For example in Figure 1A, continuous upward motion is perceived in the central square moving back and forth even though motion direction changed from frame to frame in a spatiotopic and retinotopic frame of reference. 

We employed two different types of adapting sequences (coherent and incoherent). In the incoherent adapting sequence (Figures 1A and 2A, Supplementary Video 1), the drift direction of the Gabors changed from frame to frame both in spatiotopic and retinotopic coordinates. This arrangement produced a compelling non-retinotopic percept of continuous upward (or downward) drift of the Gabor in the central square. 

Figure 2

 

Experiments 1A and 1B. (A) Schematic of the stimuli (same as in Figure 1). (B) Fixation condition, group average. Magnitudes of the MAE for the incoherent and the coherent conditions. The strongest MAE occurs in the coherent condition. (C) Fixation condition, individual MAE magnitudes. (D) Tracking condition, group average. Subjects tracked the stimulus with their eyes instead of keeping steady fixation. Magnitudes of MAE for the incoherent tracking (Incoherent T) and coherent tracking (Coherent T) conditions. Only in the incoherent tracking condition, MAE differs significantly from zero. (E) Individual MAE magnitudes. (F) Presentation maps for the incoherent and the coherent conditions for one observer while fixating. Colors indicates the proportion of coherent motion presented at each location (upward minus downward motion duration divided by the total stimulus duration). Red and blue areas indicate upward and downward motions, respectively (each contour represents a 10% change). White circles indicate where the test Gabor was presented. Upward and downward motions cancel each other out at the central locations in the incoherent condition and form two well-defined regions in the coherent condition. The testing Gabors fall on two well separated regions. Tested regions are exposed to coherent retinotopic motion in the coherent condition and little coherent motion in the incoherent condition. (G) Presentation maps for the incoherent and coherent conditions for one observer while tracking the stimulus. The maps show that the subject tracks the stimulus accurately. In the incoherent tracking condition, the motion of the central Gabor fell onto one single confined region. In the coherent tracking condition, the alternating upward and downward motions of the Gabors cancelled each other out. Consistent motion is presented retinotopically in the incoherent tracking condition whereas the coherent tracking condition shows that very little consistent motion was presented retinotopically. The testing Gabor always fell on the central part of the retina.

In the coherent adapting sequence (Figures 1B and 2A, Supplementary Video 2), the motion of the Gabors was always coherent both in spatiotopic and retinotopic coordinates. Perceptually, the drift direction of the central Gabor reversed at each frame. 

Each of the two adapting sequences (coherent and incoherent) was presented in a separate block. For each sequence, a total of 180 trials was presented (90 trials upward drift and 90 trials downward drift direction), in six blocks of 30 trials each. In each trial, the initial motion direction (leftward or rightward) of the display was randomized. Consequently, the central square in the last frame could fall at two different locations, either at the left or at the right of fixation (Figure 1). Each of the two testing locations was tested an equal number of times and the results were combined together. 

Subjects' responses plotted as a function of Gabor drift velocity were fitted with a cumulative Gaussian. The parameters of the Gaussian were estimated by means of a maximum likelihood procedure and the inflection point of the best fitting function was chosen as an estimate of the MAE. 

No flanker condition

We presented only the two central squares, i.e., we omitted the outermost squares (the other parameters of the stimulus were exactly the same; Figure 3A). As no group motion is perceived in this condition, the retinotopically consistent motion is clearly visible. Three observers were tested in this condition. 

Figure 3

 

No flanker condition. (A) Schematic of the stimulus. Only the two central squares were presented. The Gabors drifted consistently in opposite directions in the two squares. The parameters of the stimulus were as in the main experiment. (B) Magnitude of the MAE in this condition (No Flankers) is about the same as the magnitude of the MAE in the coherent condition in the first experiment (Coherent, data from three subjects from Experiment 1A).

Eye movements (presentation maps)

We created a set of presentation maps representing where the stimulus was presented on the retina. To produce these maps, we reconstructed, for every trial, the position occupied by each Gabor of the display on the retina at every instant, taking into account eye movements and stimulus motion. The display position was sampled with a 50-Hz frequency and a 0.1-deg spatial resolution. We used a simplified representation of the stimulus, consisting of three disks representing the three Gabors. The diameter of the three disks was equal to the actual stimulus square's side. 

We then registered where, at every moment, upward and downward motions were presented on the retina. For each location, we subtracted the total time a downward moving Gabor was presented from the time an upward moving one was presented in each condition. In this way, we obtained a map displaying the net time each motion direction was presented at each location. The locations of the testing stimuli were then superimposed on these maps (the white circles represent the center of the test Gabors). Presentation maps indicate how much retinotopic adaptation is expected at each tested location. 

Results and discussion

We reproduced previous results showing that MAE occurs largely in a retinotopic frame of reference (Boi et al., 2009). Whereas both conditions differed significantly from zero (one-sample t-test, incoherent: p = 0.048; coherent: p = 0.0001; Figure 2), the MAE measured in the coherent condition was by far greater than in the incoherent condition (paired t-test, p = 0.0023). The aftereffect in the coherent condition was essentially the same as the one in the no flanker condition (paired t-test, p = 0.108; Figure 3B). This is particularly interesting, because the retinotopically and spatiotopically coherent motion was not perceived as continuous because of the non-retinotopic (object-based) frame of reference, whereas in the no flanker condition, the retinotopically and spatiotopically coherent motion was clearly visible. Hence, the percept of a continuous motion seems not to be important for low-level adaptation (see also Culham et al., 1998; Maruya, Watanabe, & Watanabe, 2008). 

The presentation maps for the incoherent and the coherent conditions are presented in Figure 2F for one out of the six subjects. Color indicates the duration of upward motion minus the duration of downward motion for each location. Values of +1.0 (red) indicate that only upward motion was presented for the entire duration of the experimental condition at that particular location. Values of −1.0 (blue) indicate that only downward motion was presented at that location for the entire duration of the experimental condition. The maps of additional observers are available in the Supplementary material. Maps show that subjects were able to keep a steady central fixation during stimulus presentation, and thus, there was no interference originating from eye movements. 

Experiment 1B

Experiment 1A showed that the MAE does not occur in a non-retinotopic object-centered frame of reference. Still, as the motion of the Gabors is identical in spatiotopic and retinotopic coordinates, Experiment 1A leaves open the possibility that the MAE occurs in spatiotopic coordinates (Ezzati et al., 2008; Melcher, 2005, 2007). We addressed this question by asking observers to track the central square. 

If observers track the central square, all three squares will be static with respect to the retinotopic frame of reference but will move back and forth in spatiotopic coordinates. In this way, the incoherent condition (Figure 1A) is turned into a condition where motion is consistent in retinotopic coordinates but inconsistent in spatiotopic coordinates. On the other hand, when tracking, the coherent condition of Figure 1B produces inconsistent motion in retinotopic coordinates but a consistent motion in spatiotopic coordinates. 

Methods

Stimuli and procedure

Four out of six observers from the previous experiment (three naive plus one author, MB) participated in this experiment. The method and stimuli were the same as in Experiment 1A, except that now we instructed subjects to actively track the motion of the central square for the entire duration of the stimulus presentation. In this way, we expected the three squares to fall roughly on the same position on the retina despite the motion of the stimulus. 

Results and discussion

Only the incoherent tracking condition showed a significant MAE (one-sample t-test; incoherent tracking, p = 0.0075; coherent tracking, p = 0.4618; Figure 2D). The comparison between the incoherent and the coherent tracking conditions showed a clear trend (paired t-test p = 0.054). As in Experiment 1A, we estimated how long each motion direction was presented at each retinal location. Subjects tracked the central square accurately (Figure 2G). As mentioned earlier, when observers track the central square, the spatiotopically incoherent motion of Figure 1A is consistent in retinotopic coordinates. This can be seen in the left panel of Figure 2G showing that the motion presented to the retina was consistent. On the other hand, the spatiotopically coherent condition of Figure 1B becomes incoherent in retinotopic coordinates. 

We found a significant aftereffect only in the incoherent tracking condition (Figure 2D) eliminating the possibility of spatiotopic motion aftereffect and supporting a retinotopic basis for the MAE. 

Experiment 2

In the previous experiment, MAE was found to be largely retinotopic. Here, we tested whether the tilt aftereffect (TAE) occurs in retinotopic or non-retinotopic coordinates. 

Stimuli

The display consisted of three gray disks centered at fixation with a Gabor superimposed on the central disk. Disks of 40 cd/m² of luminance were displayed on a uniform 4 cd/m² black background. The disks had a diameter of 3.5 arcdeg and were 4 deg apart from each other's center. Gabors had a σ of 0.5 arcdeg, spatial frequency of 1.6 cycles/arcdeg, and Michelson contrast of 80%. The Gabor in the first frame was tilted by 20 deg either clockwise or counterclockwise in different blocks. The Gabor's phase was smoothly reversed with a frequency of 0.25 Hz to avoid luminance adaptation. 

Each trial began with a fixation dot (2.9 arcmin) displayed for a minimum of 1 s. Subjects had to maintain fixation for at least 300 ms within a 1 deg × 1 deg square area around the fixation dot for the trial to start. The first frame was then presented for a duration randomly varying between 4506.7 ms and 5013 ms. This variability was introduced to avoid predictive eye movements to the target. The tilt of the Gabor in the testing frame was chosen randomly to be −7, −3, −1.5, 0, 1.5, 3, or 7 deg. Two subjects were tested using a wider interval in the non-retinotopic static condition as their threshold exceeded the limits of the range tested. 

Procedure

Five new paid observers plus one of the authors (MB) took part in the experiment. 

The display consisted of two frames. The first frame displayed three disks and the central Gabor. After an ISI of 146.7 ms, the disks reappeared in a second frame of 146.7 ms, either shifted one position laterally (moving condition,Supplementary Video 3) or at the same position (static condition,Supplementary Video 4) in different blocks. The direction of the display motion was varied across blocks. In the second frame, a Gabor was presented at different locations. In the moving condition, the Gabor was presented either at the same screen position as the adapter (retinotopic test) or depending on the direction of motion, one position to the left or right of the adapter (non-retinotopic test; Figure 4). In the static condition, the Gabor occurred either at the same screen position (retinotopic test) or at a shifted position (non-retinotopic test). Subjects were instructed to maintain steady fixation throughout stimulus presentation and to indicate the tilt of the Gabor in the second frame, whether clockwise or counterclockwise. For each of the four conditions, a total of 112 trials were completed in two blocks of 56 trials each. 

Figure 4

 

Tilt adaptation. The four conditions in the tilt adaptation experiment. In the non-retinotopic conditions (left panels), adapting and test Gabors are presented at two non-overlapping locations. In the retinotopic conditions (right panels), the adaptor and test Gabors are presented at the same retinotopic location. Top and bottom rows correspond to moving (Supplementary Video 3) and stationary (Supplementary Video 4) Ternus–Pikler displays, respectively.

The responses of the subjects were plotted as a function of degrees of tilt in the test stimulus and fitted by a cumulative Gaussian, whose parameters were estimated through a maximum likelihood procedure. The 50% point of the best fitting curve was taken as an estimate of the TAE magnitude. 

Results and discussion

In the retinotopic moving condition, there is a substantial tilt aftereffect of 4.75 deg, which is clearly larger than the corresponding non-retinotopic effect (1.75 deg). As illustrated by the arrows in Figure 4, the non-retinotopic frame of reference, induced by the Ternus–Pikler display, depends on the perception of group motion between its elements. Performance in the static condition showed a very similar pattern. The static conditions can be used as a baseline to account for non-specific TAE. Hence, we subtracted the magnitude of the aftereffect of the static conditions from the moving conditions (Figure 5C). This difference was significant for the non-retinotopic conditions (one sample t-test, p = 0.02) but not for the retinotopic conditions (one sample t-test, p = 0.85). This shows that, when adapting and testing stimuli are presented at the same retinotopic location, a strong TAE is obtained, independent of the non-retinotopic frame of reference that is established by the Ternus–Pikler display. The magnitude of the non-retinotopic aftereffect was small, being only 11% of the aftereffect measured in the static retinotopic condition. 

Figure 5

 

Experiment 2. (A) Averaged TAE magnitudes for the non-retinotopic moving (NR mov), retinotopic moving (R mov), non-retinotopic static (NR stat), and retinotopic static (R stat) conditions. (B) Individual TAE magnitudes. (C) Difference between the non-retinotopic conditions (NR, non-retinotopic motion minus non-retinotopic static) and between the retinotopic ones (R, retinotopic motion minus retinotopic static; note change of scale). Only the NR condition differed significantly from zero. (D) Maps of stimulus presentation for the (top) non-retinotopic and (bottom) retinotopic moving conditions. Colored areas indicate the location and duration of the presentation of the adapting Gabor (notice that in this plot color stands for duration and not motion direction as in Experiments 1A and 1B). Fixation was consistently maintained. In the non-retinotopic condition, the testing Gabors fell far off the region presented with the adaptor and right onto it in the retinotopic condition.

As in the previous experiments, we analyzed eye movements to reconstruct where the adapting stimulus fell on the retina. We calculated how long the central adapting stimulus was presented at each location on the retina, similarly to Experiments 1A and 1B. In the plots, values of +1.0 (red) represent areas where the adapting frame had been presented for the entire adaptation; values of 0 (green) represent locations of the retina where the stimulus was never presented. As shown in Figure 5D, observers were able to maintain steady fixation during the stimulus presentation. The maps of additional observers are available in the Supplementary material. 

Discussion

Despite the movements of our eyes and body, we have a compelling percept of a stable and constant world. Because these movements cause drastic shifts of the retinal image, the visual system needs to build non-retinotopic representations wherein a stable percept is produced. Significant research has been devoted to the understanding of how non-retinotopic representations are built and which processes make use of these non-retinotopic representations. Because saccadic eye movements generate rapid and drastic shifts of the retinal image, they have been used to study whether and how information is integrated non-retinotopically (e.g., Burr et al., 2007; Ezzati et al., 2008; Golomb et al., 2008; Gordon, Vollmer, & Frankl, 2008; McKyton, Pertzov, & Zohary, 2009; Melcher, 2005; Melcher & Morrone, 2003; Ong et al., 2009; Prime et al., 2006; Ross & Ma-Wyatt, 2004; Wittenberg et al., 2008). Theories of trans-saccadic integration fall on a broad spectrum, ranging from full (Jonides, Irwin, & Yantis, 1982) to very limited integration (Cavanagh, Hunt, Afraz, & Rolfs, 2010; Irwin, 1996). Given that the full-integration theory has been refuted by a large number of studies, more recent studies focused on which information is integrated across saccades and where in the visual system this integration takes place. Using this logic, Melcher (2005, 2007, 2008) chose contrast, tilt, form, and face aftereffects to probe increasing levels of the visual hierarchy. While the contrast aftereffect was found to be retinotopic, the other aftereffects were found to be non-retinotopic. In agreement with these findings, Ezzati et al. (2008) reported non-retinotopic motion aftereffects. 

This view of trans-saccadic integration is also supported by remapping of cortical neurons in the frontal eye field (Umeno & Goldberg, 1997), lateral intraparietal area (Duhamel et al., 1992), and extrastriate cortex (Nakamura & Colby, 2002). This remapping might be triggered by a so-called “corollary discharge”, which is a copy of the saccade command (Sommer & Wurtz, 2002, 2006). In support of their theory, Melcher and Colby (2008) highlighted the similarities between increasing levels of remapping and the increasing magnitude of non-retinotopic aftereffects according to the cortical hierarchy. 

However, at odds with these results, Knapen et al. showed that motion (Knapen et al., 2009) and tilt aftereffects (Knapen et al., 2010) are produced in a retinotopic frame of reference. Similarly, Afraz and Cavanagh (2009) provided evidence for retinotopic processing of the gender face aftereffect. Finally, further evidence against the non-retinotopic nature of aftereffects comes from the direction aftereffect in a gaze modulation paradigm (Wenderoth & Wiese, 2008). Thus, evidence for non-retinotopic integration within the context of saccadic eye movements is equivocal and further research is needed to reconcile the aforementioned conflicting findings. 

The need for non-retinotopic processing goes beyond eye movements. Moving objects stimulate neurons in retinotopic areas only briefly not allowing sufficient time for in-depth processing. ¹ As a result, in the absence of non-retinotopic mechanisms, moving objects will appear extensively blurred, a problem known as “moving ghosts problem” (Ogmen, 2007; Ogmen & Herzog, 2010). Unlike trans-saccadic integration, this type of non-retinotopic processing cannot be explained in terms of a remapping induced by a corollary discharge. 

To study non-retinotopic processing in the absence of eye movements, we have recently developed a simple paradigm, based on the Ternus–Pikler display, showing that form and motion processing as well as conjunction search are accomplished in non-retinotopic coordinates (Boi et al., 2009; Ogmen et al., 2006; for comparable paradigms, see Cavanagh, Holcombe, & Chou, 2008; Nishida, 2004; Nishida, Watanabe, Kuriki, & Tokimoto, 2007; Shimozaki, Eckstein, & Thomas, 1999; Watanabe & Nishida, 2007). In this study, we applied this paradigm to visual aftereffects. 

While we found small but significant non-retinotopic motion (Experiment 1A) and tilt (Experiment 2) aftereffects, the magnitudes of these effects were rather small compared to the ones found by Melcher (2005, 2007) using SSPP. In Melcher's studies, the non-retinotopic tilt aftereffect was approximately 60% of the retinotopic aftereffect, whereas, in our paradigm, the non-retinotopic aftereffect was only 11% of the aftereffect measured in the retinotopic condition. In Experiment 1A, the motion aftereffect in the non-retinotopic condition was about 24% of the aftereffect in the retinotopic one. Still, in this experiment, the non-retinotopic adaptation might partially or completely originate from non-specific adaptation (Weisstein, Maguire, & Berbaum, 1977). In support of this point, we did not observe any spatiotopic aftereffect when the subject moved his eyes to follow the central square (Experiment 1B). 

Nishida, Motoyoshi, Andersen, and Shimojo (2003) found that the motion, tilt, and size aftereffects are modulated by gaze direction by 10% of their magnitude. Nonetheless, other studies using SSPP did not find any significant non-retinotopic component in either the tilt (Knapen et al., 2010) or the motion aftereffect (Knapen et al., 2009) suggesting that further research is needed to reconcile these discrepant findings. 

Melcher used the SSPP, which tests for spatiotopic aftereffects. We used a Ternus–Pikler display that does not involve eye movements. Hence, we cannot disentangle retinotopic and spatiotopic aftereffects. For this reason, we repeated Experiment 1A and asked observers to track the central square, finding absolutely no evidence for spatiotopic adaptation. 

In a recent paper, Cavanagh et al. (2010) proposed that visual stability is based on a remapping of attentional pointers. The authors argued that processing of low-level visual features occurs retinotopically and that the updating of attentional pointers across saccades allows attention to be efficiently and quickly repositioned across saccades. Our results support this proposal showing that mechanisms such as motion and tilt adaptation operate in retinotopic coordinates. In addition, we propose that non-attentive, spatiotemporal grouping guides non-retinotopic processing along motion trajectories. Integration of information across grouped locations compensates for the short time the object is projected at every single location on the retina. 

Supplementary Materials

Supplementary PDF - Supplementary PDF 

Supplementary Movie - Supplementary Movie 

Supplementary Movie - Supplementary Movie 

Supplementary Movie - Supplementary Movie 

Supplementary Movie - Supplementary Movie 

Acknowledgments

This work was supported by the Swiss National Science Foundation (SNF) Project “Dynamics of Feature Integration” and in part by Award R01 EY018165 from the National Institutes of Health (NIH). The content is solely the responsibility of the authors and does not necessarily represent the official views of NIH. 

Commercial relationships: none. 

Corresponding author: Marco Boi. 

Email: [e-mail]. 

Address: Laboratory of Psychophysics, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne (EPFL), Station 19, Lausanne, Vaud, CH‐1015, Switzerland. 

Footnote

Footnotes

1   ¹It may be argued that spatiotemporally oriented receptive fields can integrate information for moving objects (Burr, 1980; Burr, Ross, & Morrone, 1986). However, such a scheme can work only if the orientation of the receptive field matches the trajectory of the moving objects. Since objects can assume arbitrarily complex trajectories, a hard-wired system will necessitate a staggering number of receptive fields at each retinotopic location, each tuned to one possible stimulus trajectory. Such a system will also necessitate a selection process to inhibit those receptive fields that do not match the trajectory of the system so as to avoid motion smear (Ogmen, 2007).