The fate of object features during perisaccadic mislocalization

Abstract

Visual objects flashed before a saccade appear compressed toward the saccade target. Simultaneously flashed objects merge perceptually into one. To better understand cortical interactions in perisaccadic processing, we study the perception of features of mislocalized objects. We report four new findings: First, when multiple objects of different colors are compressed onto a single position, their color attributes remain distinguishable. Second, color attributes of objects compressed onto the same position compete for access to visual awareness. Third, objects presaccadically mislocalized onto a static background of identical color and luminance appear visible on top of that background. Object shape can be determined. Fourth, objects flashed during a saccade become invisible when a larger object is present at the mislocalized position. Thus, perisaccadic mislocalization affects the position of objects but retains other object features. Mislocalization must either occur in parallel to color and shape processing or at late stages of the visual pathway.

Introduction

Saccades are fast, goal-directed eye movements. Gaze switches from one direction to another such that the brain receives two different views of a scene before and after the saccade. Visual information from before the saccade must be mapped to the visual information after the saccade. This mapping involves a dynamic process of spatial transformations before, during, and after the saccade. This dynamic process is observable with visual stimuli that are briefly flashed shortly before or during a saccade. Such stimuli are mislocalized and perceived nearer to the saccade target (Honda, 1989; Matin & Pearce, 1965; Ross, Morrone, & Burr, 1997; Schlag & Schlag-Rey, 1995). The mislocalization consists of a shift of apparent position in the direction of the saccade (Cai, Pouget, Schlag-Rey, & Schlag, 1997; Honda, 1989, 1995; Lappe, Awater, & Krekelberg, 2000; Schlag & Schlag-Rey, 1995) and a compression of positions onto the saccade target (Bischof & Kramer, 1968; Lappe et al., 2000; Ross et al., 1997). For peripheral stimuli, mislocalization also occurs orthogonal to saccade direction (Kaiser & Lappe, 2004). 

At saccade onset, the mislocalization can become very large (up to several degrees of visual angle) such that under certain conditions, any stimulus is compressed fully onto the saccade target (Morrone, Ross, & Burr, 1997; Ross et al., 1997). Morrone et al. (1997) showed that four identical green bars on a red background are seen as one when presented immediately before a saccade. 

How this effect is generated in the visual system remains an open question. Many researchers believe that the perisaccadic shift of apparent position is related to mechanisms of space constancy across saccades (Honda, 1989; Lappe et al., 2000; Matin & Pearce, 1965; Pola, 2004; Ross, Burr, & Morrone, 1996; Schlag & Schlag-Rey, 1995). However, additional processes are needed to explain the origin of compression (Awater & Lappe, 2006; Hamker, Zirnsak, Calow, & Lappe, 2004; Morrone et al., 1997; Van Rullen, 2004). The brain areas responsible for mislocalization and compression are unknown. Experiments with dichoptic stimuli showed that areas beyond retina and lateral geniculate nucleus must be involved (Honda, 1995). 

To learn more about cortical interactions in perisaccadic processing, we investigate how perception of the features of objects is organized when the stimuli are mislocalized such that they appear to occupy the same position in visual space. The representation of space is considered mainly a feature of the dorsal visual pathway of the brain (Milner & Goodale, 1995; Ungerleider & Mishkin, 1982). It is therefore interesting to see whether nonspatial features of perisaccadic stimuli are also affected by perisaccadic compression. In two series of experiments, we study the representation of features of objects that are mislocalized in the wake of a saccade. 

In the first two experiments, we investigated the perisaccadic perception of multiple simultaneously flashed bars that differ in color. We show that even when compression is strong enough that multiple bars melt into a single object, perception of the individual color signals is retained. In the second series of experiments, we investigate whether mislocalization occurs before or after shape recognition. We create experimental conditions in which the mislocalization of a flashed object shifts the object onto a masking area of the same color and luminance as the object. If the shift of position occurs in visual processing before the form of the object is analyzed, subjects should not be able to recognize the object on the same-color area. However, if the shift occurs after object recognition or if position and form are processed independently from each other, subjects should be able to perceive the mislocalized object on top of the bar. We will show that shortly before saccade onset, subjects see the object on top of the same-color area and can still reliably discriminate its shape. We conclude that mislocalization affects only the position of the flashed object and not its other features. 

Methods

Observers

Six subjects (three female and three male, between 25 and 40 years old) participated in the study. All had normal or corrected-to-normal vision. The number of subjects varied for each experiment. All subjects gave informed consent. All procedures were in accord with institutional guidelines and conformed to the Declaration of Helsinki. 

General setup and eye movement recording

Subjects sat in a dimly lit room in front of a 19-in. monitor (Samtron 95P plus) with a visible screen area of 36.6 × 27.5 cm. The full-screen images had a resolution of 800 × 600 pixels presented with a frame rate of 144 Hz. The distance from the subject to the screen was 40 cm, and the head was fixed by a chin rest. Because the screen in all experiments displayed a luminous background, the borders of the screen were clearly visible. Movements of the eyes were recorded with a video-based eye tracker (EyeLink, SensoMotoric Instruments, Inc.) at a sampling rate of 250 Hz. The start of the eye movement was determined by a velocity threshold of 35°/s. 

Stimuli and procedure

Experiments with multiple flashed stimuli

Each trial began with the appearance of a fixation point 6.4° left of the screen center on a gray background (luminance, 5.7 cd m ⁻²; Figure 1). After a random time between 500 and 1,500 ms, the fixation point disappeared, and a saccade target was shown for 50 ms 6.4° right of the screen center. The observer performed a 12.8° saccade toward the target. At a random time between 150 ms before and 150 ms after saccade onset, the stimulus was flashed for one video frame (7 ms). Because the next video frame replaced the stimulus location with the background luminance, again, the stimulus did not persist on the screen for more than 7 ms. The stimulus in the first experiment consisted of one, two, or three 1.2°-wide bars that vertically extend across the entire display. Each bar could be of any of the three colors we used (red: luminance, 5.1 cd m ⁻²; green: luminance, 15.5 cd m ⁻²; blue: luminance, 3.5 cd m ⁻²). All possible stimulus combinations were presented randomly interleaved such that the observer never knew in any single trial how many bars or colors would be presented. Five hundred milliseconds after the flash of the stimulus, a response screen was shown. For each perceived bar, the observer was allowed to report one or several of the three colors or to click the gray field if he or she perceived a bar but did not perceive its color. When all of the colors for all of the perceived bars (0–3) were reported, a mouse click on the “next” field advanced to the next trial. 

Figure 1

 

Experimental setup for the first experiment. First, the fixation point was presented. It disappeared when the saccade target was presented for 50 ms. Around the time of the saccade, a stimulus of one, two, or three bars of one or multiple colors was flashed for 7 ms. Five hundred milliseconds later, a response screen and a mouse pointer appeared. The subject indicated the number and colors of the perceived bars in a subsequently displayed response screen. For each perceived bar, the observer was allowed to report one or several of the three colors or to click the gray field if he or she perceived a bar but did not perceive its color. This occurred occasionally presumably because observers failed to encode or retrieve all of the presented colors in visual short-term memory. When all of the colors for all of the perceived bars (0–3) were reported, a mouse click on the “next” field advanced to the next trial.

In the second experiment, the stimulus consisted of one, two, or three squares with 1.2° side length. All other stimulus properties were the same as in the first experiment. 

Experiments with static masking area

These experiments created conditions in which a flashed object was perisaccadically shifted onto a static, continuously visible area of identical color and luminance. The stimuli were presented on a red (luminance, 4.8 cd m ⁻²) screen with a black horizontal scale in the middle ( Figure 2). An 8°-wide green area (luminance, 21.5 cd m ⁻²) was presented permanently from 2° to 10° right of the screen center. Each trial started with the appearance of a fixation point 14.5° left of the screen center. The fixation point was presented for a randomized time interval (500–1,500 ms). Then, it disappeared, and the saccade target point was presented for 15 ms 14.5° right of the screen center. The subject was instructed to make a 29° saccade rightward to this target as soon as it appeared. After a randomized period, the triangle stimulus (side length, 1.85°; luminance, 21.5 cd m ⁻²) was flashed for 7 ms 7.9° above the screen center. The tip of the triangle was oriented upward or downward. Five hundred milliseconds after the disappearance of the triangle, the subject indicated the perceived position of the triangle by using a mouse pointer. Thereafter, a response screen appeared on which the subject indicated the perceived orientation of the triangle (upward or downward). If the subjects could not give an answer because they had not perceived the flashed triangle, they reported suppression by clicking on the left edge of the screen. 

Figure 2

 

Experimental setup for experiments with a static masking area. First, a fixation point appeared. The fixation point disappeared when the saccade target was presented for 50 ms. Around the time of the saccade, a triangle was flashed for 7 ms, with the tip pointing either up or down. Five hundred milliseconds later, a mouse pointer appeared, and the subject indicated the perceived position of the triangle. In the subsequent response screen, the subject reported the perceived orientation of the triangle. Clicking in the upper half of the screen signified the upward orientation. Clicking in the lower half of the screen signified the downward orientation.

Data analysis

The experimental data were analyzed using Mathematica (Wolfram Research Inc.). Trials in which saccades were initiated before target presentation or in which the saccade endpoint was further than 4.9° from the saccade target were omitted from further analysis. Between 100% and 44% of the trials could be used for analysis. 

Results

Simultaneous bars of different color

In the first experiment, observers performed 12.8° rightward saccades over a gray background. Slightly before, during, or after the saccade, a stimulus screen was presented for 7 ms. It contained between one and three vertical bars, 1.2° wide and 2.4° apart, either of a single color or of different colors (red, green, and blue). Observers were asked to report the number of bars and to name the perceived colors. 

Various possible outcomes of this experiment would constrain the mechanisms involved in perisaccadic compression to certain physiological substrates. If mixtures of colors were perceived, for example, purple when red and blue were presented, the misrepresentation of position must have occurred prior to or concurrent with the evaluation of color. This would point toward early visual areas. If mixtures of colors were not perceived, the misrepresentation of position is likely to occur in higher visual areas and in parallel to the representation of color. 

Initial experiments with free naming of perceived colors clearly showed that mixtures of colors were not perceived. Instead, in trials in which a large compression occurred, observers reported that they saw several of the presented colors within a single object located at the saccade target. This was independently confirmed from earlier observations by John Ross, David Burr, and Concetta Morrone at a laboratory in Perth, Australia (personal communication). We therefore employed a quantitative reporting technique that allowed the observers to report independently the number of objects and the number of colors they perceived in the same trial ( Figure 1). Three observers (S.K., M.L., and M.D.) took part in this experiment. 

Consistent with earlier reports (Matsumiya & Uchikawa, 2001; Morrone et al., 1997), in a brief period around the onset of the saccade, often only one bar was seen even when two or three bars were presented. This was also true when multiple bars with different colors were shown. Figure 3A shows the perceived number of bars as a function of presentation time relative to saccade onset. Black symbols denote the condition when three bars of identical color were shown. Red symbols denote the condition when three bars of different colors were shown. The curves present running averages through the single-trial data. In both the identical color and the different color conditions, the number of perceived bars is veridical long before and after the saccade. Starting about 100 ms before the saccade, observers begin to report seeing fewer than three bars. Immediately at saccade onset, they often perceive only a single bar. 

Figure 3

 

(A) Perceived number of bars as a function of their presentation time relative to saccade onset for three observers. Each symbol is a report from a single experimental trial. Lines provide running averages through the data points. Red symbols and lines represent data for three bars of different color. Black symbols and lines represent data for three bars of identical color. (B) Perceived number of colors as a function of stimulus presentation time relative to saccade onset. Red symbols and lines are for three bars of different color. Black symbols and lines are for three bars of a single identical color.

Figure 3B plots the perceived number of colors in the same trials. When all bars were of the same color (black symbols), that color was reported veridically. Also, when the bars were of different colors (red symbols), observers often veridically reported the number of colors. Particularly for stimuli flashed at saccade onset, when the average reported number of bars was between one and two, observers S.K. and M.D. reported seeing all three colors and observer M.L. reported either three or two colors. Thus, the number of colors was perceived almost veridically around the time of the saccade, although the number of bars was perceptually reduced. 

For statistical testing, we compared the mean number of perceived colors and the mean number of perceived bars in the time range when the effect was strongest, that is, from 20 ms before to 20 ms after saccade onset. When different colors were presented, the mean number of colors was significantly larger than the mean number of bars in all three subjects (Mann–Whitney U test, p < .05). 

Data in the condition with two bars of different colors gave the same result. In the time range from 20 ms before to 20 ms after saccade onset, the mean number of perceived colors was significantly larger than the mean number of perceived bars in all three subjects (Mann–Whitney U test, p < .05). The results from conditions with one and two bars or from those with one and two colors served as controls to check that subjects did not report percepts of stimuli that were not presented. We determined the number of cases in which more bars or colors were reported than were presented. A report of a nonpresented bar occurred in one trial in subject M.L., in three trials in subject M.D., and in none of the trials in subject S.K.; none of these occurred within the period of perisaccadic compression (50 ms before to 50 ms after saccade onset). A report of a nonpresented color occurred in eight trials in subject M.L. (four within 50 ms before and after saccade onset), in eight trials in subject M.D. (four within 50 ms before and after saccade onset), and in two trials in subject S.K. (none within 50 ms before and after saccade onset). Because these occurrences are evidently rare (less than 2% of trials in each case), we believe that they simply reflect reporting errors. 

For subjects S.K. and M.L., the number of perceived colors was lower than three when the stimulus was presented long before the saccade, especially when three colors were shown. This may be due to memory deterioration over time as the task is very demanding due to the short presentation time. Also, because of the difficulty of the task, we did not require our subjects to report the bars in sequential order. This had turned out to be too difficult in preliminary testing. Therefore, we cannot determine, on a trial-by-trial basis, which bar was not reported long before the saccade. Informal analysis of the response data and verbal reports of the subjects suggested, however, that it was often the color of the most eccentric bar position that was not registered long before the saccade. Thus, there may also be an influence of the visibility of the color in peripheral vision. 

Small squares of different color

What does the percept at saccade onset look like? When asked, observers described that they sometimes perceived a single vertically elongated object at the saccade target position containing multiple patches of color. The position and distribution of the color patches were difficult to ascertain. This could suggest that each bar was perceptually shifted onto the saccade target position and that the color signals from each bar then competed for perceptual awareness. Because the bars were extended objects, this competition could take place at a range of locations and could lead to locally different percepts. Similar local competitions have been observed in experiments on binocular rivalry in which two different inputs in the two eyes compete for perceptual awareness in local image patches (Kovacs, Papathomas, Yang, & Fehr, 1996; Lee & Blake, 2004). To test whether such a local competition underlies the perception of multiple colors in our perisaccadic experiments, we reduced the size of the stimuli. We reasoned that if stimuli were smaller than the patch size over which perceptual rivalry occurs, then the number of simultaneously perceived colors should be lower. 

All experimental methods were the same as in the previous experiment except that small squares (1.2° × 1.2°) rather than long bars were used as stimuli. This stimulus size was chosen as a compromise between the introspective size of the patches and the visibility of the stimuli during the saccade. Preliminary testing with smaller stimuli showed that these were increasingly hard to see during the saccade such that many trials were unusable because of suppression. Subjects S.K. and M.L. participated. Figure 4 shows results for the condition with three differently colored squares (blue symbols and lines) together with those for three differently colored bars from the previous experiment (red symbols and lines). Around saccade onset, the number of perceived squares, like the number of bars in the previous experiment, was low ( Figure 3A). However, unlike in the previous experiment, the number of simultaneously perceived colors was also low around saccade onset ( Figure 4B). The mean number of perceived colors in the time range from 20 ms before to 20 ms after saccade onset was significantly lower in the squares condition than in the bar condition for both subjects (Mann–Whitney U test, p < .05). This was true in the condition with three simultaneous objects ( Figure 4B) and in the condition with two simultaneous objects (not shown). Thus, reducing the size of the stimuli reduced the likelihood to perceive multiple colors. This is consistent with our hypothesis that, for smaller stimuli, perceptual rivalry induced by saccadic mislocalization suppresses the perception of color features. 

Figure 4

 

Comparison of the number of perceived objects and the number of perceived colors for three differently colored elongated bars (red) and three differently colored small squares (blue). (A) Number of perceived objects. (B) Number of perceived colors.

Flashed object shifted onto a static background

The above experiments showed that when stimuli are flashed simultaneously at saccade onset, they appear to occupy the same spatial position but compete for awareness in terms of their color features. Multiple stimuli of the same color melt into one. In the next experiment, we wanted to test whether multiple objects of the same color can be distinguished even when they come to occupy the same spatial locations because of the perisaccadic mislocalization. We looked for conditions under which a flashed object would be mislocalized onto a static area of the same color and asked whether the shape of the object could still be discriminated under these conditions. The stimulus was a flashed green triangle presented on a red background ( Figure 2). The background screen had a black horizontal scale in the middle. An 8°-wide green area (same color and luminance as the triangle) was presented permanently from 2° to 10° right of the midpoint of the screen (position 0°). Subjects made a 29° saccade rightward from a fixation point to the target. The triangle stimulus was flashed for 7 ms 7.9° above the screen center. The tip of the triangle was oriented upward or downward. Observers had to report the apparent position and orientation (upward or downward) of the triangle. 

In a sizable proportion of trials, the observers failed to see the flash of the triangle. In which case, the observer was instructed to click on the left edge of the screen. These trials were also recorded and classified as suppression trials. In separate blocks of trials, a control condition in which the large green area was absent was presented. 

A prerequisite for the data analysis of this experiment was that mislocalization had to be strong enough so that the triangle was shifted at least 2.9° toward the saccade target. Only in this case would the triangle appear completely on top of the same-color area. This prerequisite was reached in three of the five subjects who participated in the experiment. We therefore report only data from these three subjects. 

Figure 5 shows the perceived position of the flashed triangle as a function of the presentation time relative to the start of the saccade. The symbols identify the response to the shape discrimination. Triangles mark trials in which the shape of the triangle was classified correctly. Asterisks mark trials in which the shape of the triangle was classified incorrectly. Diamonds mark suppression trials in which the subject did not perceive the stimulus and thus could not report the orientation or the location of the triangle. 

Figure 5

 

(A) The perceived horizontal position of the green triangle as a function of its presentation time with respect to saccade onset. A large green area is present throughout the trial. The two horizontal lines at 2.9° and 9.1° indicate the response range over which the perceived triangle appears fully within the large green area. Each symbol corresponds to a single trial and identifies the subject's response in that trial. Triangles correspond to correct shape discrimination. Asterisks show cases where subjects reported the orientation of the triangle incorrectly. Black diamonds at the bottom of the figure indicate cases where the subject did not perceive the flashed object. The gray area indicates the beginning and the average duration of the saccade for each individual. (B) Data from a control condition in which the large green area was absent throughout the trial.

Figure 5 reveals distinct time phases in which different perceptual effects occurred. First, more than 50 ms before and after the saccade, the stimulus is seen near its veridical location and the shape discrimination is correct in all trials. Second, within 50 ms before saccade onset, the apparent location of the stimulus is shifted in saccade direction. In many trials, the shift is strong enough that the stimulus appears completely within the static green area of the screen. This is the case for all reported apparent positions between 2.9° and 9.1°. This region is identified by the horizontal lines in Figure 5. In nearly all cases in which the observer reportedly saw the stimulus completely within the green area was the orientation of the triangle correctly identified. Third, from within a few milliseconds before saccade onset until close to saccade end did all three observers (as well as the two observers that were not reported because they did not show sufficient presaccadic mislocalization) consistently fail to see the stimulus flash at all. We conclude that, within a brief time interval before saccade onset, the stimulus is shifted such that it appears fully confined within the static area of the same color, but, at the same time, the shape of the stimulus can be perceived reliably. When asked about their percept, subjects indeed reported that they clearly saw the triangle flashed on top of the green area. We conclude, secondly, that stimuli flashed during the saccade are suppressed and cannot be located or classified. 

Because of the suppression, it is difficult to quantify the recognition score precisely. Because our subjects had to report both the location and the orientation of the stimulus, a forced-choice procedure did not seem sensible as there is no chance quantification for the perceived location. We thus calculated the recognition score among all trials in which the stimulus was perceived fully within the static green area along with 99% confidence intervals from the binomial distribution. Recognition scores were 94% for B.O. ( n = 15; confidence interval, 74–100%), 100% for H.A. ( n = 16; confidence interval, 97–100%), and 97% for M.L. ( n = 32; confidence interval, 86–100%). Thus, all recognition scores were well above chance level. 

In earlier preliminary experiments with a slightly different setup, we had found recognition scores to be also significantly above the chance level of 50% for the forced-choice procedure. In those experiments, a rectangular stimulus (6° wide and 2° high) was used instead of a triangle. The rectangular stimulus was either solid or contained a 2°-wide gap in the middle. Subjects had to decide in each trial whether the stimulus contained a gap or not and, secondly, had to indicate with the mouse pointer the apparent position of the left edge of the stimulus. The decision whether the stimulus contained a gap or not was forced choice; that is, it had to be made also when the subject was unable to report the location of the stimulus. For the analysis, all trials in which the left edge of the stimulus was within 2° of the masking area (i.e., the gap was fully within the masking area) or in which the stimulus could not be located because of suppression were used. For all three participating subjects in that experiment, the percentage of correct object identification was significantly above chance ( p < .01), with values of 73% (M.K., 104 trials), 72% (M.L., 94 trials), and 63% (H.A., 101 trials). 

In the control condition without the green area, there was no suppression ( Figure 5). Time course and extent of mislocalization before the saccade were similar to the experimental condition. The maximal horizontal mislocalization was observed around saccade start. These results confirm that all three subjects showed sufficient mislocalization before the saccade to shift the apparent position of the stimulus inside the region of the green area in the experimental condition. 

Saliency from early visual processing

The above results show that in a brief time interval before a saccade, the flashed object may appear shifted onto an area of identical color and luminance but still be visible in terms of its shape. We ran two further conditions to explore the limits of this percept. One condition presented a black triangle (luminance, 0.03 cd m ⁻²) and a black area of the same luminance on top of a light gray background (luminance, 0.6 cd m ⁻²). Three subjects (B.O., H.A., and M.L.) participated. All subjects could perceive the shifted triangle on top of the bar ( Figure 6A). The proportion of triangles with correctly recognized orientation was 95% for B.O. ( n = 22; confidence interval, 81–100%), 94% for H.A. ( n = 17; confidence interval, 76–100%), and 91% for M.L. ( n = 53; confidence interval, 79–100%). 

Figure 6

 

(A) The perceived horizontal position of a black triangle as a function of its presentation time with respect to saccade onset. A large black area is present throughout the trial. The two horizontal lines at 2.9° and 9.1° indicate the response range over which the perceived triangle appears fully within the large black area. Each symbol corresponds to a single trial and identifies the subject's response in that trial. Triangles correspond to correct shape discrimination. Asterisks show cases where subjects reported the orientation of the triangle incorrectly. Black diamonds at the bottom of the figure indicate cases where the subject did not perceive the flashed object. The gray area indicates the beginning and the average duration of the saccade for each individual. (B) Results from a condition in which the edges of the triangle were blurred. The blurred triangle and the large masking area were yellow in this condition.

In the second condition, the edges of the triangle stimulus were blurred to eliminate early contrast enhancement from lateral inhibition. In this condition, a blurred yellow triangle and a yellow static area were presented on a red background (cf. Figure 6B). The luminance in the center of the triangle was 12.4 cd m ⁻². Luminance decreased toward the blurred borders of the object until it matched the luminance of the red background (4.8 cd m ⁻²). The luminance of the static yellow area was 24.7 cd m ⁻². Three subjects (B.O., H.A., and M.L.) participated. B.O. and M.L. could reliably discriminate the blurred triangle even when it appeared shifted onto the static yellow area. The proportion of correct responses was slightly smaller than in the preceding experiments: 95% for B.O. ( n = 43; 95% confidence interval, 68–100%) and 83% for M.L. ( n = 24; 95% confidence interval, 66–100%). Subject H.A. could not reliably discriminate objects on top of the bar. However, this condition was generally more difficult due to the blurring. Recognition rates for triangles that were not shifted onto the yellow area were also low: 87% for B.O. ( n = 102; 99% confidence interval, 78–96%), 84% for H.A. ( n = 93; 99% confidence interval, 73–94%), and 84% for M.L. ( n = 57; 99% confidence interval, 71–98%). 

Masking by the static area

In the experiments described above, observers could reliably discriminate the orientation of a flashed triangle that appeared shifted onto an area of the same color and luminance. However, from about 5 ms before to 50 ms after the beginning of the eye movement, observers consistently failed to perceive the flashed object at all. This failure is not simply due to saccadic suppression because in the control condition where the green area was not present, no suppression occurred ( Figure 6). It seems likely that the suppression is due to some form of visual masking by the static green area. We were interested in the properties of this masking. 

We first asked whether the masking is specific for the color of the object and the masking area. Therefore, we tested whether suppression still occurred even if the triangle and static area had different colors. We presented blue triangles (luminance, 2.8 cd m ⁻²) and the standard green static area. Three subjects participated in this experiment (B.O., H.F., and S.K.). All subjects showed complete suppression of the triangle during the saccade. We conclude that the suppression is not dependent on the triangle and masking area having the same color. 

We next tested at what time the presence of the green area interferes with the information about the flashed triangle. Therefore, we examined the extent of suppression/masking when the green area was only present before or after the triangle presentation. In the first condition, the bar was present from the beginning of the trial and disappeared immediately before the triangle was flashed. This condition was tested with only one subject (B.O.), but it was clear that the triangle was perceived in all trials. Therefore, forward masking did not take place. 

In the second condition, the bar only appeared immediately after the triangle was flashed and remained present until the end of the trial. Three subjects (B.O., H.F., and S.K.) participated. They exhibited complete suppression when the triangle was flashed between about 5 ms before and 50 ms after the start of the saccade. Figure 7B plots the time course of suppression in this condition and in the condition of the earlier experiments. Suppression begins just before saccade onset and ends not later than 60 ms after the start of the saccade. 

Figure 7

 

Time course of suppression. Each panel shows the percentage of trials in which the flashed object was not perceived as a function of presentation time relative to saccade onset. (A) The green masking area is visible throughout the trial. (B) The green masking area appears after the triangle appears on the screen.

In the third condition, the masking area appeared only after the saccade. Eye position was monitored online by the stimulus program. The green area was presented 45 ms after the eye position crossed the vertical midline of the screen during the saccade. This ensured that the green area in most trials appeared shortly after saccade termination. For the data analysis, all trials in which the green area appeared before the end of the saccade were discarded. Subjects B.O., H.F., M.L., and S.K. were tested in this condition. No suppression occurred for subjects B.O., H.F., and M.L. S.K. reported that she did not see the stimulus in 1 of 325 trials. We conclude that presentation after the saccade does not induce masking but that presence of the masking area during the saccade is necessary. 

Discussion

Our results show that perisaccadic compression affects the position coding of stimuli but leaves color and form attributes intact. A dissociation between position encoding and form attributes during saccades has also been reported by Matsumiya and Uchikawa (2001). These dissociations suggest that the neurobiological origin of perisaccadic compression is restricted to spatial processing. 

Both of our series of experiments address the question of whether mislocalization occurs at a processing stage where stimulus attributes are kept separate or at a stage where they have been combined. To discuss this question, we assume that the different features (color, shape, and position) are initially registered separately by sensory mechanisms selective for color, shape, and position. We further assume that the registration of color and shape occurs in retinotopic maps in which the activation is initially at the retinal position of the stimulus. Thus, the representation of position may be derived from the activation locations in the color or shape maps, or it may be derived from a separate position map. 

For the colored bars, if mislocalization occurs in the color map, then the activations of different colors should become overlapping. If this overlap occurs before the spectral distribution of wavelength is transformed into a color percept, then the activations should merge and the percept should be a color mixture, for example, purple if red and blue were mixed. Because this was not observed, the mislocalization must have occurred after the wavelength distribution is transformed into a color percept. Because this is believed to occur in V4, it is unlikely that the mislocalization occurs in V1 or V2. If, consequently, the mislocalization occurs in a color map in V4 or beyond, then the representations of the different colors would overlap in the map. The activation in the map would indicate multiple objects with different colors at the same location. This conflict could be resolved by competition between the percepts, ideally leading to a single perceptual object with one of the colors or, if competition is local, leading to an object (or area) with patches of color. Our findings are consistent with the latter. 

There is, however, an alternative route by which this percept may arise. We assume that the position information is not derived directly from the perceptual color map but rather from a separate map of position. In this case, the colors are registered veridically from a color map (sensory or perceptual) in which the activations are not mislocalized. The mislocalization occurs in the position map such that the overlap of the shifted activities in the position map signals a single position. If the percept of the scene is derived from combing the information from multiple maps in multiple visual areas, then the percept should be that of a single object and multiple colors at the same time. This is also consistent with our results. We can, therefore, not distinguish between those two possibilities, but we can rule out a mislocalization in an early map in which wavelength, not color, is represented. 

Similar arguments apply to the shape experiments with a static masking area. In these experiments, it was possible to recognize a flashed object that was perceived on top of a same-colored area. If we assume that the static masking area leads to activation in the initial sensory map, then, if the activation by the stimulus is mislocalized in this sensory map, the activation at the mislocalized position should be indistinguishable from the background activation and the stimulus should become invisible. There are two possible caveats here, however. First, it may be that the activation by the stimulus is added to the background activation such that the total activation at that position exceeds the background activation. We have tried to address this issue with the experiment in which a black stimulus and a black background were used. Secondly, the briefly presented stimulus is processed and represented differently than the static stimulus. Because of its transient nature, it will have an onset response stronger than the activities in the static background area, and its perceived luminance may be less than its actual luminance because there is no time for integration processes acting in the early visual system. However, we believe that the results from the first experiments, in which all bar stimuli were transiently flashed, already show that the mislocalization does not occur in early maps such as V1 or V2. It is therefore sensible to assume that the mislocalization in the static masking experiments also occurs in higher visual areas such as V4 or parietal cortex. The static mask experiments, furthermore, show that shape information can be assessed for stimuli that appear to lie within the masking area. Again, this can be explained if the representation of the position of the flashed stimulus is processed independently from the shape of the stimulus or if the representation of the shape is mislocalized at a stage in which shape has already been established. The latter possibility seems consistent with mislocalization in area V4 or inferotemporal cortex. The former possibility is consistent with mislocalization arising in the parietal stream and the percept being generated from the interaction between areas. 

We thus conclude that either the shift of perceived position occurs after the shape of the object is recognized or mislocalization and form analysis take place independently from each other on separate processing pathways. In both cases, mislocalization must occur after the separation into dorsal and ventral streams, that is, after V2. Thus, compression must arise at intermediate or high stages of cortical processing such as area V4 (Tolias et al., 2001), areas MT/MST (Krekelberg, Kubischik, & Bremmer, 2000), and the inferotemporal or the parietal cortex (Duhamel, Colby, & Goldberg, 1992). 

If the perceived spatial position of perisaccadically flashed objects is compressed independently of other object attributes such as color, then multiple objects can come to occupy the same perceived location during a saccade. In this case, their color attributes become conflicting. The observation of color patches in the experiment with extended bars and the reduction of multicolor percepts in the experiment with small squares suggest that this conflict is resolved by local competition. This effect may be similar to processes of local competition observed in binocular or perceptual rivalry in which, also, two inputs compete for access to perceptual awareness at a single spatial position (Kovacs et al., 1996; Lee & Blake, 2004). In perceptual rivalry, competing stimuli lead to alternating percepts over time courses of seconds. Because stimuli were flashed for only 7 ms in our experiments, perceptual alternations are unlikely to occur. Yet, the mechanisms of mutual inhibition and dominance of one percept over the other may be similar and, if local, may lead to different percepts dominating at different positions. 

We also tested the possibility that the object is shifted onto the masking area but becomes salient for later object recognition stages by luminance addition or edge contrast enhancement. Lateral inhibition intensifies the contrast between object and background. Therefore, neurons in the retina and in V1 are activated more by contrast than by absolute brightness of an object. When edges are detected, the form is transferred to the shifted position. But even with reduced edge contrast, triangle orientation could be correctly recognized for objects that were perceived on top of the same-colored area. We conclude that sharp edges are not an essential condition for perceiving a triangle on top of a same-colored area. 

Our observers were only able to perceive the triangle on top of the same-colored area if mislocalization was large enough within the time window from the onset of mislocalization to the onset of suppression. This time window begins about 50 ms before saccade onset and lasts until a few milliseconds before saccade onset. If the stimulus is flashed later during the saccade, then it is not perceived. This inability to see the stimulus during the saccade did not result from regular saccadic suppression (Ross, Morrone, Goldberg, & Burr, 2001) because it occurred only in the presence of the green area and not when a homogeneous red background was used. Moreover, whereas the time course of saccadic suppression and that of perisaccadic mislocalization are very similar, both beginning 50 ms before the saccade (Diamond, Ross, & Morrone, 2000; Michels & Lappe, 2004), the time course of suppression of the triangle stimulus in the present experiments is clearly different, as it begins only very shortly before saccade onset and much later than the mislocalization. 

We therefore suggest that the inability to see the stimulus during the saccade is due to some form of visual masking. The green area comes into the center of gaze after the saccade and could diminish the visibility of the flashed object perceived at that position (the target position) via backward masking (Enns & Di Lollo, 2000). A further possibility is that the saccade-induced motion signal of the green area directly masks the visibility of the flashed object. This possibility may seem unlikely because visual motion perception is suppressed during the saccade. However, the visual motion signals generated by the saccade are represented in area MT (Thiele, Henning, Kubischik, & Hoffmann, 2002) and thus may interact with the perception of the stimulus. 

In conclusion, we have shown that perisaccadic mislocalization occurs after V2 in either the dorsal pathway or at areas where dorsal and ventral streams converge again. The ventral stream seems to receive the correct feature information as both color and form can be processed veridically. Further tracking down of the responsible area might be difficult in the domain of psychophysics. On the other hand, the effects are too rapid for functional imaging, and electrophysiology in humans is not an option. As models that explain mislocalization rely either on the form of cortical maps or on feedback projections (Hamker et al., 2004; Van Rullen, 2004), a careful study of connectivity (Sporns, Chialvo, Kaiser, & Hilgetag, 2004) might help to answer these questions. 

Acknowledgments

This work benefited from many discussions with C. Morrone, J. Ross, and D. Burr within a collaborative project funded by HFSP (RG149/1999B). Further support to M.L. was provided by the German Science Foundation DFG LA-952/2 and LA-952/3, the German Federal Ministry of Education and Research BioFuture Prize, and the EC Project Drivsco. M.K. gratefully acknowledges support from the Fritz-ter-Meer Foundation and the German National Academic Foundation. 

Commercial relationships: none. 

Corresponding author: Markus Lappe. 

Email: [e-mail]. 

Address: Psychological Institute II, West. Wilhelms University, Fliednerstr. 21, 48149 Muenster, Germany.