The effect of perceptual grouping on perisaccadic spatial distortions

Abstract

Abstract

Abstract:

Abstract  Perisaccadic spatial distortion (PSD) occurs when a target is flashed immediately before the onset of a saccade and it appears displaced in the direction of the saccade. In previous studies, the magnitude of PSD of a single target was affected by multiple experimental parameters, such as the target's luminance and its position relative to the central fixation target. Here we describe a contextual effect in which the magnitude of the PSD for a target was influenced by the synchronous presentation of another target: PSD for simultaneously presented targets was more uniform than when each was presented individually. Perisaccadic compression was ruled out as a causal factor, and the results suggest that both low- and high-level perceptual grouping mechanisms may account for the change in PSD magnitude. We speculate that perceptual grouping could play a key role in preserving shape constancy during saccadic eye movements.

Introduction

Saccadic eye movements occur several times per second, continually shifting the retinal locus of stationary objects in the visual field, yet their perceived spatial locations remain stable (Bridgeman, Van der Heijden, & Velichkovsky, 1994). This visual constancy phenomenon is a consequence of the cancellation between the retinal image input signals and the efferent copy of the saccadic eye movement signals (von Holst & Mittelstaedt, 1950). However, under transient stimulus conditions, the timing between the onset of the efferent copy signal and the onset of visual input signals may not be perfectly matched, and spatial distortions can occur. For example, foveal targets flashed immediately before the onset of a saccade appear displaced in the direction of the saccade (Dassonville, Schlag, & Schlag-Rey, 1992; Honda, 1989, 1991; Matin, Matin, & Pearce, 1969; Matin & Pearce, 1965). The magnitude of this perisaccadic spatial distortion (PSD) can vary extensively under different stimulus conditions. 

One factor that plays an important role in determining the magnitude of the PSD is the luminance of the flashed target. Zhang, Cantor, and Schor (2008) found that PSD magnitudes for low-luminance (near detection) targets were much greater than those obtained with higher luminance targets. Luminance also changed the way that PSD magnitude varies as a function of the time between the presentation of the target and the time of saccade onset (TSO). With high-luminance stimuli, the PSD magnitude remained the same or increased gradually as the target was flashed closer in time to the saccade onset (TSO = 0 ms). In a low-luminance condition, the PSD for targets presented before saccade onset increased only up to about 20 ms before saccade onset (TSO = −20 ms), after which the PSD decreased as the TSO approached zero. A similar luminance effect for a bar target was reported by Georg, Hamker, and Lappe (2008). 

Numerous studies have shown that PSDs are also affected by the spatial configuration of the test flash relative to the saccadic endpoint. For example, test flashes between the fixation target and saccade endpoint appear displaced with the direction of the saccade, and test flashes presented beyond the saccade end point appear displaced against the direction of the saccade, toward the saccade endpoint. This anisotropic pattern of perceived displacement is referred to as perisaccadic compression of visual space (Lappe, Awater, & Krekelberg, 2000; Ross, Morrone, & Burr 1997). 

Natural scenes consist of complex visual stimuli with different surface textures and luminance levels, distributed at different distances from any given fixation point. What should we expect to happen to such stimuli when they are presented (flashed) perisaccadically? If stimuli with different characteristics are expected to undergo different magnitudes of PSD when presented in isolation, will an ensemble of such differing features also produce unequal magnitudes of PSD for the component parts, producing spatial distortion or scrambling of an object? To what extent will the ensemble group together perceptually, undergoing a uniform PSD that maintains its relative spatial arrangement? 

Most studies of PSD have used single targets that are flashed in isolation, and there is no clear consensus about what happens when a perisaccadic target is presented in the context of other visual stimulation. The compression of visual space can alter the perception of complex stimuli, resulting in perceived deformation of complex images or a reduction in the apparent number of objects in an array (Lappe, Kuhlmann, Oerke, & Kaiser, 2006; Ross et al., 1997). On the other hand, there is also evidence that a perceptual grouping effect contributes to shape constancy during saccades. Prior studies found weak or no perisaccadic shape distortion for a texture-defined rectangle and a solid rectangle flashed near the saccade target (Matsumiya & Uchikawa, 2001; Noritake et al., 2009; Sogo & Osaka, 2005). Another study (Sogo & Osaka, 2007) found certain shape distortions of a flashed outlined object presented under low-luminance conditions, but the distortions were much smaller than would be estimated from the PSD of individual targets. A recent study (Luo, Garaas, Pomplun, & Peli, 2010) directly compared the perceived size and location of a solid horizontal bar during a horizontal saccade and found a systematic shift of perceived location but not size compression of the solid bar target. 

Several studies (Brenner, Meijer, & Cornelissen, 2005; Sogo & Osaka 2001, 2002) have suggested that the retinotopic relationship between two asynchronously flashed targets can influence their perceived locations. This could be considered evidence of a grouping process that mitigates or overrides the distortions expected of single perisaccadic targets. However, these studies were based on the premise that PSD magnitudes for single targets depend only on TSO and are not influenced by the targets' luminance or spatial location (i.e., an absence of compression). We developed an experimental paradigm based on the opposite premise: that it is possible to generate stimulus dependence for factors other than TSO. This allowed us to present paired targets simultaneously, the configuration in which grouping effects should be strongest. 

Our study measured PSD for synchronously flashed target pairs with different luminance levels or different spatial configurations relative to the central fixation and saccadic endpoint. The elements of each target pair were presented in isolation (single-target condition) to obtain baseline PSDs for comparison to the perceived distortion elicited in a paired condition, where the pair of targets was presented simultaneously. If the PSD for flashed targets depended only on presentation location and timing, then we would expect the same distortions for our targets regardless of whether they were presented in isolation (single-target condition) or simultaneously with a second target element (paired-target condition). Although the different stimulus attributes of the two flashed objects (luminance and retinal locus) did produce different magnitudes of PSD in single-target conditions, PSD magnitudes in the paired-target conditions were much more uniform. As a result, the perceived relative spatial configuration of flashed target pairs was maintained better than we would predict based on the distortions of the single targets. We interpret our results as suggesting that both low-level and high-level grouping mechanisms affect the magnitude of PSD and help preserve the perceptual stability of perisaccadic visual stimuli. 

Methods

Apparatus and visual stimuli

Flashed targets were presented before a horizontal saccadic near a small initial fixation cross that was presented in primary position. Visual stimuli were displayed monocularly to the left eye on a 20-inch monochrome monitor (DP 104 phosphor with decay time 0.4 ms to 0.1% of peak luminance, Monoray Model M20ECD5RE; Clinton Electronics, Loves Park, IL). The monitor screen was positioned in a completely dark room at 36-cm viewing distance and run at a 120-Hz refresh rate with 1024 × 768 pixel resolution. The edges of the monitor screen were carefully covered by black paper so that they could not be used as a reference landmark. The test stimuli were filled squares (1° × 1°) at luminance 10 cd/m² or 1 cd/m² that were flashed on a dark background (0.7 cd/m²) during one frame with a 2-ms duration. The fixation and saccade targets were 0.5° × 0.5° crosses. 

Experimental design

Experiment 1: Grouping effect of targets with different luminance

Two target configurations were used. The schematic stimulus display is shown in the left diagram of Figure 1c. In a single-target condition, a test square was flashed at an elevation of 1° above or below the fixation cross at the same azimuth as the fixation cross. The luminance of the test square was randomized, set at either 10 cd/m² or 1 cd/m². In a paired-targets condition, two squares were flashed simultaneously (one at 10 cd/m² and the other at 1 cd/m²) at 1° above and 1° below the fixation cross, at the same azimuth as the fixation cross. The locations of the two targets (which target was presented above and which target below fixation) were randomized. Following presentation of the test stimuli, observers indicated the magnitude of the PSDs using adjustable cursors. 

Figure 1

 

Experimental setup of the paired flash condition. (A) Temporal sequence of stimulus presentation in a trial using paired flashes of low and high luminance. (B) Timing of the experimental display. During the saccadic latency, paired squares were simultaneously presented for 2 ms. The TSO is defined as the time from stimulus onset to saccade onset, and the trial is accepted only if TSOs were between −60 and −6 ms. (C) Examples of the two paired flash conditions used in Experiment 1 and 2. In the first experiment (left image), one bright square (10 cd/m²) and one dim square (1 cd/m²) are presented 1° above and below the central fixation cross, both at the same azimuth as the fixation point. In the second experiment (right image), two elevated squares were presented at 1° to the left and right of the azimuth of the fixation point.

Experiment 2: Grouping effect of targets of different horizontal spatial configurations

The schematic stimulus display is shown in the right diagram of Figure 1c. In the single-target condition, squares were always flashed at 1° above the fixation in the vertical meridian and either 1° left or 1° right to the fixation cross in the horizontal meridian. The luminance of the square was always set to 10 cd/m². In the paired-targets condition, a pair of squares at 10 cd/m² was flashed at 1° left and 1° right of the central fixation and 1° above the fixation cross in the vertical meridian. Following presentation of the test stimuli, observers indicated the magnitude of the PSDs using adjustable cursors. 

No-saccade control condition for Experiments 1 and 2

Experiments 1 and 2 were repeated while fixation was maintained. Flashes were presented following a variable time delay after the button press. Following presentation of the test stimuli, observers indicated the perceived screen position of flashed targets using adjustable cursors. This allowed us to measure a bias for perceived position, to give context to the PSD recorded in Experiments 1 and 2. 

Procedure

In each trial, the observer fixated at a small cross (0.5° × 0.5°) at the center of the screen and initiated the trial by pressing a button on a joystick. After a random delay of 400 to 600 ms, the fixation-cross disappeared and a saccade target cross was presented 5° to the left (Figure 1a). The saccade target remained visible for the duration of the trial. The observer then initiated a leftward saccade toward the saccade target, and the test was flashed during the saccade latency period. Based on the preliminary data for saccadic latency of each observer, the test stimuli were flashed for 2 ms at a specific time after the button press to increase the chances that they would be presented in the desired temporal window before the saccade onset. We prioritized collection of responses for flashes presented within 60 ms of the saccade because prior work has shown that PSDs are greatest for TSOs equal to or shorter than this magnitude (Zhang et al., 2008). 

After the test flash stimuli disappeared, the fixation cross reappeared and was kept visible along with the saccade target until the end of each trial (see Figure 1a and 1b as a schematic example of spatial and temporal relationships of the saccade and flashed stimuli). In pilot experiments, we found that presenting the original fixation cross alongside the saccade target during the measurement phase gave observers a sense of scale and reduced the variability of the data. The total presentation time of each trial was 600 ms after the saccade target appeared on the screen. After each trial, the program immediately analyzed eye position signals and accepted trials only in which the TSOs fell between −60 and −6 ms. Negative values of TSO signify presentations occurring prior to saccade onset. 

Quantifying PSD

When a TSO occurred in the accepted range (−60.0 ms to −6.0 ms), two square cursors were presented on the screen after the trial. The cursors appeared at an azimuth position halfway between the test flash position and the saccade target position, at the same elevations as the test flashes, with an added position noise from a uniform normal distribution within −0.3° and +0.3°. The fixation cross and the saccadic target cross were also presented on the screen as references for cursor adjustment. The observer was asked to adjust the horizontal and vertical positions of the cursors using button presses to the screen locations where they had perceived the test flashes (Lappe et al., 2000; Sogo & Osaka, 2002; Zhang et al., 2008). The resolution of adjustment was 0.1° at each button press. Each experiment was repeated on different days. 

In both of our experiments, we used the no-saccade control measurements to account for any positional biases for flashed targets that might affect the magnitude of the measured PSD. The data points in this no-saccade condition were subtracted from PSD measures with saccades to compute a normalized PSD. 

Data analysis

Online trial selection

After each trial, the horizontal conjugate saccade recordings were differentiated with a 2-point-difference algorithm to obtain velocity profiles. The saccade onset time was defined as the first point that velocities of three successive conjugate data points exceeded 30°/s. TSO was then calculated as the difference between the time when the flash stimuli occurred and onset time of the saccade. Only the trials in which the program detected a saccade and the TSO was between −60 and −6 ms were accepted for further analysis. 

Offline trial selection

The accepted trials were reanalyzed by a custom Matlab program. A trial was rejected if the saccadic amplitude in horizontal meridian was beyond 5° ± 1° for the 5° saccade or if a secondary horizontal saccade occurred within 200 ms of the completion of the primary saccade. Accepted trials in the same conditions were pooled together from experiments on different days, and data were also pooled for targets above and below fixation in Experiment 1. The pooled results were binned into six groups ranging from −60 ms to −6 ms. PSDs within each bin were averaged to get a mean value of perceived distortion for each TSO. Overall, 21% of the trials of Observer 1, 40% of the trials of Observer 2, and 19% of the trials of Observer 3 were rejected, and the mean number of measurements within each bin (across conditions) was 70 for Observer 1, 63 for Observer 2, and 64 for Observer 3. A t test with Bonferroni correction was applied to study the effects of different experimental conditions on the mean PSDs. The corrected alpha level was 0.05/number of comparisons. Unless otherwise noted, the data points (and error bars) in the figures specify the mean value (±standard error). 

Eye movement recording and calibration

Horizontal eye positions were sampled at 500 Hz using a video-based binocular Eyelink II tracking system on a slave PC, which is synchronized and controlled by a master PC through Ethernet. A custom Matlab program with Psychtoolbox 3 (Brainard, 1997; Pelli, 1997) running on the master PC displayed the stimulus and retrieved and analyzed eye position data for each trial. The observer's head was stabilized using a bite bar and forehead rest. Calibration of the eye tracker was run before each session by asking observers to fixate sequentially presented horizontal and vertical spots on the screen, and the output voltages were converted to degrees by linear regression. Calibration of the right and left eyes was performed separately. 

Observers

Five adults with normal or optically corrected-to-normal vision and normal binocular eye alignment served as observers. The study was approved by the human subjects committee at the University of California, Berkeley, and each observer signed a consent form before the experiments. 

Results

Experiment 1: Grouping effects of paired targets with unequal luminance

Our first experiment compared the PSD elicited by an isolated test target and PSD when the same stimulus was presented alongside a second test target of different luminance. Targets were high- and low-luminance squares flashed 1° above or below the fixation cross, just before a 5° horizontal leftward saccade. We arranged the targets vertically above and below the fixation point so that the horizontal retinal eccentricity remained the same for the upper and lower test flash. This ensured that differences in PSDs would mainly be influenced by the luminance level. Based on our previous studies, we expected that the low-luminance square would generate larger magnitudes of PSD than the high-luminance square when presented on its own. 

Figure 2 presents data for the single-target condition, which establishes a baseline for the effect of luminance of the individual flashed targets on their resultant PSD. The data in this condition are pooled from the two elevation conditions and are plotted for 5 individual observers (and averaged for all 5 subjects) as square symbols connected with dashed lines (each column represents one observer). The upper and lower rows of Figure 2 show the magnitude of mislocalization (PSD) in the direction of the saccade and orthogonal to the saccade, respectively. Because the stimuli are presented at (0°) the original fixation point, Figure 2 may also be read as a plot of perceived location for the target stimuli. For the horizontal component, positive displacements indicate that the flashed targets appeared to the left of their physical positions (or closer to the saccade target), and negative values indicate that the flashed targets appeared the right side of their physical positions (or further away from the saccade target). The estimated horizontal components of PSDs are presented relative to the fixation point because the test and fixation stimuli were presented at the same azimuth. The vertical component of PSD is a measure of the vertical separation between the observers' setting from the corresponding test target. Positive displacements indicate the flashed target appeared at a greater vertical retinal eccentricity than the test flash, and negative values indicate that the flashed target appeared at a lesser retinal eccentricity than the test flash (or closer than 1° to the fixation point). 

Figure 2

 

Averaged PSDs for single and paired flashes at different luminance are plotted as a function of TSO. Results are shown for all 5 observers as well as the averaged data for all 5. The PSD component in the direction of the saccade (horizontal component) is shown in the upper panels. The separation between the dotted lines (single-flash condition) indicates a large difference in PSD for a high- and low-luminance flash presented individually. With a paired presentation of these stimuli, the solid lines in the upper panel virtually superimpose,showing no difference in PSD for the high- and low-luminance stimuli, illustrating a possible effect of grouping. In both cases, PSD varies as a function of time to saccade onset (TSO), but when stimuli are presented simultaneously, they exhibit a uniform PSD. As a control, PSDs were also obtained under no saccade (steady fixation condition) as a control for all four conditions. The vertical component of PSD is plotted in the bottom panels, and the scale shows a negligible PSD magnitude orthogonal to the saccade. The four individual points at the left of each graph are measurements taken in a no-saccade condition and illustrate that objects were localized fairly veridically in the horizontal meridian in the absence of a saccade and that the vertical component of PSD due to this bias was negligible. Points with error bars represent the means of binned data (10-ms bins), along with their standard errors.

The results shown in the top panel of Figure 2 for the horizontal component of PSD clearly show a large influence of luminance on PSD. PSDs for the low-luminance condition are as much as 3° greater than for the high-luminance condition. Paired t-test analysis of the normalized PSDs at accepted TSOs indicated that the PSDs at low luminance are significantly higher than that at high luminance for each subject (Observer 1: t = 10.01, p = 0.0001; Observer 2: t = 6.52, p = 0.001; Observer 3: t = 5.62, p = 0.002; Observer 4: t = 6.66, p = 0.001; Observer 5: t = 4.76, p = 0.005). These PSD magnitudes are best represented separately as a function of presentation time rather than averaged together because their magnitudes are highly dependent on TSO. Consistent with previous studies (Georg et al., 2008; Zhang et al., 2008), the horizontal component of PSDs for low-luminance test flashes increases with TSO, reaching a maximum at TSOs of ∼−25 ms and then decreasing, whereas the PSDs of high-luminance stimuli increase as TSO approaches 0. We quantified similarity in the shape or the profile of the PSD plots by the computing the cross-covariance between the data for the two luminance conditions in this single-target condition. The correlation coefficient for no relative time shift (at 0 time lag) is −0.16 for Observer 1, 0.91 for Observer 2, 0.87 for Observer 3, 0.81 for Observer 4, and 0.97 for Observer 5. 

In the vertical meridian (orthogonal to the saccade), the averaged normalized PSDs across the accepted TSOs for all 5 observers are: −0.01° ± 0.03° for the low-luminance target and 0.01° ± 0.02° for the high-luminance target. These PSD magnitudes are negligible; they are smaller than the minimum increment adjustment level of the cursor (0.1°). Unlike the horizontal component of PSDs, there is no clear pattern of change of the vertical component of PSDs with TSOs. In addition, the sign of PSDs of the vertical component varies between observers. These small and variable vertical components of PSDs suggest little effect from a horizontal saccade on the perisaccadic shift in the vertical meridian in the current experimental design. The absolute magnitudes of (and the comparison between) vertical components of PSD in the single- and paired-target conditions do not show a significant grouping effect. Normalized vertical PSD remains constant with a magnitude less than 0.1°, less than the adjustment step size in our method of adjustment paradigm. 

If PSD was independent of stimulus context, PSDs for paired targets would mirror their single-target distortions, so target pairs of unequal luminance, presented simultaneously at the same horizontal retinal eccentricity, should be perceived with a significant horizontal separation. However, the paired-target data in Figure 2 (diamond symbols connected with solid lines) illustrate a significant deviation from that prediction. Both targets undergo the same distortion, and consequently, they are still perceived in vertical alignment. The normalized horizontal component of PSDs in the paired condition (across all five observers) is 1.2° ± 0.4° for both low- and high-luminance targets. 

For Observer 1, whose results were idiosyncratic throughout the experiment, the horizontal component for PSDs with the paired targets fell between the PSDs of high- and low-luminance levels of single targets and largely followed the pattern seen with the low-luminance single target. For the remaining observers, the horizontal component for PSDs of paired targets overlaps with the PSDs of the high-luminance single target, suggesting the dominance of the high-luminance target in the paired condition. The PSDs for the paired high- and low-luminance flashes shown by the solid lines in the top panel of Figure 2 are superimposed compared with a 1.6° separation for the single-target condition (dashed curves). As expected from their similarity, the correlation coefficient of variation of PSD with TSO is very high (0.99 at 0 time lag for all five observers). 

We considered whether our observers might have used different strategies to perform the task in the single versus paired condition. The observers could have set the probes in our experiment by localizing the first target and then setting the second target on the basis of a correctly perceived separation between the two targets. Reeve, Clark, and O'Regan, (2008) showed that the relative spatial configuration of target elements may be available to the observer independently of any mislocalizations induced by the saccade. Reeve et al. presented pairs of vertical lines during a horizontal saccade, and on each trial, they asked observers to do one of three tasks: judge the separation of the lines or set the perceived location of the left or right line of the target pair. The results showed that separation judgments were much less susceptible to compression effects than would be predicted by the judged locations of the individual lines. 

In Experiment 1, we randomized which target was set first (sometimes it was the high- and sometimes the low-luminance probe). Thus, if the location judgments were made based on the perceived location of the first target set by the observer, then half the time both targets would be localized based on the perceived location of the dim target, and half the time both would be localized to the perceived location of the bright target. In that case, the combined data for a set of paired targets would lie between the single-target setting for the dim and the bright stimulus, and the data would show a lot more variability than the single-target setting. However, such a pattern is not present in our data. In four of five observers, the paired data overlaps the bright single-target prediction, and in the remaining observer, it is in between the two single-target predictions, closer to the dim target prediction. Most important, none of the paired data show significantly more variability than the single-target data. 

Experiment 2: Grouping effect of paired targets with unequal azimuth

In Experiment 1, the vertical separation between our two targets was orthogonal to the direction of the saccade: Both flashed squares occupied the same horizontal position (zero-azimuth) relative to the fixation point. We found that paired targets of unequal luminance underwent an almost identical distortion, which is indeed consistent with a grouping mechanism that perceptually binds two targets together before they undergo perisaccadic distortion. However, the results of Experiment 1 would also be consistent with a remapping of visual space. In other words, if objects flashed perisaccadically underwent a distortion that was entirely determined by a remapping of visual space, stimulus properties of the single or paired targets could influence the remapping itself (producing a different remapping, and thus PSD, in a high- and low-luminance condition), but in a given trial, objects at the same azimuth would exhibit equal magnitudes of PSD (as we saw in the paired condition). 

To differentiate between grouping of the target pair and stimulus-driven changes in a remapping of visual space, Experiment 2 tested target pairs at different eccentricity relative to the fixation point. Both target elements were high luminance (10 cd/m²) squares flashed at positions 1° above and either 1° to the left or 1° to the right of the fixation point. Therefore, the only difference between the two flashed target elements was their horizontal retinal location. We chose this arrangement because we expected that differences in azimuth would be sufficient to produce different magnitudes of PSD for individual targets. 

As in Experiment 1, we confirmed this expectation and established a baseline for PSD in the single-target condition. In a single-target condition, target squares were flashed individually, either to the left or right of fixation. The upper row of Figure 3 shows the horizontal component of PSD for all 3 observers. As expected, the dotted lines (single-flash results) show a clear influence of flash position on PSD. There is also a large intersubject difference in the horizontal component of PSDs for targets to the left and right of the fixation cross. For Observer 1, the averaged normalized PSDs across TSOs are 2.6° ± 0.3° for the left target and 1.3° ± 0.2° for the right target. For Observer 2, the averaged results are 0.8° ± 0.1° for the left target and 1.4° ± 0.1° for the right target. For Observer 3, the averaged results are 0.7° ± 0.2° and 1.4° ± 0.1° for the left and right targets. For Observer 4, the averaged results are −0.09° ± 0.04° for the left target and 0.79° ± 0.17° for the right target. For Observer 5, the averaged results are 1.56° ± 0.11° for the left target and 1.94° ± 0.34° for the right target, respectively. For Observer 1, a paired t test indicates significantly larger PSDs for the left side target as opposed to those for right side target (t = 4.67, p = 0.005), but this pattern is idiosyncratic and is reversed (consistent with the averaged data) for three of the remaining four subjects: Observer 2 (t = 6.86, p = 0.001), Observer 3 (t = 17.04, p < 0.001), and Observer 4 (t = 6.62, p = 0.001). In Experiment 2, averaged normalized vertical components of PSDs (bottom row of the figure) were again smaller than 0.1° for both left and right targets, illustrating effectively veridical localization of these targets in the direction orthogonal to the saccade. 

Figure 3

 

The PSD for single and paired flashes presented to the left and right of the fixation point is plotted as a function of TSO for all 5 observers as well as the averaged data. Top panels show the horizontal component of PSDs. Although not as pronounced as in Figure 2, the separation between the dashed lines illustrates differences in PSD magnitude for single targets caused by differences in their location relative to the fixation and saccade endpoints. Consistent with perisaccadic compression, the PSD magnitude for the rightmost target (red lines) tends to be greater than that for the leftmost target. For targets presented simultaneously (solid lines), the measured PSDs are moresimilar. Although the solid lines do not superimpose as they did in the previous experiment, the shape of the profile of the PSD plots in the paired targets condition is basically the same. The bottom panels show vertical component of PSDs, and the four individual points at the left of each graph represent position biases recorded in a no-saccade condition. Points with error bars represent the means of binned data (10-ms bins), along with their standard errors.

When the target pair was presented in unison, so that squares were flashed simultaneously to the left and right of the fixation point, the resultant mislocalizations in the direction of the saccade differed significantly from the single-target condition. Paired targets are presented in Figure 3 as symbols connected with solid lines; position-dependent differences in PSD magnitude were reduced. Although PSDs for paired targets were not as similar to one another as we observed in Experiment 1 (the solid lines in Figure 3 are not completely superimposed the way they were in Figure 2), we note that the profiles of the solid line plots for the left and right target are almost identical in how they vary as a function of TSO. As in Experiment 1, we compute a maximal correlation coefficient of 0.99 at 0 time lag for four observers (and Observer 4's correlation coefficient is 0.96). Overall, the normalized and averaged horizontal components of PSD in the paired condition, across all five observers, are 1.5° ± 0.2° and 1.7° ± 0.2° for the left and right targets, respectively. 

As in Experiment 1, the vertical components of PSD do not appear to be particularly significant in Experiment 2. There exist large intersubject differences in how the vertical PSD changes with TSO. Even though the downward PSD pattern of Observer 3 is consistent with a possible compression effect in which objects shift toward the saccade endpoint, such patterns do not appear in the data for the other observers. 

Compression and grouping

Compression in perisaccadic distortions is generally described as a nonuniform remapping of visual space (Lappe et al., 2000; Ross et al., 1997). The PSD of an object undergoing a compression is determined primarily by its location in the visual field around the time of the saccade. The perceived locations of points in different parts of the visual field gravitate toward the saccade endpoint, which can stretch or compress the space between the points, and the effect may distort perceived spatial relationships between simultaneously presented objects. A grouping mechanism, such as the one we have posited, could act to mitigate that distortion by preserving the relative spatial configuration of an object. 

The current results indicate that, consistent with PSD patterns described as compression, the PSD of the right target (farthest from the saccade endpoint) is larger than the PSD of the left target. We can further illustrate these patterns by reconsidering the data from Experiment 2 and focusing specifically on the perceived compression of visual space. The top row of Figure 3 is replotted in Figure 4 in terms of perceived horizontal location in screen coordinates (as opposed to PSD) as a function of TSO. The two horizontal dashed lines at +1 and −1 indicate the physical location of the flashed targets and serve as reference points to interpret the perceived locations resulting from PSD (note that the positions of the fixation and saccade target are at 0° and 5°). 

Figure 4

 

Comparison of apparent horizontal positions of the left and right targets in the single-target and paired-target conditions for all 5 observers as well as the averaged data (top row). A compression index quantifies the perceived compression (or expansion) of visual space measured with single as opposed to paired targets (bottom row). The blue and red dashed horizontal lines represent the actualphysical position of the left and right targets, respectively. Four single points to the left of the graph indicate the perceived location in the no-saccade condition. Points with error bars represent the means of binned data, along with their standard errors. The compression index tends to increase as single targets (dotted line) are presented closer to saccade onset, but in the paired target condition, compression is reduced and does not vary as a function of TSO.

Because the vertical axis now represents the physical cursor setting corresponding to observers' perception of target location, Figure 4 makes it easier to see the perceived relative spatial configuration between the target pair and how it changes with TSO. It is immediately evident that when targets are flashed as a pair, there is a virtually constant perceived separation between the two targets at all TSOs. The mean normalized apparent separation of these paired targets (with standard errors) was 1.40° ± 0.01° for Observer 1, 1.86° ± 0.02° for Observer 2, 1.88° ± 0.01° for Observer 3, 1.49° ± 0.02° for Observer 4, and 1.91° ± 0.02° for Observer 5. By contrast, the apparent separation between the single-target conditions varies as a function of time. 

To quantify the amount of compression for different target locations in the single and paired conditions, we computed a compression index. In the single-target experiment, this index is a hypothetical measure of the compression of space: It is what we would expect if the PSD magnitude for each target was totally independent of other targets (i.e., context free) and depended only on its own stimulus attributes (this methodology is consistent with how compression has been reported in the literature). The compression index in the paired conditions, however, is not hypothetical: It is an empirical measure of how the perceived separation between the two paired targets changes when they are presented perisaccadically. The index is computed as the ratio between the physical separation of the stimulus target pair and the normalized apparent positions of the left and right test targets (see Equation 1): 

 

An index value of 0 indicates no compression, whereas negative values indicate expansion (with a value of −1 corresponding to a doubling of perceived separation between targets) and positive values indicate compression (reaching a maximum at 1 if targets appear superimposed and no separation is perceived). 

We plotted compression indices for each observer in the bottom row of Figure 4. The data for Observer 1 was again idiosyncratic, showing a significant expansion of the separation between the left and right targets (a negative compression index). However, the single-target plots for all of the remaining observers, as well as the averaged data, indicate a variable compression of visual space (positive values of the compression index). The averaged compression index for all five observers shows that this measure is TSO dependent for single targets: Compression increases as the targets are presented closer to the saccade onset. In the paired-target condition, although the spatial separation between targets was not perceived veridically, the compression index for all observers was much closer to 0 (a value that indicates no change in perceived separation). The magnitude of the compression was also much less variable in the paired-target condition. 

Because previous studies have suggested that the maximum saccadic velocity is related to the amount of perisaccadic compression (Ostendorf, Fischer, Finke, & Ploner, 2007), we calculated the mean maximal saccade velocity and compared it with the magnitude of compression. Intersubject variation in the compression index agrees with this observation, in that Observer 2 had the smallest maximal saccadic velocity among the five observers and also the smallest amount of compression. However, the maximal saccadic velocities were similar under single- and paired-target conditions for individual observers, so they could not account for the perceptual changes between the single- and paired-target conditions. 

The data for Experiment 2 show some additional patterns. First, in the paired condition, while the perceived locations of the two targets maintain a constant separation, their locations as an ensemble (i.e., their midpoints) do not appear to follow the TSO-dependent pattern of either single target, but rather an average of the two. For Observer 1, the PSD pattern for paired targets largely follows the pattern for the left single-target condition, suggesting the target closest to the saccadic endpoint determines the pattern in the paired condition. For Observer 2, the reverse is true, and for the remaining three observers, neither of the single-target predictions dominates the percept. 

Second, although the perceived location of the paired target ensemble remains within the range defined by the single-target predictions, its magnitude is consistently greater than the single targets' ensemble predictions. Figure 3 (the averaged data illustrates this best) suggests that the paired targets undergo a greater magnitude of PSD than the single targets, and Figure 4 shows that this means the perceived location of the left target (blue data) in the paired condition moves outside the range defined by the single-target predictions. This enhancement of PSD magnitude in the paired condition appears greater when the stimulus is presented at greater TSOs (further in time from the saccade onset). 

Because the leftmost flash was always set first in every trial of Experiment 2, observers would have been better able to use the perceived separation strategy derived from Reeve et al. (2008) to perform the task, by localizing the first target and then setting the second target on the basis of a (more) correctly perceived separation between the two targets. However, if this were the case, then we would expect the single and paired data for this flash to overlap for each of the 5 observers, even if the magnitudes of the PSD were different for individual observers. Figure 4 shows that the perceived locations of single and paired data for the leftmost flash (blue dots and blue lines) do not overlap for most of our observers. In Reeve et al. (2008), perceived separation judgments remain subject to compression and depend on TSO. If observers were using a perceived separation strategy to set the second target, then the amount of grouping we found in the paired target condition should also depend on TSO. But our data (see the compression index plots in Figure 4) show significantly reduced compression, but more importantly, there is no dependence of this compression on TSO in the paired data. 

Discussion

Evidence for grouping

The literature suggests two main components in perisaccadic spatial mislocalization: one is the perisaccadic shift in which the magnitude of PSD is uniform across the visual field (Honda, 1989, 1991), and the other is the perisaccadic compression in which the amount of PSD is nonuniform across the visual field, especially near the saccadic goal (Kaiser & Lappe, 2004; Lappe et al., 2000; Ross et al., 1997). The relative spatial relationships between stimulus features may be distorted by both of these components of PSD. Compression can be characterized as a nonlinear warping of visual space, where the targets further away from the saccadic ending point generally have larger absolute distortion than those closer targets (but the relationship is reversed if the compression index is used instead of absolute value; Lavergne, Vergilino-Perez, Lappe, & Dore-Mazars, 2010). Compression therefore changes the appearance and spatial layout of a visual object, and this has been confirmed experimentally (e.g., Ross et al., 1997). By contrast, a uniform shift of the entire visual field should not affect the apparent spatial configuration of visual object. Instead, it would produce an error in localization, where the relative spatial relationships between image features would be unaffected. However, because the shift component of PSD depends on the stimulus attributes of image features, we expect different magnitudes of shift to occur for different stimulus features, and such unequal shifts would result in the perceived distortion of relative spatial configuration of visual objects. And the compression component has been shown to be stimulus dependent as well (Lappe et al., 2000). Therefore, an analysis of the perisaccadic distortion of visual space should take into account both shift and compression components of PSD. 

In most previous studies of PSD, observers localized a single, simple target flashed individually in an empty visual field. The current study examined pairs of flashes and asked whether contextual effects, in the form of perceptual grouping, might affect the magnitude of peri-saccadic spatial distortions of paired targets with different stimulus properties. The data clearly indicate that the PSD magnitudes for targets presented at unequal luminance levels or at different spatial locations become much more similar in paired presentation than that in a single-target presentation. Observers' perception of the relative spatial configuration between two flashed targets flashed simultaneously is more veridical than expected based on how the same targets are perceived when they are presented separately. This suggests that a contextual effect such as perceptual grouping could play an important role in regulating the pattern of perisaccadic spatial distortions of multiple or spatially extended targets. 

The compression index in Experiment 2 is the clearest illustration of our contextual/grouping effect. It is described as a compression index because it is based on the difference in the PSD for the left and right flashed targets, and it eliminates the magnitude of any common or uniform mislocalization (shift) for the two targets. The index measures the relative distortion (compression/expansion) of visual space, regardless of whether this distortion accompanies a small or large shift. The lower row of Figure 4 plots the compression index for targets when they are presented in pairs alongside the compression index for targets presented individually: There is a reduction in magnitude of the distortion of perceived separation between our target stimuli. This shows that the PSD measured for isolated targets does not predict the spatial distortion that will occur for targets presented in pairs: The distortion is reduced when targets are presented in the presence of other stimuli. Consequently, we expect that adding targets to a perisaccadic display will help observers more veridically perceive relative spatial relationships between those targets and thus improve the perceptual stability of stimuli. 

Such a grouping process may be responsible for shape-grouping effects reported in previous studies. For example, previous studies found weak or no perisaccadic shape distortion for texture-defined and outline-defined rectangles flashed near the saccade goal (Matsumiya & Uchikawa, 2001; Sogo & Osaka, 2005). The same authors, however, did report strong perceived shape distortion induced for a Kanizsa illusory rectangle (Sogo & Osaka, 2005), which suggests that their terminators did not group the way our flashed targets did. Multiple factors could have contributed to the failure of grouping to maintain perceptual stability in their Kaniza stimuli: larger saccade sizes (which increase compression), centering the target around the saccade endpoint (where compression is greatest), as well as using ∼10° separations between the contour-inducing terminators (which should reduce grouping). Furthermore, to the extent that Sogo and Osaka and other authors do not measure a single-target or simple stimulus baseline to compare to more complex multielement stimuli, we cannot ascertain from their results whether grouping between feature elements was absent or actually occurred and mitigated a compression effect that might have been otherwise bigger. Indeed, a later study by Sogo and Osaka (2007) flashed a low-luminance (0.5 cd/m²) outline-defined triangle near the saccade goal just before the onset of a saccade. They reported that the perceived location of the peak of a triangle had a smaller magnitude of PSD than the single vertical bar flashed at the same location, suggesting that grouping of the triangle's features had reduced the PSD for the more complex stimulus. 

In the Introduction, we identified three previous studies that made a direct comparison between mislocalization of targets in single versus paired conditions (Brenner et al., 2005; Sogo & Osaka, 2001, 2002). The basic question posed by these three studies was similar to ours, a question of grouping in the context of multiple stimuli. All three of these prior studies do conclude that at short interstimulus intervals (ISIs), paired stimuli demonstrate grouping such that the retinotopic organization of the flashes determines the percept, rather than the prediction derived from the distortions of the single targets. 

However, a number of factors complicate this evidence and make it hard to evaluate in the context of grouping interactions for perisaccadic stimuli. To generate different single-target and paired-target predictions, these studies used asynchronously presented target pairs and assumed that the magnitude of mislocalization of each target was exclusively dependent on its TSO. They then reduced asynchrony as much as possible to stimulate grouping interactions, but in this paradigm, such manipulations have the consequence of reducing the differences in the TSO-dependent single-target predictions. Moreover, it becomes hard to know if changes in the pattern of TSO dependence for paired targets are produced by the ISI (e.g., masking) or by grouping interactions, or by some combination of both. By contrast, our paradigm allows us to measure grouping in conditions that introduce the strongest possible spatial interaction between paired stimuli (by setting ISI = 0), and because the TSO is the same for the paired targets in a given presentation, we can directly compare the TSO-dependent time course of PSD for the paired and single conditions. 

In addition, the methodology of these studies was based on an assumption that compression does not occur for their perisaccadic stimuli (they pooled data over spatial locations). This makes their results impossible to reconcile with studies such as Ross et al. (1997) that do show compression of visual space and perceived interval. Because compression is a well-established phenomenon in the literature, our study bridges the gap between studies that hint at grouping in asynchronously presented target pairs and the more common case of synchronously presented targets: (a) we used simultaneous rather than asynchronous flash pairs, as Ross et al. did in their compression study; (b) we found compression in our single data; and (c) we demonstrated that the magnitude of this compression was reduced and its TSO dependence was absent for paired targets. 

Mechanisms

What kind of perceptual grouping mechanisms might give rise to our results? At what point in the visual processing stream does perceptual grouping influence the magnitude of perceived PSD? Sogo and Osaka's (2005) results for Kaniza figures showed that the grouping afforded by an illusory contour did not minimize the PSD as effectively as texture or connected edges, suggesting a grouping mechanism that happens at an earlier stage of visual processing. Comparing studies, we also see that smaller separations of target elements make for more veridical perception (less distortion). For example, our stimuli were arranged nearer to one another and grouped better than those of Sogo and Osaka. We also found that when targets were aligned vertically in Experiment 1, they underwent virtually identical mislocalizations, so that their relative spatial configuration was perceived veridically. Unlike Experiment 1, for targets distributed horizontally, there was a distortion of visual space, even in the paired presentation condition. Thus, PSD difference between vertically aligned paired targets was more effectively minimized in Experiment 1, even though the targets were less similar in terms of their luminance qualities. 

A gestalt or higher-level grouping process that extracts target features and forms groups by similarity and proximity should result in reduced binding of objects that have a different appearance (Experiment 1) instead of those that have the same appearance (Experiment 2). And although the retinal separation of the two targets is the same in Experiment 1 and 2 and thus cannot explain why the grouping effect was stronger in Experiment 1 than Experiment 2, a low-level mechanism could explain the discrepancy. We would expect that low-level mechanisms that provide the best signal for localizing stimuli in the direction of the saccade (horizontal) would be subject to an aperture problem and would therefore be selective for position of orthogonal (vertical) orientated patterns. If these mechanisms were responsible for a grouping effect, that effect should then exhibit anisotropy, with stronger grouping for stimuli separated vertically rather than horizontally. We would expect a higher-level grouping mechanism to not exhibit such anisotropy and grouping effects to depend instead on the distance between the objects. 

A computational model of perisaccadic spatial mislocalization based on several well-known early visual mechanisms (Pola, 2007; Zhang et al., 2008) can easily account for the result of Experiment 1. In this model, the temporal impulse response function (tIRF) of a target flashed before saccade onset overlaps in time with the subsequent extraretinal eye position signal generated by the saccadic eye movement and thus integrates that changing position signal into the position estimate for the flashed object. Because the tIRF itself is nonlinear and increases in duration when the luminance of a stimulus is lowered (Swanson, Ueno, Smith, & Pokorny, 1987), it will therefore recruit more of the changing extraretinal signal to produce a larger perisaccadic mislocalization for low-luminance stimuli. 

In the single-target presentation, the stimulus luminance of the target (dim or bright) would therefore affect the shape and duration of the tIRF and determine the PSD magnitude. For paired stimuli presented close together, the tIRF for each target should also be affected by the stimulus characteristics of the neighboring target. If the duration and profile of each target's tIRF is determined by the combined luminance of a dim and bright target, both targets could be processed by (nearly) identical tIRFs. On the other hand, adding our bright target alongside a dim target will significantly increase the background illumination, whereas adding a dim target to a bright background will not have as much impact on the general level of illumination (Kelly, 1971; Stromeyer & Martini, 2003). Accordingly, the tIRF for the paired presentation should be similar to the tIRF for the higher luminance single-target condition, and the paired stimuli should be mislocalized following the prediction of the high luminance. Our data reflect this pattern: In four of the five observers, the paired condition PSD magnitudes were almost identical to the (smaller) single-condition distortions of the higher luminance stimulus. 

Although our model is quite capable of explaining these aspects of our data, we do not believe this is strong evidence for our or indeed any particular computational model. The logic above can easily be extended to support other computational models of PSD: (a) The results of Experiment 1 can be explained by any model that can account for the luminance dependence of single targets and (2) because the nonlinear dynamics of the tIRF reflect changes in the temporal processing speeds at the earliest stages of the visual system, models that account for PSD as the result of temporally sensitive processing of transients could just as easily incorporate changes in temporal processing (e.g., Binda, Cicchini, Burr, & Morrone, 2009; Hamker, Zirnsak, Calow, & Lappe, 2008). 

By contrast, the results of Experiment 2, which looked at compression, are not as easy to explain on the basis of a low-level grouping mechanism. We note that the PSDs for targets in the paired condition are identical in Experiment 1, and consistent with the tIRF model, their magnitudes show dependence on TSO that mirrors that of a single high-luminance target. In Experiment 2, although PSD magnitudes for paired targets are also more similar than in the single-target condition, these measures flatten out as a function of TSO in the paired condition and do not mirror either of the single-target profiles. Likewise, in the paired condition of Experiment 2, the shift and compression components of PSD are less dependent on TSO than they are for either of the single targets alone. It would be unlikely for a single low-level mechanism to reduce TSO dependence for the paired condition without positing some sort of interaction or higher-level modifications to the model. We also noted another strong interaction in Experiment 2: In general, the paired stimuli undergo a greater magnitude of PSD than the single-target data would predict. This is the opposite of what we saw in Experiment 1. 

In Experiment 2, the rightmost target has a larger PSD magnitude than the left target in the single condition because of compression. Although the PSD for that right target does not increase very much in the paired condition, the PSD for the companion flash on the left is enhanced. It appears as if the perceived location of the left flash is pushed by the displacement of the right flash and thus moves beyond its normal single-target endpoint. We believe the grouping of the two targets might also involve some kind of grouping, pooling, or additive interaction between the displacement signaled for each target. In other words, PSD in the paired condition would increase because following a pooling of an apparent motion, displacement, or extra retinal signals over multiple targets, the signal responsible for mislocalizing the target would be more robust. 

We note that the presence of the fixation cross in the middle of the two targets creates a reference point and a possibly a prior assumption about the relative order of the stimuli. The fixation cross is presented in between the two paired targets in Experiment 2, and it reappears at the end of the trial when the observers are making their judgments, so if observers tried to preserve the veridical relative position of the rightmost flash, the fixation cross would keep the rightmost target from moving as much. Figure 4 shows that the perceived location of the rightmost flash, at the longest TSO, remains generally to the right side of the fixation cross and does not cross over. This would explain why there is less grouping at higher TSO, because the stimuli have less temporal overlap with the fixation cross (e.g., in the averaged data, the red data show a big difference in PSD at low TSOs but not at the higher ones). These observations suggest that higher-level mechanisms may contribute to preserving the relative (retinotopic) organization of perceived locations for our paired targets. 

In summary, the current results indicated that multiple targets flashed near the onset of a saccade appear to be perceptually grouped and that the magnitude of PSD of individual features depends on this grouping context. We believe the evidence points to both high- and low-level mechanisms and that this grouping plays a key role in maintaining shape constancy and perceptual stability of objects during saccadic eye movements with rich visual stimuli and in natural scenes. 

Acknowledgments

This research was supported by the National Science Foundation NSF-BCS-0715076. 

Commercial relationships: none. 

Corresponding author: Clifton M. Schor. 

Email: [e-mail]. 

Address: School of Optometry, University of California Berkeley, Berkeley, CA, USA.