Open Access
Article  |   March 2018
Probing transsaccadic correspondence with reverse correlation
Author Affiliations
  • Eva R. M. Joosten
    Université Paris Descartes, Sorbonne Paris Cité, Paris, France
    CNRS (Laboratoire Psychologie de la Perception, UMR 8242), Paris, France
    pointer1313@gmail.com
  • Thérèse Collins
    Université Paris Descartes, Sorbonne Paris Cité, Paris, France
    CNRS (Laboratoire Psychologie de la Perception, UMR 8242), Paris, France
    collins.th@gmail.com
Journal of Vision March 2018, Vol.18, 10. doi:https://doi.org/10.1167/18.3.10
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Eva R. M. Joosten, Thérèse Collins; Probing transsaccadic correspondence with reverse correlation. Journal of Vision 2018;18(3):10. https://doi.org/10.1167/18.3.10.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The phenomenon of visual stability is classically explained by an internal forward model predicting the sensory consequences of an eye movement based on efference copy. However, this model cannot explain why some object displacements go undetected, a phenomenon that may depend on a passive prior belief that the world is stable. With reverse correlation, we investigated saccadic suppression of displacement and found that transsaccadic correspondence operates differently depending on the position of the postsaccadic visual target relative to the primary landing position; when the signal falls within the extent of primary saccadic scatter, observers are less able to remap accurately. Furthermore, we observed that the neural representations driving perceptual and saccadic decisions are similar when saccading to a target, but that the representations driving transsaccadic correspondence are different from those driving secondary saccades.

Introduction
When we explore the world around us, our retinal signals change with each saccadic eye movement. Yet we perceive the world as stable. The classic explanation for visual stability is that presaccadic positions are remapped to postsaccadic coordinates, where the mapping itself is an active process that requires information about the eye position or displacement. This information is assumed to be derived from the efference copy of the motor command that alerts sensory cortices to upcoming feedback (Helmholtz & Southall, 1962; Sperry, 1950; von Holst & Mittelstaedt, 1950). This efference copy provides input to an internal forward model that predicts the sensory consequences of the motor act. These predictions are compared with the actual sensory consequences. In this way, sensory signals from external stimuli can be distinguished from reafferent signals (Wolpert & Miall, 1996). However, the classic explanation gets into trouble if object displacement across a saccade is not detected. The phenomenon of saccadic suppression of displacement (SSD) occurs when a visual stimulus is displaced during the saccade. Depending on the observer, displacements as large as 10% to 30% of the saccade amplitude may go unseen. The retinal displacement differs in amplitude from that of the efference copy, so it should not go unnoticed (Bridgeman, Hendry, & Stark, 1975). Deubel, Bridgeman, and Schneider (2004) suggested a modification of the classic efference-copy explanation of visual stability, according to which stability requires a zero sum of retinal and internal signal, and suggested instead that a null hypothesis or a passive prior belief that the world is stable was also at play (Deubel et al., 2004; Niemeier, Crawford, & Tweed, 2003). 
Wexler and Collins (2014) showed that an irrelevant displacement of the target orthogonal to saccade direction unmasks displacements parallel to saccade direction, and therefore relieves SSD. Similarly, inserting a task-irrelevant temporal blank between the saccade and the target displacement also relieves SSD (Deubel, Schneider, & Bridgeman, 1996). These results suggest that visual stability arises from an interplay between the efference copy and the null hypothesis. When the postsaccadic target position falls within an elliptic region roughly equivalent to saccadic variability, displacements are not seen and stability is assumed. When the displacements fall outside this region, as with orthogonal steps, displacements are seen and positions are remapped (Wexler & Collins, 2014). This “window of uncertainty” is also limited in time and disappears within about 200 ms postsaccade (Deubel et al., 2004). Niemeier et al. (2003) showed that optimal transsaccadic integration can explain SSD given noisy sensory signals and an assumption that stationary objects seldom jump exactly during saccades. The a priori information that the visual world does not move just during a saccade and the sensorimotor information consisting of the retinal signal and the efference copy are integrated in the fashion of a tug-of-war in which the optimal percept is pulled away from the sensorimotor information towards the prior (Niemeier et al., 2003). 
Another often-reported perceptual phenomenon that occurs around the time of saccades is perisaccadic compression (Bischof & Kramer, 1968; Lappe, Awater, & Krekelberg, 2000; Morrone, Ross, & Burr, 1997). The perceived location of point targets flashed just before saccade onset are compressed towards the saccade landing position. Explanations of perisaccadic compression usually involve an imperfect efference copy that develops slowly, starting shortly before the saccade. Other explanations reject the involvement of an eye-movement signal, since similar compression effects have been demonstrated in a fixation condition with moving stimuli (Ostendorf, Fischer, Gaymard, & Ploner, 2006), and no compression occurs when a saccade is planned but not executed (Atsma, Maij, Corneil, & Medendorp, 2014). Whatever the case may be, perisaccadic compression is not directly comparable to SSD. Indeed, in perisaccadic compression, all of the relevant visual information needed to perform the task occurs presaccadically and in the same reference frame (retinotopic), and the flash location has to be reconstructed from memory. In SSD, the task is by design transsaccadic and must be performed in a spatiotopic reference frame. 
In this study, we apply classification-image (CI) analysis to investigate SSD. CI analysis, a form of reverse correlation—that is, cross-correlating noise input with a system's output (Abbey & Eckstein, 2002; Ahumada, 2002; Ahumada & Lovell, 1971; Murray, Bennett, & Sekuler, 2002; Neri & Levi, 2006)—is used to estimate behavioral perceptive fields of psychological mechanisms. CI analysis is not only a unique tool to visualize what stimulus features an observer is using to perform a psychophysical task; it also uncovers the internal representations of perception in noisy environments. Therefore, it provides us with richer data sets than metrics based on performance alone, such as percentage correct. A CI can best be treated as a behavioral measure, like a threshold (albeit more complex), that can be used to test the predictions of competing theories of visual processing (Murray, 2011). Although this technique is common in vision and audition, it has been rarely applied to study eye movements, and more specifically, it has never been applied to SSD. In our study, we aim to understand how retinally disparate locations of an object are matched across a saccade. This is an inherently spatial question, and so CI analysis is particularly adapted to our question. 
Panichi, Burr, Morrone, and Baldassi (2012) applied CI analysis successfully to probe the spatiotemporal structure of perceptive fields during perisaccadic remapping, providing a potential mechanism for integrating stimuli across saccades. Another study (Eckstein, Beutter, Pham, Shimozaki, & Stone, 2007) estimated behavioral perceptive fields of the visual mechanisms responsible for perceptual and saccadic responses during a visual search task. Results showed that the behavioral perceptive fields mediating the perceptual decisions are indistinguishable from those driving the oculomotor decisions, suggesting that similar neural mechanisms are responsible for both perception and oculomotor action during search. For motor control of eye movements, several studies have suggested that pathways for perception and oculomotor control might overlap (at least partially). For instance, signals encoding the direction of target motion that drive steady-state pursuit and concurrent perceptual judgments may emanate from a shared ensemble of cortical neurons (Stone & Krauzlis, 2003). And when errors in perception and pursuit on a trial-by-trial basis were compared, there was no correlation between perceptual and eye-movement errors, but oculometric and psychometric performance were similar, suggesting that the motor system and perception share the same constraints in their analysis of motion signals but act independently and have different noise sources (Gegenfurtner, Xing, Scott, & Hawken, 2003). 
The aim of our study was to visualize the entire process of perceptual and saccadic behavior, step by step, in an SSD task. In this task, observers make a saccade to a presaccadic target that is displaced during saccade execution and report the direction of displacement (left or right). More specifically, we investigated how postsaccadic CIs relate to primary-saccade endpoints. Previous work with indirect methods suggests that perceptual performance in the SSD task correlates with the extent of the primary-saccade endpoint scatter; the more the scatter, the shallower the slope of the psychometric function and consequently the lower the performance (Niemeier et al., 2003, 2007; Wexler & Collins, 2014). In the case of spatial uncertainty, it has been shown that the spatial extent of the uncertainty can be estimated from the classification subimages. Noise patterns with pixels that are negatively correlated with the target signal will likely lead to an error in the response. Hence, a classification subimage obtained from the error trials will contain a clear negative image of the observers' template of the target signal (Tjan & Nandy, 2006). 
We thus hypothesized that the spatial extent of the CIs that reveal the stimulus information used to respond in the transsaccadic task will correlate with the spatial extent of primary-saccade endpoint distributions. We use CIs to obtain a spatial representation of the window of uncertainty for transsaccadic correspondence. This spatial window is best represented by the misses, which reveal the noise elements that can drive decisions in the absence of the target signal (for a detailed discussion on CIs with uncertainty, see Tjan & Nandy, 2006). Given the observers' intrinsic uncertainty, we expect that modulations representing the misses will cover a larger area around target location and will correlate with the spatial extent of primary-saccade endpoints. Since the literature mentions on several occasions that neural representations for perception and oculomotor control largely overlap (Eckstein et al., 2007; Gegenfurtner et al., 2003; Stone & Krauzlis, 2003), we additionally investigated whether CIs are affected when the decisional input is changed from psychometric to oculometric. 
Materials and methods
Instruments and stimuli
Stimuli were 2° × 2° square targets embedded in an 80 × 64 matrix with white-noise segments of 0.5° × 0.5° spanning the entire screen. Each element of the noise matrix was sampled from a Gaussian distribution and was randomly assigned independently for each stimulus and element. Stimuli were presented on a 22-in. Formac ProNitron 22,800 screen with a resolution of 1,280 × 1,024 and a refresh rate of 90 Hz. Observers were seated 58 cm from the screen and their heads kept stable by chin and forehead rests. A fixation dot was presented at screen center. After a successful fixation within 2° of the fixation dot for 300 ms, the saccade target appeared 10° to the left or right. During saccade execution, the noise matrix refreshed and the target stepped horizontally (2°) to the left or right. Figure 1 visualizes stimulus presentation, step by step with three panels of screen excisions on the right side of fixation. 
Figure 1
 
Stimulus presentation. After a successful fixation of 300 ms (A), fixation point F was replaced with target T, 10° to the right or left of the fixation point (B). The target was a 2° × 2° white square embedded in a noise matrix with segments of 0.5° × 0.5°. Observers reported the location of the presaccadic target (left or right) by pressing a button and then saccading to the target. Saccade execution was detected when a radius of 2° (visualized by a dashed circle) around the fixation point was exceeded. During saccade execution, the noise matrix refreshed (C) and the target stepped horizontally backward or forward. Observers reported the direction of the transsaccadic target step (left or right), the screen turned gray, and observers were presented with visual feedback.
Figure 1
 
Stimulus presentation. After a successful fixation of 300 ms (A), fixation point F was replaced with target T, 10° to the right or left of the fixation point (B). The target was a 2° × 2° white square embedded in a noise matrix with segments of 0.5° × 0.5°. Observers reported the location of the presaccadic target (left or right) by pressing a button and then saccading to the target. Saccade execution was detected when a radius of 2° (visualized by a dashed circle) around the fixation point was exceeded. During saccade execution, the noise matrix refreshed (C) and the target stepped horizontally backward or forward. Observers reported the direction of the transsaccadic target step (left or right), the screen turned gray, and observers were presented with visual feedback.
Eye-movement recording and analysis
Viewing was binocular. Movements of the right eye were monitored with an EyeLink 1k (SR Research, Mississauga, Ontario, Canada) at a sampling rate of 1 kHz. At the beginning of each block, the EyeLink was calibrated and validated with the standard five-point EyeLink procedure. Eye-movement traces were analyzed off-line. Instantaneous velocity and acceleration were computed for each data sample and compared to a threshold (30°/s and 8000°/s2). Saccade onset was defined as two consecutive above-threshold samples for both criteria. Saccade offset was defined as the beginning of the next 20-ms period of below-threshold samples. 
Procedure
Each trial was composed of a presaccadic (peripheral) and a postsaccadic (foveal) task. For the presaccadic task observers fixated on the center dot. If fixation was not preserved for 300 ms, the trial was aborted. Observers indicated on which side the target appeared by pressing the corresponding arrow on the keyboard (left or right). After their response, observers were instructed to saccade towards the target. During saccade execution the target was displaced. Observers made a second decision by pressing on the corresponding arrow to report the horizontal direction of the step. The displaced target remained on screen until observers gave their response. After the response a gray screen appeared and the fixation dot changed color, providing the subject with visual feedback (green for two correct responses and red for one or two incorrect responses). The next trial was automatically initiated after a 1-s delay. At the end of each 100-trial block, observers were provided with a summary detailing the total number of collected trials and the percentage of correct pre- and postsaccadic responses averaged across all blocks collected to that point. We tested five observers (all female), all unaware of the purpose of this study except S1 (author ERMJ). All signed a consent form in agreement with the Declaration of Helsinki and received financial compensation (10 euros/hr). Observers were initially familiarized with the task by performing 100 trials, of which the first did not contain any noise. Noise intensity was then increased (following a 2-up, 1-down staircase) until performance reached ∼75% correct responses. After the noise threshold was set, participants performed the experiment in six sessions of 2 hr. 
Derivation of CIs
Each noise element can be described as Display Formula\(\def\upalpha{\unicode[Times]{x3B1}}\)\(\def\upbeta{\unicode[Times]{x3B2}}\)\(\def\upgamma{\unicode[Times]{x3B3}}\)\(\def\updelta{\unicode[Times]{x3B4}}\)\(\def\upvarepsilon{\unicode[Times]{x3B5}}\)\(\def\upzeta{\unicode[Times]{x3B6}}\)\(\def\upeta{\unicode[Times]{x3B7}}\)\(\def\uptheta{\unicode[Times]{x3B8}}\)\(\def\upiota{\unicode[Times]{x3B9}}\)\(\def\upkappa{\unicode[Times]{x3BA}}\)\(\def\uplambda{\unicode[Times]{x3BB}}\)\(\def\upmu{\unicode[Times]{x3BC}}\)\(\def\upnu{\unicode[Times]{x3BD}}\)\(\def\upxi{\unicode[Times]{x3BE}}\)\(\def\upomicron{\unicode[Times]{x3BF}}\)\(\def\uppi{\unicode[Times]{x3C0}}\)\(\def\uprho{\unicode[Times]{x3C1}}\)\(\def\upsigma{\unicode[Times]{x3C3}}\)\(\def\uptau{\unicode[Times]{x3C4}}\)\(\def\upupsilon{\unicode[Times]{x3C5}}\)\(\def\upphi{\unicode[Times]{x3C6}}\)\(\def\upchi{\unicode[Times]{x3C7}}\)\(\def\uppsy{\unicode[Times]{x3C8}}\)\(\def\upomega{\unicode[Times]{x3C9}}\)\(\def\bialpha{\boldsymbol{\alpha}}\)\(\def\bibeta{\boldsymbol{\beta}}\)\(\def\bigamma{\boldsymbol{\gamma}}\)\(\def\bidelta{\boldsymbol{\delta}}\)\(\def\bivarepsilon{\boldsymbol{\varepsilon}}\)\(\def\bizeta{\boldsymbol{\zeta}}\)\(\def\bieta{\boldsymbol{\eta}}\)\(\def\bitheta{\boldsymbol{\theta}}\)\(\def\biiota{\boldsymbol{\iota}}\)\(\def\bikappa{\boldsymbol{\kappa}}\)\(\def\bilambda{\boldsymbol{\lambda}}\)\(\def\bimu{\boldsymbol{\mu}}\)\(\def\binu{\boldsymbol{\nu}}\)\(\def\bixi{\boldsymbol{\xi}}\)\(\def\biomicron{\boldsymbol{\micron}}\)\(\def\bipi{\boldsymbol{\pi}}\)\(\def\birho{\boldsymbol{\rho}}\)\(\def\bisigma{\boldsymbol{\sigma}}\)\(\def\bitau{\boldsymbol{\tau}}\)\(\def\biupsilon{\boldsymbol{\upsilon}}\)\(\def\biphi{\boldsymbol{\phi}}\)\(\def\bichi{\boldsymbol{\chi}}\)\(\def\bipsy{\boldsymbol{\psy}}\)\(\def\biomega{\boldsymbol{\omega}}\)\(\def\bupalpha{\unicode[Times]{x1D6C2}}\)\(\def\bupbeta{\unicode[Times]{x1D6C3}}\)\(\def\bupgamma{\unicode[Times]{x1D6C4}}\)\(\def\bupdelta{\unicode[Times]{x1D6C5}}\)\(\def\bupepsilon{\unicode[Times]{x1D6C6}}\)\(\def\bupvarepsilon{\unicode[Times]{x1D6DC}}\)\(\def\bupzeta{\unicode[Times]{x1D6C7}}\)\(\def\bupeta{\unicode[Times]{x1D6C8}}\)\(\def\buptheta{\unicode[Times]{x1D6C9}}\)\(\def\bupiota{\unicode[Times]{x1D6CA}}\)\(\def\bupkappa{\unicode[Times]{x1D6CB}}\)\(\def\buplambda{\unicode[Times]{x1D6CC}}\)\(\def\bupmu{\unicode[Times]{x1D6CD}}\)\(\def\bupnu{\unicode[Times]{x1D6CE}}\)\(\def\bupxi{\unicode[Times]{x1D6CF}}\)\(\def\bupomicron{\unicode[Times]{x1D6D0}}\)\(\def\buppi{\unicode[Times]{x1D6D1}}\)\(\def\buprho{\unicode[Times]{x1D6D2}}\)\(\def\bupsigma{\unicode[Times]{x1D6D4}}\)\(\def\buptau{\unicode[Times]{x1D6D5}}\)\(\def\bupupsilon{\unicode[Times]{x1D6D6}}\)\(\def\bupphi{\unicode[Times]{x1D6D7}}\)\(\def\bupchi{\unicode[Times]{x1D6D8}}\)\(\def\buppsy{\unicode[Times]{x1D6D9}}\)\(\def\bupomega{\unicode[Times]{x1D6DA}}\)\(\def\bupvartheta{\unicode[Times]{x1D6DD}}\)\(\def\bGamma{\bf{\Gamma}}\)\(\def\bDelta{\bf{\Delta}}\)\(\def\bTheta{\bf{\Theta}}\)\(\def\bLambda{\bf{\Lambda}}\)\(\def\bXi{\bf{\Xi}}\)\(\def\bPi{\bf{\Pi}}\)\(\def\bSigma{\bf{\Sigma}}\)\(\def\bUpsilon{\bf{\Upsilon}}\)\(\def\bPhi{\bf{\Phi}}\)\(\def\bPsi{\bf{\Psi}}\)\(\def\bOmega{\bf{\Omega}}\)\(\def\iGamma{\unicode[Times]{x1D6E4}}\)\(\def\iDelta{\unicode[Times]{x1D6E5}}\)\(\def\iTheta{\unicode[Times]{x1D6E9}}\)\(\def\iLambda{\unicode[Times]{x1D6EC}}\)\(\def\iXi{\unicode[Times]{x1D6EF}}\)\(\def\iPi{\unicode[Times]{x1D6F1}}\)\(\def\iSigma{\unicode[Times]{x1D6F4}}\)\(\def\iUpsilon{\unicode[Times]{x1D6F6}}\)\(\def\iPhi{\unicode[Times]{x1D6F7}}\)\(\def\iPsi{\unicode[Times]{x1D6F9}}\)\(\def\iOmega{\unicode[Times]{x1D6FA}}\)\(\def\biGamma{\unicode[Times]{x1D71E}}\)\(\def\biDelta{\unicode[Times]{x1D71F}}\)\(\def\biTheta{\unicode[Times]{x1D723}}\)\(\def\biLambda{\unicode[Times]{x1D726}}\)\(\def\biXi{\unicode[Times]{x1D729}}\)\(\def\biPi{\unicode[Times]{x1D72B}}\)\(\def\biSigma{\unicode[Times]{x1D72E}}\)\(\def\biUpsilon{\unicode[Times]{x1D730}}\)\(\def\biPhi{\unicode[Times]{x1D731}}\)\(\def\biPsi{\unicode[Times]{x1D733}}\)\(\def\biOmega{\unicode[Times]{x1D734}}\)\({\bf{n}}_i^{[q,r]}\)(j, k), where (j, k) denote rows and columns in the noise matrix, respectively. On the presaccadic task, q = 0 for targets presented left and q = 1 right of fixation. On the postsaccadic task, q = 0 for noise elements containing a backward and q = 1 a forward target step. On every trial, i observers responded correctly (r = 1) or incorrectly (r = 0). The classification image is then computed as  
\begin{equation}{\bf{CI}}(j,k) = \langle {\bf{n}}_i^{[1,1]}(j,k)\rangle + \langle {\bf{n}}_i^{[0,0]}(j,k)\rangle - \langle {\bf{n}}_i^{[1,0]}(j,k)\rangle - \langle {\bf{n}}_i^{[0,1]}(j,k)\rangle ,\end{equation}
where Display Formula\(\langle \ \rangle \) is used to indicate the mean across trials (Ahumada, 2002). For some of our analyses, we split the trials into backward versus forward target steps and analyzed them separately. CIs for backward and forward displacements were computed respectively as  
\begin{equation}{\bf{CI}}{(j,k)^{[0]}} = \langle {\bf{n}}_i^{[0,0]}(j,k)\rangle - \langle {\bf{n}}_i^{[0,1]}(j,k)\rangle \end{equation}
and  
\begin{equation}{\bf{CI}}{(j,k)^{[1]}} = \langle {\bf{n}}_i^{[1,1]}(j,k)\rangle - \langle {\bf{n}}_i^{[1,0]}(j,k)\rangle {\rm {.}}\end{equation}
 
For certain analyses we differentiated hits and misses (by isolating the respective elements of the CI), because one of our motivations for using CIs was to obtain a spatial representation of the window of uncertainty for transsaccadic correspondence. This spatial window is best represented by the misses, which reveal the noise elements that can drive decisions in the absence of the target signal. Therefore, for comparison purposes, negative CI modulations (i.e., misses and false alarms) are multiplied by −1. CIs presented in this article do not represent the entire noise matrix as seen on screen but are focused on our region of interest, a 24 × 24 matrix centered at the target location (recall that the target was a 4 × 4 matrix). 
Scalar metrics for assessing individual CI structure
Assessment of CIs is qualitative and is based on visual inspection of aggregate data. However, we performed additional analyses that captured the relevant aspects of CI structure, and quantified each aspect using single values (scalar metrics). This made it possible to perform simple population statistics (paired t tests, Pearson's linear correlation coefficients, and bootstrapped confidence intervals) to confirm or reject specific hypotheses about the overall shape of the CIs. Our conclusions are based on individual-observer data, not on the aggregate observer. This distinction is important because there is no generally accepted procedure for generating an average from individual images for different observers (Neri & Levi, 2008). 
To access CI structure on an individual level, we used four measures. The first, target resemblance, is a metric which reflects the presence (or absence) of a positive peak at the target spectral location. We extracted an index for the presence/absence of a peak by applying template matching to the target location; as a template we used the original signal. Target resemblance therefore revealed how much a given CI resembles the original target; higher values reflect higher amplitudes at the CI's target spectral location. For any given CI we first extracted a matrix large enough to cover the target location plus surrounding noise elements. We then computed the inner product of this matrix with a matrix of equal size consisting of equal negative values everywhere except for the 16 target elements (4 × 4), which were set to positive values in such fashion that the average was zero. This operation is similar to a measurement of local convexity via a second-order differential operator (Marr, 1985). The result of the inner product is an index that equals 0 for a flat surface and is positive for peaks and negative for dips. To give an example, the target-resemblance metric will decrease if a peak in the CI is low in amplitude but due to negative values around the target location. The metric can also decrease when the peak modulation is shifted and therefore covers the target location only partially. Target resemblance can therefore be interpreted as a metric for target tuning (Joosten & Neri, 2012). 
We expect that if spatial perception gets distorted, the CIs will be affected (e.g., by modulation shifts or expansions). Therefore, we calculated the center of mass (COM) by taking the weighted mean along all rows and columns. The threshold of the weights was set to Z ≥ 2 (significant elements in units of standard deviation of the Z-score map). We denoted the COM with a yellow circle in all figures. Additionally, we investigated the distance between COM and target center on an individual level. The other measures involved inspection of the shape of the modulations in the CIs. The spatial extent of the modulations was characterized by the marginal averages quantified by their full width at half maximum (FWHM), a parameter commonly used to describe the width of a peak in a curve. It is given by the distance between points of the independent variable at which the dependent variable is equal to half of its maximum value. We also calculated an anisotropy index: the ratio between the horizontal and vertical FWHM (values of 1 represent isotropy). 
Double-pass experiment
We derived performance metrics by a series of double-pass experiments consisting of 100-trial blocks in which the same set of stimuli was presented twice and the percentage of same responses to the two sets was computed (Burgess & Colborne, 1988). Observers were not aware of any difference with respect to blocks for the main experiment. In double-pass blocks, the second half of 50 trials showed the same stimuli presented during the first half of 50 trials, but in randomly permuted order. We collected 1,600 ± 250 (presaccadic) and 1,100 ± 177 (postsaccadic) double-pass trials (mean ± SD across observers). Response biases were calculated by taking the inverse of a normal cumulative distribution function of the ratio between forward responses and the size of the respective data set. Sensitivity (in units of d′) was defined as the inverse of the normal cumulative distribution function of the individual performances (in units of percentage correct). 
Furthermore, we estimated internal noise (Burgess & Colborne, 1988; Neri, 2010), where we assumed that the internal response before the addition of external noise follows a normal distribution. Each response was added to a Gaussian noise source with standard deviation σ; this noise source differs for repeated presentations and represents internal noise. On each trial the model selects the stimulus associated with the largest response. Different Display Formula\({d^{ \ \prime} _{in}}\) and σ values correspond to different percentages of correct responses and of same responses to repeated presentations (percentage agreement). We selected the Display Formula\({d^{\ \prime} _{in}} - \sigma \) values that minimized the mean-square error between the predicted and observed values for percentage correct and percentage agreement. In addition, individual efficiency values were calculated as Display Formula\({\left( {d^{ \ \prime} _{human}}/{d^{ \ \prime} _{ideal}} \right)^2}\), where ideal sensitivity is Display Formula\({d^{\ \prime} _{ideal}} = 2\rho /\sqrt {2\eta } \). Efficiency values specify human performance relative to ideal performance—that is, a system without internal noise (Green & Swets, 1966). We emphasize that the ideal observer is not intended as a realistic representation of the human process, but rather as a theoretical construct that allows performance to be evaluated on the absolute scale of efficiency (Geisler, 2003). 
Results
Relationship between transsaccadic performance and saccades
Primary-saccade metrics
After reporting the location of the presaccadic target, subjects made a saccade towards it. Figure 2 illustrates the raw saccadic landing errors on correct trials (∼44,000 trials). Habitual saccade undershooting behavior is evident; the marginal average on top of Figure 2 and the COM (indicated by the yellow circle) are shifted to the left by about 1.5°. Indeed, the mean amplitude of the saccades was 9.2° ± 0.4°. Furthermore, FWHM for the horizontal component was wider than for the vertical component. This anisotropic distribution of saccade landing positions is another classic oculomotor characteristic, and can be expressed by the ratio between horizontal and vertical FWHM. The anisotropy index in the aggregate was 1.7 (individually 1.9 ± 0.8); the horizontal extent of saccade landing positions was 1.7 times as wide as the vertical extent. Note that, for comparison, data derived from trials presented on the left of the fixation cross were multiplied by −1. All images in this article are therefore rendered as if the fixation cross were on the left side of each panel. Data points of all eye data were binned and normalized to match the plotting conventions of the CI analysis. 
Figure 2
 
Raw primary-saccade landing positions on correct trials. All images are rendered as Z-score maps (red pixels refer to Z ≥ 2; see legend at right). Contour lines show interpolated surfaces Z ≥ 1 (black) and Z ≥ 2 (red). Center of mass is illustrated by the yellow circle. Spectral cross sections are shown above and to the right of each panel and correspond to the regions indicated by the dashed black rectangles. Cross sections display marginal averages and shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude). Full width at half maximum is visualized by solid lines under the curve, with values in degrees denoted below.
Figure 2
 
Raw primary-saccade landing positions on correct trials. All images are rendered as Z-score maps (red pixels refer to Z ≥ 2; see legend at right). Contour lines show interpolated surfaces Z ≥ 1 (black) and Z ≥ 2 (red). Center of mass is illustrated by the yellow circle. Spectral cross sections are shown above and to the right of each panel and correspond to the regions indicated by the dashed black rectangles. Cross sections display marginal averages and shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude). Full width at half maximum is visualized by solid lines under the curve, with values in degrees denoted below.
Transsaccadic performance
We hypothesized that the spatial extent of the CIs that reveal the information used to respond to the postsaccadic task would correlate with the spatial extent of primary-saccade endpoint distributions. Figure 3 presents aggregate CIs for backward (Figure 3A) and forward (Figure 3B) step directions. Each CI has two modulations: The left modulation for the backward steps and the right modulation for the forward steps correspond to hits (indicated by black dashes), whereas the opposite modulations in each CI correspond to misses (red dashes). The yellow circles indicate the COMs, which were shifted to the left by 1° for backward hits and by 0.5° for forward misses. As expected, FWHM for the horizontal component was larger for misses than for hits, but it was surprisingly larger for forward (3.6°) than backward misses (3.4°). Anisotropic distributions were 1.6° (forward misses), 1.5° (backward misses), 1.2° (forward hits), and 1° (backward hits). 
Figure 3
 
Aggregate postsaccadic classification images for backward (A) and forward (B) step directions. Plotting conventions as before. Spectral cross sections (above and to the right of each panel) correspond to the regions indicated by the dashed rectangles (black for hits and red for misses). Black and red traces display marginal averages, shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude).
Figure 3
 
Aggregate postsaccadic classification images for backward (A) and forward (B) step directions. Plotting conventions as before. Spectral cross sections (above and to the right of each panel) correspond to the regions indicated by the dashed rectangles (black for hits and red for misses). Black and red traces display marginal averages, shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude).
Even though there were no significant differences between anisotropy indices of backward and forward modulations (p = 0.49 for hits and p = 0.75 for misses; see Figure 4B and 4C, hits in gray and misses in red), on average the anisotropy indices were higher for backward (1.2 ± 0.2) than for forward hits (1.1 ± 0.1), but lower for backward (1.5 ± 0.6) than for forward misses (1.6 ± 0.4). Similarly, the distances between COM and respective target center (Figure 4D and 4E) were longer for backward (−0.9 ± 0.2) than for forward hits (0.1 ± 0.6), but shorter for backward (−0.1 ± 0.6) than for forward misses (−1 ± 0.6). Overall, backward-hit and forward-miss modulations are visibly shifted to the left; this differs significantly from the backward-miss and forward-hit modulations (p < 0.001), which do not display such shifts. Additionally, target resemblance is significantly higher (Figure 4A) for forward hits (1.6 ± 0.3) as opposed to backward hits (1.1 ± 0.3; p < 0.05). In other words, only backward modulations displayed spatial distortion, whereas forward hits were best tuned in to the target region. For misses, CI modulations were wider and less tuned in to the target location, revealing spatial uncertainty in the observers. Remarkably, this effect was significant for forward (p < 0.05) but not backward (p = 0.35) anisotropy indices, suggesting that uncertainty levels in backward hits and misses were more similar than were forward hits and misses. 
Figure 4
 
Scalar metrics of individual postsaccadic classification images. (A) Forward-against-backward template resemblance of the individual classification images. Black data points refer to hits and red to misses. Anisotropy indices and distances between center of mass and target center for backward (B and D) and forward (C and E) are displayed with a similar color scheme (gray for hits and red for misses). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 4
 
Scalar metrics of individual postsaccadic classification images. (A) Forward-against-backward template resemblance of the individual classification images. Black data points refer to hits and red to misses. Anisotropy indices and distances between center of mass and target center for backward (B and D) and forward (C and E) are displayed with a similar color scheme (gray for hits and red for misses). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Transsaccadic performance and primary-saccade metrics
We compared the width of the modulations (by taking the individual horizontal FWHMs) of the primary-saccade endpoints (cf. Figure 2) with those of the perceptual CIs (Figure 3A and 3B, respectively). Again, two measures of FWHM can be obtained for each CI, one for each modulation. The relevant modulations for this analysis were those corresponding to misses (see Methods), because misses reveal the spatial window of uncertainty. We found that the widths of backward hits (2.4 ± 0.4), forward hits (2.9 ± 0.5), and backward misses (2.8 ± 0.7) differed significantly from the width of the primary landing scatter (3.8 ± 0.7; all ps < 0.05), but the width of the forward misses did not (3.1 ± 0.6; p = 0.12). In fact, FWHM of forward misses (i.e., when the target stepped forward but the subject responded “backward”) correlated (r = 0.80; 95% CI [0.61, 0.96]) with the landing scatter of primary saccades. The correlation was not significant for backward misses (r = −0.36, 95% CI [−0.83, 0.40]), backward hits (r = −0.27, 95% CI [−0.96, 0.34]), or forward hits (r = −0.08, 95% CI [−0.99, 0.72])—that is, confidence intervals included 0. In addition, the direction of data points in Figure 5A is positive and has very low residuals from the predictive values (red line), as opposed to those in Figure 5B–5D
Figure 5
 
Relation between primary landing scatter and postsaccadic classification images. The width of primary scatter correlated with forward misses (A), forward hits (B), backward misses (C), and backward hits (D). Red lines indicate linear fits. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 5
 
Relation between primary landing scatter and postsaccadic classification images. The width of primary scatter correlated with forward misses (A), forward hits (B), backward misses (C), and backward hits (D). Red lines indicate linear fits. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
As expected, miss modulations (Figure 3) were wider than hits (observers used a larger area to search for the stimulus). In this case, observers more likely to be fooled by the experimental noise and interpreted the randomly assigned noise as the target. Surprisingly, uncertainty in the observers was larger when the “fool” signal overlapped with the error distribution of the primary landing positions; there is a relation between the spatial distribution of the primary landing positions and the CI modulations of the forward-miss step directions. This finding led us to investigate whether the increased perceptual uncertainty in forward-miss step directions is reflected by secondary (or corrective) saccades. 
Secondary-saccade metrics
Figure 6A shows landing positions of secondary saccades on correct trials (∼29,000 trials). As expected, secondary saccades clustered around the two possible displaced target locations. Most landing positions fell on the target location (indicated by the white rectangles). Distance from COM to target center was around 1° for both modulations (1.2° ± 0.6° for backward and 1.1° ± 0.4° for forward correct modulations). Figure 6F and 6G displays distance from COM to target center for separate modulations (hits in gray and misses in red). Figure 6B and 6C presents secondary saccades as a function of observers' reports (forward or backward). Note that in these panels, saccades falling on the backward or forward target can originate from any location in which Z ≥ 1 in Figure 2
Figure 6
 
Landing positions for secondary saccades. (A) Aggregated landing positions of secondary saccades. (B–E) Secondary saccades as a function of step direction (forward in C and E, backward in B and D) which were either correct (B, C) or incorrect (D, E). Plotting conventions are the same as before. (F and G) Distances from center of mass to target centers for backward (F) and forward (G) steps (hits in gray and misses in red).
Figure 6
 
Landing positions for secondary saccades. (A) Aggregated landing positions of secondary saccades. (B–E) Secondary saccades as a function of step direction (forward in C and E, backward in B and D) which were either correct (B, C) or incorrect (D, E). Plotting conventions are the same as before. (F and G) Distances from center of mass to target centers for backward (F) and forward (G) steps (hits in gray and misses in red).
Figure 6B through 6E presents secondary saccades as a function of observers' reports (forward or backward). The origin in these figures is the landing position of the primary saccade, such that left modulations correspond to backward saccades and right modulations to forward saccades. When observers responded that the target had stepped forward, they also made a saccade in that direction, regardless of whether their response was correct (Figure 6C) or incorrect (Figure 6D). When observers responded correctly that the target had stepped backward, their response was accompanied by a saccade to the backward target (Figure 6B), but when they responded incorrectly (i.e., responded “backward” to a forward step), their response was as often accompanied by a saccade to the backward target as to the forward target (Figure 6E). 
Note that even though Figure 6B through 6E might suggest that more forward than backward eye movements were made, incorrect trials represent only ∼25% of the entire data set. A brief inspection of the proportion of backward and forward eye movements was slightly but not significantly (p = 0.35) in favor of forward directions (56% ± 5%). Taken together, these observations suggest that neural representations for postsaccadic perception and motor behavior of secondary saccades might not be similar. 
Are neural representations similar for perception and action in the SSD task?
To address this question, four different data sets were considered. Two perceptual data sets included decisions based on the observers' button presses on pre- and postsaccadic tasks. Two saccadic data sets contained decisions based on the direction of the first saccade that occurred after the presaccadic decision and decisions based on the direction of the second saccade. To avoid having the postsaccadic data set contain trials unseen by the observer (i.e., looking at the wrong side of the screen), we included only those trials on which the preceding presaccadic response was correct, and the sample size therefore roughly matched ∼75% of the presaccadic data set's sample size. For the saccadic data sets, trials were also excluded when landing positions were not recorded (e.g., due to blinks). We derived CIs from saccades as a decisional input and compared those with the perceptual CIs separately for each task (pre- vs. postsaccadic). To investigate the relationship between noise sources for perceptual decisions and saccadic execution, we used data from the double-pass experiments. 
Presaccadic task
Performance metrics
On each trial, participants first reported the location of the presaccadic target (left vs. right). Performance metrics in this presaccadic task were obtained in the double-pass experiments and are illustrated in Figure 7. The number of collected double-pass trials was 1,500 ± 250 (perceptual) and 1,500 ± 278 (saccadic). For the presaccadic task, no significant differences in performance metrics were observed between perceptual and saccadic decisions (see Figure 7). 
Figure 7
 
Presaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 7
 
Presaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Perceptual (gray) and saccadic (red) bars hovered around a sensitivity of 1 (Figure 7A, 0.8 ± 0.2 and 1 ± 0.1, respectively; p = 0.15). No differences in decisional biases were observed (Figure 7B, −0.1 ± 0.1 and 0 ± 0.1; p = 0.54). Internal noise (Figure 7C) was slightly higher for perceptual (1.2 ± 0.4) than for saccadic decisions (0.9 ± 0.2), where all data points hover around 1 in units of external-noise SD, but this difference was not significant (p = 0.11). Finally, efficiency (Figure 7D) was 1.8 ± 0.6 (perceptual) and 2.4 ± 0.4 (saccadic) and did not differ between perceptual and saccadic decisions (p = 0.12). 
Aggregate presaccadic CIs
The number of collected trials was 13,200 ± 2,600 (perceptual) and 11,700 ± 2,600 (saccadic). We adjusted the external noise source so that observers performed with a sensitivity Display Formula\(d^{\ \prime} \sim 1\), which results in an area under the receiver operating curve of ∼75 (Green & Swets, 1966). Percentage correct was 79% ± 3% (perceptual) and 77% ± 3% (saccadic). Following the manual response, observers performed a saccade towards the target. Figure 8 presents the aggregate CIs for the perceptual and saccadic decisions. CIs for perceptual and saccadic responses are nearly identical. Distances from COM to target center were similar for perceptual and saccadic CIs (−0.2° ± 0.5° and 0° ± 0.6°, respectively; note that some data points are therefore overlaid in Figure 8D). Scalar metrics (Figure 8C through 8E) confirmed that COMs did not differ significantly between the perceptual and saccadic CIs (p = 0.60), just like FWHM (3.5° ± 0.8° and 3.6° ± 0.7°, p = 0.93), anisotropy (1.3 ± 0.2 and 1.3 ± 0.2, p = 0.91), and target resemblance (1.4 ± 0.05 and 1.4 ± 0.04, p = 0.90). 
Figure 8
 
Aggregate classification images and scalar metrics of the presaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼66,000 (perceptual) and ∼58,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from element with the highest amplitude to target center (D), and anisotropy index (E). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 8
 
Aggregate classification images and scalar metrics of the presaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼66,000 (perceptual) and ∼58,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from element with the highest amplitude to target center (D), and anisotropy index (E). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
The three lower panels in Figure 8 present the correlation between saccadic and perceptual decisions. Target resemblance, distance from COM to center, and anisotropy correlated strongly between saccadic and perceptual decisions (respectively, r = 0.98, 95% CI [0.96, 0.99]; r = 0.71, 95% CI [−0.25, 0.97]; and r = 0.95, 95% CI [0.89, 0.99]). The CIs differed from saccadic behavior itself, which showed undershooting and anisotropy as illustrated in Figure 2
Postsaccadic task
Postsaccadic performance metrics
Following the decision about the side of target presentation and the saccade to that target, observers performed the postsaccadic task (i.e., reported whether the target had moved to the left or right during the saccade). Performance metrics for perceptual and saccadic decisions in the postsaccadic task were obtained in the double-pass experiments. The number of collected double-pass trials was 1,000 ± 177 (perceptual) and 972 ± 232 (saccadic). 
Sensitivity was higher (p < 0.05) in the perceptual task (1.4 ± 0.2, gray bars in Figure 9A) than in the saccadic task (1.2 ± 0.1, red bars). No difference in bias was observed (p = 0.23); there was no tendency to prefer one of the responses (backward or forward). All data points in Figure 9B hover around 0 in units of external-noise standard deviation (−0.1 ± 0.1 and 0.1 ± 0.3 for perceptual and saccadic responses, respectively). Internal noise (1 ± 0.3 and 0.9 ± 0.3, Figure 9C) was similar for perceptual and saccadic decisions (p = 0.49). The efficiency of perceptual decisions was marginally higher (p = 0.07) than for saccadic decisions (4.8 ± 1.3 and 3.4 ± 0.9, Figure 9D). 
Figure 9
 
Postsaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 9
 
Postsaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Aggregate postsaccadic CIs
There was no explicit instruction to make a secondary saccade to the postsaccadic location of the target, but subjects did so on average within 287 ms on 99% of the trials. As in the presaccadic task, it is possible to compare the perceptual CIs (Figure 10A) to the saccadic CIs (Figure 10B) by deriving the scalar metrics for both modulations (backward and forward). The number of collected trials was 10,300 ± 1,900 (perceptual) and 7,500 ± 1,800 (saccadic). Percentage correct was 83% ± 3% (perceptual) and 78% ± 6% (saccadic). 
Figure 10
 
Aggregate classification images and scalar metrics of the postsaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼52,000 (perceptual) and ∼38,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from center of mass to target center (D), and anisotropy index (E). Scalar metrics for backward modulations are displayed in gray and forward in black. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 10
 
Aggregate classification images and scalar metrics of the postsaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼52,000 (perceptual) and ∼38,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from center of mass to target center (D), and anisotropy index (E). Scalar metrics for backward modulations are displayed in gray and forward in black. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
CIs for saccadic responses were noisier than the perceptual postsaccadic CI; saccadic CIs contained lower amplitude modulations, modulations with Z ≥ 1 across the board, and larger marginal errors (shaded areas around the averages). The COMs fell inside the respective target regions. In addition, the backward modulation covered many elements left of the target. Note that the perceptual CI was a summation of the CIs in Figure 3 and that the left modulations were the result of the increased uncertainty in backward-hit and forward-miss targets (indicated by red dashes in Figure 3B). Target resemblance (Figure 10C) was lower for backward than forward modulations (perceptual: 1.8 ± 0.4 and 1.9 ± 0.3; saccadic: 1.2 ± 0.6 and 1.5 ± 0.9), but this difference was not significant (p = 0.41). However, perceptual target resemblance was marginally higher than saccadic resemblance (p = 0.08). 
Distances between COM and target center (Figure 10D) were −0.9° ± 0.4° and −0.1° ± 0.7° for perceptual backward and forward CIs, respectively. For saccadic CIs these values were −1° ± 0.6° and 0.3° ± 0.9°. There were no significant differences between perceptual and saccadic distances from COM to center (Figure 10D) for backward (p = 0.79) or forward displacements (p = 0.51). Note that on backward targets, S2 and S5 display identical values in Figure 10D, and the S3 backward value is on top of the forward value of S2. Anisotropy indices (Figure 10E) for perceptual CIs were 1.3 ± 0.3 and 1.4 ± 0.3 (backward and forward), and for saccadic, 1.2 ± 0.9 and 0.9 ± 0.7. Even though the saccadic indices were more variable (ranging from 0° to 2.5°), no significant differences were found from the perceptual indices (ranging from 1° to 2°; p = 0.34). However, and in stark contrast with the presaccadic task, perceptual and saccadic decisions on forward steps did not correlate (target resemblance: r = 0.46, 95% CI [−0.26, 0.96]; distance to target: r = 0.34, 95% CI [−0.65, 0.51]; anisotropy: r = −0.11, 95% CI [−0.92, 0.62]). For backward steps, target resemblance correlated weakly (r = 0.66, 95% CI [0.36, 0.90]). No correlations were found between distance of COM to target and anisotropy of saccadic and perceptual backward modulations (r = 0.1, 95% CI [−0.83, 0.83], and r = −0.56, 95% CI [−0.88, 0.30]). 
General discussion
In order to perceive the world as stable, the brain must match pre- and postsaccadic images into a unified visual percept, a process called transsaccadic correspondence (Hayhoe, Lachter, & Feldman, 1991; Melcher & Morrone, 2003; Prime, Niemeier, & Crawford, 2006). Mechanisms that may support transsaccadic correspondence are remapping—the realignment of retinal locations across the saccade via a motor signal—and a null hypothesis that objects do not move just during a saccade. Both mechanisms are believed to be at play in saccadic suppression of displacement, in which displacements of the target during the saccade go unnoticed. Perfect efference copy should allow such displacements to be detected, and so the poor performance has often been attributed to the existence of a prior in favor of object stability during a saccade. We sought to map out the spatial extent of performance by using classification images. We wanted to determine the spatial region around the saccade target in which displacements are ignored—or, in other words, to reveal participants' internal representation or perceptive field (Neri & Levi, 2006). 
Perceptual performance in the SSD task correlates with the extent of the primary-saccade endpoint scatter (Niemeier et al., 2003, 2007; Wexler & Collins, 2014). We expected that the primary landing scatter would function as a window of uncertainty and that this window would be best represented by the CIs of the misses, which reveal stimulus features that can drive decisions in the absence of the target signal. We thus hypothesized that the spatial extent of miss modulations in the postsaccadic CIs (Figure 3) would correlate with the spatial extent of primary-saccade endpoint distributions (Figure 2). We found that this was the case for forward but not backward misses (Figure 5). In other words, the region within which noise elements could drive an erroneous backward perceptual response depended on the extent of saccadic scatter: the wider the scatter, the wider the perceptive field. 
If we overlay primary-saccade endpoint distributions and postsaccadic backward and forward target locations, the forward displacement of the target falls outside the ellipse, while backward target steps fall inside (regardless whether there is an actual target signal or the observer is fooled by the experimental noise). This leads us to propose two interpretations. First, the spatial extent of the primary landing scatter functions as a window of uncertainty; when the relevant signal is within saccadic scatter, uncertainty is high and observers are likely to respond erroneously. The transsaccadic correspondence of objects that fall within this window is thus poor. When the displacements fall outside this region, displacements are seen and positions are accurately remapped (Wexler & Collins, 2014). This interpretation is also compatible with the fact that target resemblance was high for forward steps; subjects relied on information at the real target location to make their perceptual response. Target resemblance was lower for backward steps: Subjects were more willing to rely on information outside the target area to make a perceptual response. 
Second, since the primary saccade undershoots the target, the eyes are already on or near the location of the backward target, whereas the target eccentricity for secondary saccades when the target stepped forward is larger. The undershoot bias might reflect a strategy to minimize saccadic flight time, given that saccades are often inaccurate (Harris, 1995). In addition to a movement cost, an optimal strategy can be expressed in terms of fixation costs; for larger eccentricities it pays off to make a saccade to see the target rapidly, but for very small eccentricities it pays off to drift slowly or move not at all, at the price of spending more time with poor vision (Harris & Wolpert, 2006). This might explain how the primary landing scatter operates as a window of uncertainty. When the postsaccadic target information is far from the landing position, as is most often the case for forward steps, it pays off to make a secondary saccade to obtain better visual information and tune in to the target rapidly. When the postsaccadic information is inside the scatter, subjects seem to cope with poorer vision, leading to errors. The spatial shifts in our backward modulations might be similar to the phenomenon of graded contraction (Niemeier et al., 2003, 2007). The backward modulations display a similar dimension-specific spatial distortion, namely that the perceived target is shifted in one specific direction. Despite being different perceptual phenomena obtained from different experimental paradigms, perisaccadic mislocalization describes similar spatial shifts. The direction of the shift is related to the amplitude of the saccade (Kaiser & Lappe, 2004; Morrone et al., 1997). 
The analysis of secondary-saccade endpoints (Figure 6) suggested that the signal for perceptual decisions and saccadic execution was different. For virtually all cases in which observers reported perceiving forward shifts, even in the case of uncertainty (when the perceptual report was erroneous), they also performed a saccade to the forward location. However, in cases when observers reported backward shifts incorrectly, they saccaded as often to the backward as to the forward location. The threshold for eliciting a saccade to the backward target is therefore higher than that necessary for a perceptual response—in other words, more signal is needed to evoke a backward saccade than a backward response. This is in line with the phenomenon of saccadic adaptation; although transsaccadic steps are not seen, they nevertheless cause adaptive modifications of saccade amplitude across trials (Collins, 2014). Taken together, these results point towards the following scenario: When the eyes land, the mechanisms that match postsaccadic information to (remapped) presaccadic information take into account a wide area corresponding to the individual's saccadic scatter. Even weak information in this region is likely to lead to a perceptual response—that is, the perception that information at that location is the target. This weak information, however, is not sufficient to drive a secondary saccade. Transsaccadic correspondence with information outside saccadic scatter is accurate: Only real target information drives the perceptual response, and when it does, it also drives a saccade. 
An interesting question is what kind of information is actually carried by the efference copy. Physiological investigations have revealed pathways carrying efference copy from the superior colliculus to higher brain areas such as frontal eye fields (Sommer & Wurtz, 2004) and MT (Berman & Wurtz, 2010). The superior colliculus is involved in saccade programming and is organized topographically: Amplitude is coded along a rostrocaudal axis and direction on a mediolateral axis. An efference copy emanating from the superior colliculus is thus susceptible of carrying accurate metric information about the planned saccade. The independence of responses in the in-flight displacement task from landing positions suggests that efference copy is metrically accurate in that it includes oculomotor noise (Collins, Rolfs, Deubel, & Cavanagh, 2009). Some studies have also suggested accurate metric information in the efference copy in a perisaccadic identification task (Fracasso, Kaunitz, & Melcher, 2015). However, as noted in the Introduction, other studies have suggested less accurate internal information about the saccade (Dassonville, Schlag, & Schlag-Rey, 1992). Because behavioral studies rely on a perceptual or oculomotor response, it is difficult to ascertain whether accurate (or poor) performance is due to an accurate (or poor) efference copy or whether other factors come into play to improve (or degrade) performance. 
Finally, we examined whether neural representations were comparable for perception and action. Eckstein et al. (2007) investigated a similar question with CIs in a visual search paradigm and found that behavioral perceptive fields mediating the perceptual decisions were indistinguishable from those driving the oculomotor decisions, and suggested that similar neural mechanisms were responsible for both perception and oculomotor behavior. The correlation between saccadic and perceptual CIs in our presaccadic task confirmed that finding; the signal that drives these two decisions is the same. In contrast, perceptual and saccadic decisions in the postsaccadic task (after the target had stepped during the saccade) were less similar. Even though the CI modulations of decisions based on saccades roughly matched the modulations of the perceptual CI, they suffered from a large amount of noise and were less efficient to tune in to backward and forward target steps. In a pursuit task, Gegenfurtner et al. (2003) found no correlation between perceptual errors and eye-movement errors, but perceptual and saccadic performance were similar. They suggested that perceptual and saccadic systems act independently and have different noise sources but share the same constraints in their analysis of motion signals. Our results show that the addition of noise cannot be explained by the saccadic system's intrinsic noise (intrinsic-noise estimates fell within the normal range and did not differ from the perceptual intrinsic noise) and that perceptual and saccadic performance was not similar (saccadic decisions displayed lower sensitivity values then perceptual decisions). As for primary saccades, we can compare secondary-saccade endpoint scatter to the oculometric CI. The difference for secondary saccades is not as apparent as for primary saccades. This may suggest that the intervening noise source does not occur for secondary saccades. Of course, the amplitude of secondary saccades is much smaller than for primary saccades; it may therefore also be the case that the noise source is present but proportional to the signal. 
Acknowledgments
This study was funded by an Agence Nationale de la Recherche/Deutsche Forschungsgemeinshaft (ANR-DFG) research grant to TC. The authors declare that they have no conflicts of interest to disclose. ERMJ designed and performed the experiments. ERMJ and TC analyzed the data and wrote the article. 
Commercial relationships: none. 
Corresponding author: Eva R. M. Joosten. 
Address: Laboratoire Psychologie de la Perception, Paris, France. 
References
Abbey, C. K., & Eckstein, M. P. (2002). Optimal shifted estimates of human-observer templates in two-alternative forced-choice experiments. IEEE Transactions on Medical Imaging, 21, 429–440.
Ahumada, A. J. (2002). Classification image weights and internal noise level estimation. Journal of Vision, 2 (1): 8, 121–131, https://doi.org/10.1167/2.1.8. [PubMed] [Article]
Ahumada, A. J., & Lovell, J. (1971). Stimulus features in signal detection. The Journal of the Acoustical Society of America, 49, 1751–1756.
Atsma, J., Maij, F., Corneil, B. D., & Medendorp, W. P. (2014). No perisaccadic mislocalization with abruptly cancelled saccades. The Journal of Neuroscience, 34 (16), 5497–5504.
Berman, R. A., & Wurtz, R. H. (2010). Functional identification of a pulvinar path from superior colliculus to cortical area MT. The Journal of Neuroscience, 30 (18), 6342–6354.
Bischof, N., & Kramer, E. (1968). Untersuchung und Überlegungen zur Richtungswahrnehmung bei willkürlichen sakkadischen Augenbewegungen [Investigations and considerations of directional perception during voluntary saccadic eye movements]. Psychologische Forschung, 32 (3), 185–218.
Bridgeman, B., Hendry, D., & Stark, L. (1975). Failure to detect displacement of the visual world during saccadic eye movements. Vision Research, 15 (6), 719–722.
Burgess, A. E., & Colborne, B. (1988). Visual signal detection. IV. Observer inconsistency. Journal of the Optical Society of America A, 5, 617–627.
Collins, T. (2014). Trade-off between spatiotopy and saccadic plasticity. Journal of Vision, 14 (12): 28, 1–14, https://doi.org/10.1167/14.12.28. [PubMed] [Article]
Collins, T., Rolfs, M., Deubel, H., & Cavanagh, P. (2009). Post-saccadic location judgments reveal remapping of saccade targets to non-foveal locations. Journal of Vision, 9 (5): 29, 1–9, https://doi.org/10.1167/9.5.29. [PubMed] [Article]
Dassonville, P., Schlag, J., & Schlag-Rey, M. (1992). Oculomotor localization relies on a damped representation of saccadic eye displacement in human and nonhuman primates. Visual Neuroscience, 9 (3–4), 261–269.
Deubel, H., Bridgeman, B., & Schneider, W. X. (2004). Different effects of eyelid blinks and target blanking on saccadic suppression of displacement. Perception and Psychophysics, 66 (5), 772–778.
Deubel, H., Schneider, W. X., & Bridgeman, B. (1996). Postsaccadic target blanking prevents saccadic suppression of image displacement. Vision Research, 36 (7), 985–996.
Eckstein, M. P., Beutter, B. R., Pham, B. T., Shimozaki, S. S., & Stone, L. S. (2007). Similar neural representations of the target for saccades and perception during search. The Journal of Neuroscience, 27 (6), 1266–1270.
Fracasso, A., Kaunitz, L., & Melcher, D. (2015). Saccade kinematics modulate perisaccadic perception. Journal of Vision, 15 (3): 4, 1–9, https://doi.org/10.1167/15.3.4. [PubMed] [Article]
Gegenfurtner, K. R., Xing, D., Scott, B. H., & Hawken, M. J. (2003). A comparison of pursuit eye movement and perceptual performance in speed discrimination. Journal of Vision, 3 (11): 19, 865–876, https://doi.org/10.1167/3.11.19. [PubMed] [Article]
Geisler, W. S. (2003). Ideal observer analysis. In Chalupa L. & Werner J. (Eds.), The visual neurosciences (pp. 825–837). Boston: MIT Press.
Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics. New York: John Wiley.
Harris, C. M. (1995). Does saccadic undershoot minimize saccadic flight-time? A Monte-Carlo study. Vision Research, 35 (5), 691–701.
Harris, C. M., & Wolpert, D. M. (2006). The main sequence of saccades optimizes speed-accuracy trade-off. Biological Cybernetics, 95 (1), 21–29.
Hayhoe, M., Lachter, J., & Feldman, J. (1991). Integration of form across saccadic eye movements. Perception, 20 (3), 393–402.
Helmholtz, H. v., & Southall, J. P. C. (1962). Helmholtz's treatise on physiological optics. New York: Dover Publications.
Joosten, E. R., & Neri, P. (2012). Human pitch detectors are tuned on a fine scale, but are perceptually accessed on a coarse scale. Biological Cybernetics, 106 (8–9), 465–482.
Kaiser, M., & Lappe, M. (2004). Perisaccadic mislocalization orthogonal to saccade direction. Neuron, 41 (2), 293–300.
Lappe, M., Awater, H., & Krekelberg, B. (2000). Postsaccadic visual references generate presaccadic compression of space. Nature, 403 (6772), 892–895.
Marr, D. (1985). Vision. San Francisco: Freeman.
Melcher, D., & Morrone, M. C. (2003). Spatiotopic temporal integration of visual motion across saccadic eye movements. Nature Neuroscience, 6 (8), 877–881.
Morrone, M. C., Ross, J., & Burr, D. C. (1997). Apparent position of visual targets during real and simulated saccadic eye movements. The Journal of Neuroscience, 17 (20), 7941–7953.
Murray, R. F. (2011). Classification images: A review. Journal of Vision, 11 (5): 2, 1–25, https://doi.org/10.1167/11.5.2. [PubMed] [Article]
Murray, R. F., Bennett, P. J., & Sekuler, A. B. (2002). Optimal methods for calculating classification images: Weighted sums. Journal of Vision, 2 (1): 6, 79–104, https://doi.org/10.1167/2.1.6. [PubMed] [Article]
Neri, P. (2010). How inherently noisy is human sensory processing? Psychonomic Bulletin and Review, 17 (6), 802–808.
Neri, P., & Levi, D. M. (2006). Receptive versus perceptive fields from the reverse-correlation viewpoint. Vision Research, 46, 2465–2474.
Neri, P., & Levi, D. M. (2008). Evidence for joint encoding of motion and disparity in human visual perception. Journal of Neurophysiology, 100, 3117–3133.
Niemeier, M., Crawford, J. D., & Tweed, D. B. (2003). Optimal transsaccadic integration explains distorted spatial perception. Nature, 422 (6927), 76–80.
Niemeier, M., Crawford, J. D., & Tweed, D. B. (2007). Optimal inference explains dimension-specific contractions of spatial perception. Experimental Brain Research, 179 (2), 313–323.
Ostendorf, F., Fischer, C., Gaymard, B., & Ploner, C. J. (2006). Perisaccadic mislocalization without saccadic eye movements. Neuroscience, 137 (3), 737–745.
Panichi, M., Burr, D., Morrone, M. C., & Baldassi, S. (2012). Spatiotemporal dynamics of perisaccadic remapping in humans revealed by classification images. Journal of Vision, 12 (4): 11, 1–15, https://doi.org/10.1167/12.4.11. [PubMed] [Article]
Prime, S. L., Niemeier, M., & Crawford, J. D. (2006). Transsaccadic integration of visual features in a line intersection task. Experimental Brain Research, 169 (4), 532–548.
Sommer, M. A., & Wurtz, R. H. (2004). What the brain stem tells the frontal cortex. I: Oculomotor signals sent from superior colliculus to frontal eye field via mediodorsal thalamus. Journal of Neurophysiology, 91 (3), 1381–1402.
Sperry, R. W. (1950). Neural basis of the spontaneous optokinetic response produced by visual inversion. Journal of Comparative Psychology, 43 (6), 482–489.
Stone, L. S., & Krauzlis, R. J. (2003). Shared motion signals for human perceptual decisions and oculomotor actions. Journal of Vision, 3 (11): 7, 725–736, https://doi.org/10.1167/3.11.7. [PubMed] [Article]
Tjan, B. S., & Nandy, A. S. (2006). Classification images with uncertainty. Journal of Vision, 6 (4): 8, 387–413, https://doi.org/10.1167/6.4.8. [PubMed] [Article]
von Holst, E., & Mittelstaedt, H. (1950). Das Reafferenzprinzip. Naturwissenschaften, 37, 464–476.
Wexler, M., & Collins, T. (2014). Orthogonal steps relieve saccadic suppression. Journal of Vision, 14 (2): 13, 1–9, https://doi.org/10.1167/14.2.13. [PubMed] [Article]
Wolpert, D. M., & Miall, R. C. (1996). Forward models for physiological motor control. Neural Networks, 9 (8), 1265–1279.
Figure 1
 
Stimulus presentation. After a successful fixation of 300 ms (A), fixation point F was replaced with target T, 10° to the right or left of the fixation point (B). The target was a 2° × 2° white square embedded in a noise matrix with segments of 0.5° × 0.5°. Observers reported the location of the presaccadic target (left or right) by pressing a button and then saccading to the target. Saccade execution was detected when a radius of 2° (visualized by a dashed circle) around the fixation point was exceeded. During saccade execution, the noise matrix refreshed (C) and the target stepped horizontally backward or forward. Observers reported the direction of the transsaccadic target step (left or right), the screen turned gray, and observers were presented with visual feedback.
Figure 1
 
Stimulus presentation. After a successful fixation of 300 ms (A), fixation point F was replaced with target T, 10° to the right or left of the fixation point (B). The target was a 2° × 2° white square embedded in a noise matrix with segments of 0.5° × 0.5°. Observers reported the location of the presaccadic target (left or right) by pressing a button and then saccading to the target. Saccade execution was detected when a radius of 2° (visualized by a dashed circle) around the fixation point was exceeded. During saccade execution, the noise matrix refreshed (C) and the target stepped horizontally backward or forward. Observers reported the direction of the transsaccadic target step (left or right), the screen turned gray, and observers were presented with visual feedback.
Figure 2
 
Raw primary-saccade landing positions on correct trials. All images are rendered as Z-score maps (red pixels refer to Z ≥ 2; see legend at right). Contour lines show interpolated surfaces Z ≥ 1 (black) and Z ≥ 2 (red). Center of mass is illustrated by the yellow circle. Spectral cross sections are shown above and to the right of each panel and correspond to the regions indicated by the dashed black rectangles. Cross sections display marginal averages and shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude). Full width at half maximum is visualized by solid lines under the curve, with values in degrees denoted below.
Figure 2
 
Raw primary-saccade landing positions on correct trials. All images are rendered as Z-score maps (red pixels refer to Z ≥ 2; see legend at right). Contour lines show interpolated surfaces Z ≥ 1 (black) and Z ≥ 2 (red). Center of mass is illustrated by the yellow circle. Spectral cross sections are shown above and to the right of each panel and correspond to the regions indicated by the dashed black rectangles. Cross sections display marginal averages and shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude). Full width at half maximum is visualized by solid lines under the curve, with values in degrees denoted below.
Figure 3
 
Aggregate postsaccadic classification images for backward (A) and forward (B) step directions. Plotting conventions as before. Spectral cross sections (above and to the right of each panel) correspond to the regions indicated by the dashed rectangles (black for hits and red for misses). Black and red traces display marginal averages, shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude).
Figure 3
 
Aggregate postsaccadic classification images for backward (A) and forward (B) step directions. Plotting conventions as before. Spectral cross sections (above and to the right of each panel) correspond to the regions indicated by the dashed rectangles (black for hits and red for misses). Black and red traces display marginal averages, shaded regions the standard error of the mean (±1 SEM, where the dashed line represents zero amplitude).
Figure 4
 
Scalar metrics of individual postsaccadic classification images. (A) Forward-against-backward template resemblance of the individual classification images. Black data points refer to hits and red to misses. Anisotropy indices and distances between center of mass and target center for backward (B and D) and forward (C and E) are displayed with a similar color scheme (gray for hits and red for misses). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 4
 
Scalar metrics of individual postsaccadic classification images. (A) Forward-against-backward template resemblance of the individual classification images. Black data points refer to hits and red to misses. Anisotropy indices and distances between center of mass and target center for backward (B and D) and forward (C and E) are displayed with a similar color scheme (gray for hits and red for misses). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 5
 
Relation between primary landing scatter and postsaccadic classification images. The width of primary scatter correlated with forward misses (A), forward hits (B), backward misses (C), and backward hits (D). Red lines indicate linear fits. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 5
 
Relation between primary landing scatter and postsaccadic classification images. The width of primary scatter correlated with forward misses (A), forward hits (B), backward misses (C), and backward hits (D). Red lines indicate linear fits. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 6
 
Landing positions for secondary saccades. (A) Aggregated landing positions of secondary saccades. (B–E) Secondary saccades as a function of step direction (forward in C and E, backward in B and D) which were either correct (B, C) or incorrect (D, E). Plotting conventions are the same as before. (F and G) Distances from center of mass to target centers for backward (F) and forward (G) steps (hits in gray and misses in red).
Figure 6
 
Landing positions for secondary saccades. (A) Aggregated landing positions of secondary saccades. (B–E) Secondary saccades as a function of step direction (forward in C and E, backward in B and D) which were either correct (B, C) or incorrect (D, E). Plotting conventions are the same as before. (F and G) Distances from center of mass to target centers for backward (F) and forward (G) steps (hits in gray and misses in red).
Figure 7
 
Presaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 7
 
Presaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 8
 
Aggregate classification images and scalar metrics of the presaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼66,000 (perceptual) and ∼58,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from element with the highest amplitude to target center (D), and anisotropy index (E). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 8
 
Aggregate classification images and scalar metrics of the presaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼66,000 (perceptual) and ∼58,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from element with the highest amplitude to target center (D), and anisotropy index (E). Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 9
 
Postsaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 9
 
Postsaccadic performance metrics for the perceptual (gray) and saccadic (red) decisions. Sensitivity (A) and bias (B) are plotted in units of d′. Internal noise (C) is plotted in units of external-noise standard deviation, and efficiency (D) as a ratio between d′ values from the observer and the ideal observer. Error bars plot ±1 standard error of the mean with a 95% confidence interval (smaller than symbol if not visible).
Figure 10
 
Aggregate classification images and scalar metrics of the postsaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼52,000 (perceptual) and ∼38,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from center of mass to target center (D), and anisotropy index (E). Scalar metrics for backward modulations are displayed in gray and forward in black. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
Figure 10
 
Aggregate classification images and scalar metrics of the postsaccadic task. Classification images were derived from perceptual (A) and saccadic (B) decisions, with a total of ∼52,000 (perceptual) and ∼38,000 (saccadic) trials. Conventions are as before. Scalar metrics present target resemblance (C), distance from center of mass to target center (D), and anisotropy index (E). Scalar metrics for backward modulations are displayed in gray and forward in black. Error bars plot ±1 standard error of the mean with a 95% confidence interval.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×