Transsaccadic perception is affected by saccade landing point deviations after saccadic adaptation

Abstract

Whenever we move our eyes, some visual information obtained before a saccade is combined with the visual information obtained after a saccade. Interestingly, saccades rarely land exactly on the saccade target, which may pose a problem for transsaccadic perception as it could affect the quality of postsaccadic input. Recently, however, we showed that transsaccadic feature integration is actually unaffected by deviations of saccade landing points. Possibly, transsaccadic integration remains unaffected because the presaccadic shift of attention follows the intended saccade target and not the actual saccade landing point during regular saccades. Here, we investigated whether saccade landing point errors can in fact alter transsaccadic perception when the presaccadic shift of attention follows the saccade landing point deviation. Given that saccadic adaptation not only changes the saccade vector, but also the presaccadic shift of attention, we combined a feature report paradigm with saccadic adaptation. Observers reported the color of the saccade target, which occasionally changed slightly during a saccade to the target. This task was performed before and after saccadic adaptation. The results showed that, after adaptation, presaccadic color information became less precise and transsaccadic perception had a stronger reliance on the postsaccadic color estimate. Therefore, although previous studies have shown that transsaccadic perception is generally unaffected by saccade landing point deviations, our results reveal that this cannot be considered a general property of the visual system. When presaccadic shifts of attention follow altered saccade landing points, transsaccadic perception is affected, suggesting that transsaccadic feature perception might be dependent on visual spatial attention.

Introduction

Whenever we move our eyes, visual processing is suppressed. Still, we experience a continuous world despite these frequent disruptions of visual processing. One of the proposed mechanisms that may allow for visual continuity of feature information is transsaccadic feature integration (Herwig, 2015; Irwin, Brown, & Sun, 1988; Melcher & Colby, 2008). Transsaccadic integration refers to the combination of presaccadic and postsaccadic visual input, allowing the visual system to bridge the gap in visual processing during an eye-movement (Fabius, Fracasso, & Van der Stigchel, 2016; Ganmor, Landy, & Simoncelli, 2015; Oostwoud Wijdenes, Marshall, & Bays, 2015; Prime, Niemeier, & Crawford, 2006; Wittenberg, Bremmer, & Wachtler, 2008; Wolf & Schütz, 2015). In colloquial terms, if an object is present before and after an eye-movement, it was likely to be there during the eye-movement as well, and our visual system retroactively fills in the missing information during the eye-movement. The resulting percept across an eye-movement has been demonstrated to be a mixture of the presaccadically and postsaccadically acquired information, weighted by the strength of the presaccadic and postsaccadic information (Ganmor, Landy, & Simoncelli, 2015; Oostwoud Wijdenes, Marshall, & Bays, 2015; Wolf & Schütz, 2015). 

In a recent study, we observed that the landing point of a saccade relative to a visual target does not influence the weights of the presaccadic and postsaccadic visual input, even in situations in which the saccade endpoint lands in between the target and a closely aligned distractor (Schut, Van der Stoep, Fabius, & Van der Stigchel, 2018). Although there was extensive variance in the endpoint of the saccade across the two reported experiments, this variance did not influence transsaccadic feature integration. We explained this finding by the idea that the presaccadic deployment of visual attention is not affected by the actual execution of a saccade, but driven by the active oculomotor programs before saccade execution (Van der Stigchel & de Vries, 2015): although the actual landing position of the saccade might be altered, the presaccadic shift of attention is allocated to the intended saccade target location (Deubel & Schneider, 1996). Because transsaccadic feature integration is a mechanism that is mainly dependent upon visual spatial attention (Steward & Schütz, 2018a; Van der Stigchel & Hollingworth, 2018), it should be robust against variance in saccade landing point in situations in which the locus of the presaccadic deployment of attention is unaffected. In such cases, the weights attributed to the presaccadic and postsaccadic information should remain the same despite the fact that saccadic landing points deviate from the intended target location. 

In the current study, we set out to expand this previous observation that transsaccadic feature integration is landing point invariant by altering the saccade vector using a saccadic adaptation paradigm (Hopp & Fuchs, 2004; McLaughlin, 1967). Saccadic adaptation is particularly interesting in this respect, because previous studies have shown that saccadic adaptation does not only change the saccade vector, but also modulates the presaccadic shift of attention (Doré-Mazars & Collins, 2005; Collins & Doré-Mazars, 2006). To adapt saccades, participants are typically instructed to make saccades towards a target, which is then moved to a new position (always in the same direction) during the saccade (e.g., two degrees closer to fixation). As a result, participants adapt their saccade amplitude to correct for the mismatch between the predicted and the executed saccade error. After adaptation, participants take the visual target displacement into account and land correctly on the target despite the displacement. Importantly, the presaccadic attentional shift is coupled to the adapted landing position rather than to the intended saccade target (Doré-Mazars & Collins, 2005). If transsaccadic perception is indeed mainly dependent on visual spatial attention, saccadic adaptation should therefore lead to a robust effect on transsaccadic perception on trials where the target remains at the same location and is not displaced. 

In our paradigm, observers were tasked with reporting the color of a saccade target, which was presented shortly before and after the saccade. On a subset of trials, the color of the saccade target changed slightly during the saccade. For unadapted saccades, we expect observers to report a mixture of the presaccadic and postsaccadic color. After saccadic adaptation, we expect transsaccadic hue perception to be weighed more strongly towards the postsaccadic hue. That is, because the presaccadic shift of attention follows the adapted saccade vector (and is not allocated to the saccade target), the presaccadic color information is not attended in the same way as before saccadic adaptation. This should decrease the influence of the presaccadic color and influence transsaccadic hue perception. 

Methods

Participants

Participants (N = 32, after exclusion N = 26 because of a lack of adaptation effect in six participants, 19 Female, M_age = 24 years) completed 388 to 580 trials in a single two-hour session (participants who completed their first task quickly completed extra trials in both tasks, although the number of adaptation trials is the same for all participants [100]). Participants varied in the number of trials they could complete in a two-hour session. Rather than sending participants home early, we chose to include extra trials if a participant was done after approximately 70 minutes. The analyses chosen, i.e., hierarchical models, are suitable for a variable number of trials per participant and benefit from extra data per participant, making this a beneficial approach. All participants reported normal or corrected-to-normal vision and signed an informed consent form prior to the start of the experiments. The participants were compensated for their time with €7 an hour. The experiment was approved by the Faculty Ethics committee of Utrecht University. 

Apparatus and stimuli

The experiment was conducted with the same apparatus/stimuli as Schut and colleagues (2018). Observers supported their head with a chinrest at 70 cm distance from an Asus ROG Swift PG278Q monitor (60.1 × 34.0 cm, 2560 × 1440 pixels, 100 Hz; AsusTek Computer Inc., Taipei, Taiwan). The left eye was recorded at 1000 Hz with an EyeLink 1000 (SR Research Ltd., Kanata, Ontario, Canada), calibrated with the native nine-point calibration procedure. Eye-movement events were detected offline using the native EyeLink saccade and fixation detection algorithms. The experiment was programmed in Python 2.7.3, using the Pygaze library for eye tracking research (Dalmaijer, Mathôt, & Van der Stigchel, 2014). Colors were generated using the HSL_uv Python library (Boronine & McGinley, 2017). The data were analyzed in R using the JagsUI package (R Development Core Team, 2008). 

Each participant was randomly selected to make leftward or rightward saccades, which did not change within a participant. Sixteen participants made rightward saccades, and 16 made leftward saccades (14 and 12, respectively, after exclusion). The fixation point was positioned 8° to the left or right from the vertical meridian of the screen approximately at eye-height. The target stimuli (colored circles) were 2° in diameter (a larger stimulus than our prior study, Schut et al., 2018), presented at an eccentricity of 16° from fixation. The larger target size was chosen to make the color more visible in the periphery. In adaptation trials, the stimuli moved 6° in the opposite direction of the saccade (i.e., toward the center of the screen), when a saccade was initiated. Thus, in adaptation trials the target was positioned at 10° distance from fixation after displacement. A saccade was defined online as a recorded gaze sample leaving a 3° region of interest around the fixation cross. 

Procedure

We distinguished two types of trials: integration trials and adaptation trials. These trials occurred across three types of trial blocks (see Figure 1): integration blocks (all integration trials), adaptation blocks (all adaptation trials), and mixed blocks (2/3 adaptation, 1/3 integration trials). Integration trials (240 or 320 trials) started with a drift correction procedure, after which the trial started. A fixation point was shown for 300 to 700 ms, then a target appeared. The target was either a white annulus (Post Only condition) or a colored circle. Participants were instructed to make a saccade to the target as fast as they could. During the saccade, the target either remained on screen (Color Constant), turned into a white annulus (Pre Only) or changed hue by 30° (Color Change). The stimulus stayed on screen for 50 ms, followed by a blank period of 500 ms (we made the postsaccadic presentation time very short to somewhat equate the performance in the presaccadic and postsaccadic trials). 

Figure 1.

 

Overview of the different trial types in the four conditions. The different trial types in the current study. All trials consisted of a single eye-movement to a colored target after which the color of the target was reported. After a block of integration trials participants were adapted with adaptation trials. The target moved backwards during these trials. For illustration purposes, the presaccadic position of the target is indicated by a light grey circle. After the adaptation trials, participants completed a block of trials in which integration and adaptation trials were intermixed. Half of the participant completed the trials in this order, the other half started with the adaptation trials, followed by the mixed trials, and finishing with the integration trials. Note that the background and the annulus in the figure are presented in inverted contrast.

After the blank period a color wheel appeared at the last recorded gaze position, and participants reported the color of the target. Participants were aware that the color was sometimes absent before or after the eye-movement but were naïve to the Color Change condition. 

In two thirds of the trials, the participants did not have to report the color, but pressed space the bar to continue to the next trial. The nonresponse trials were added to make this block of trials like the Mixed trials (see below). After the integration trials, a block of adaptation trials (100 trials) was started. In these trials, participants made an eye-movement to the colored target, which jumped back by 6°. After the target was displaced, it remained on screen for 1000 ms. Participants pressed the spacebar to continue to the next trial and were instructed to make eye-movements as accurately as possible to the target. 

When the adapted trials were completed, a block of Mixed trials (240 or 320 trials, matched to the number of integration trials) was completed. These trials were the same as the integration trials, but in two thirds of the trials the participants did not have to report the color, and the target jumped back by 6°. In one third of the trials in the mixed block, the target remained stationary and the participants had to report the color (same as in the Integration block). Participants were instructed to make a fast eye-movement to the target. 

To counterbalance the conditions between participants, half of the participants followed the previously described blocked procedure (Integration, Adaptation, Mixed), the remaining half of the participants first completed the Adaptation trials, then completed the Mixed trials, then did an additional training block of integration trials where the target did not move (50 trials without the target changing position were used to eliminate adaptation; in these trials, responses were not recorded), and completed a block of Integration trials. We verified that 50 trials were enough to extinguish adaptation for all participants. Furthermore, there were no differences in performance between “adaptation-first” and ‘adaptation-last” participants. 

Data exclusion

Participants completed between 388 to 580 trials. For each participant, we tested whether their saccade landing point was different in the mixed trials as compared to the integration trials with paired Bayesian t-tests (prior density at point 0 of a Cauchy distribution centered on 0, width of 0.707). All participants that showed a BF₀₁ > 1, that is evidence in support for the null hypothesis (i.e. no saccadic adaptation), were excluded from further analysis. With these criteria, 6 of the 32 participants were excluded. The overall performance on the color report task of these excluded participants was comparable to the included participants. Of the 12971 remaining trials across all participants, 771 trials were excluded based on the point of fixation being further than 3 degrees from the fixation point when the saccade was initiated. Another 779 trials were excluded based on the recorded saccade not landing on the opposite side of the screen (these exclusions were mostly eye-blinks). After exclusion of these trials, a total of 11418 trials (88%) were used for further analysis. For the transsaccadic integration analysis, a total of 3112 trials were used (as 2 out of 3 trials were non-response trials). On average, participants contributed 117 trials to the integration analysis (min = 75 trials, max = 159 trials). 

Statistical analyses

The response given by the participant during the color report was recoded as the error between the presaccadic hue and the reported color (from −180° to 180° with 0° being perfectly accurate). In the Color change condition, the postsaccadic color had an angle of 30°. We analyzed the data using a Bayesian Hierarchical mixture model. This statistical approach provides three advantages over other methods. First, it allows us to perform Bayesian Hypothesis testing and to model complex distributions of responses. Second, it can be used even when a subject has a high rate of excluded trials (rather than needing to remove a subject from the analysis). Third, and this is crucial to our approach, the response distribution per participant can be assumed to be a mixture of a normal and a uniform distribution. Modeling the data in this way has the advantage of being able to separate targeted responses (normal distribution) and random guesses (uniform distribution; Bays, Catalao, & Husain, 2009), thus being able to dissociate how often participants are reporting the color less accurately, or not reporting the color at all. We chose this approach, as participants reported that they were randomly guessing a portion of the time. 

In this model, participants are either reporting the color (thus drawing their response from a normal distribution) or guessing randomly (thus drawing their response from a uniform distribution, where each response is equally likely). Per condition (Adaptation: Yes or No, Target: Pre, Post, Color Constant, or Color Change) we estimated three parameters. We fitted the average of a normal distribution (µ), the standard deviation of a normal distribution (σ), and the proportion of trials drawn from a uniform distribution (guess rate; γ). Although we considered fitting a Von Mises distribution instead of a normal distribution, writing the Von Mises plus uniform model code in a Bayesian framework caused many non-convergence issues, as well as greatly increasing the time to complete the fitting procedure (from several hours to several days). It should be noted, however, that we checked the fits for the Pre Only condition and observed that the fits for the two distributions were nearly identical. 

An example uniform/gaussian mixture distribution is shown in Figure 2. Because the guess rate is bounded by 0 and 1, we fitted the guess rate with a logit link function to account for ceiling effects. We report the response rate (1- γ) rather than the guess rate, because it is slightly more intuitive to interpret. We expected the average of the normal distribution to be centered around 0° in the non-change conditions (Pre Only, Post Only, and Color Constant) indicating accurate color reports. If participants are integrating presaccadic and postsaccadic information, the mean estimate should be in between the presaccadic (0°) and postsaccadic color (30°) in the Color Change condition. It was expected that participants reported the color less accurately (higher standard deviation) and guessed randomly more frequently (lower response rate) in the Pre Only condition as color perception in the periphery is generally more difficult. If where the eyes land relative to the colored target (saccade landing point deviation from the target) does not affect transsaccadic perception, we expect no difference between the adapted and unadapted trials for any of the parameters across any of the conditions. If where the eyes land does effect transsaccadic perception, we expect the color percept on color change trials to shift toward the postsaccadic hue. 

Figure 2.

 

Example of the three-parameter mixture distribution. The distributions shown vary in guess rate (mean = 0, standard deviation = 30). The dashed line shows the distribution at response rate = 0 (100% guessing). The solid thin line shows the distribution at response rate = 1, the remaining bold line at response rate = 0.5.

Bayesian hierarchical modeling also allows us to investigate hyperparameters (i.e., population parameters). The concept of a hierarchical model is to add a layer of abstraction between the population parameters and the responses, resulting in a multilevel model. For example, the estimate of hyperparameter µ is, prior to observing the data, modeled as a normal distribution, from which each participant draws their own µ_i. The participant average (µ_i) is also a normal distribution, from which a value gets drawn for each trial. Thus a response is drawn from a participant distribution, and the parameters for a participant distribution are drawn from a population distribution. This allows us to model within- and between-subject variability and removes contamination of within-subject variability with between-subject variance. We constructed JAGS code and used JagsUI through the R statistical environment to initialize and analyze the model. JAGS estimates model parameters through Markov chain Monte Carlo (MCMC) sampling (rather than minimizing squared error used in regular fitting). For the hierarchical model we set five chains, with a thinning of two samples, running 260,000 iterations of which the first 250,000 were discarded as burn-in samples (note that the number of burn-in samples does not affect the results as these are discarded for the final analysis). Our estimates of the posterior distribution are based on the 10,000 remaining samples. We report the median and the 95% credible interval of the above-mentioned hyper parameters. Because MCMC sampling allows us to approximate the distribution of the parameters, rather than a point estimate, we can do Bayesian inference on the samples, through the Savage Dickey density ratio method (Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010). For Bayesian hypothesis testing we use a weakly informative prior, a Cauchy distribution with a width of .707, centered on 0 as the prior difference between parameters (Wagenmakers et al., 2010). For the hypothesis tests, we standardized the data per participant to have a mean of 0 and a standard deviation of 1. For the remaining statistical analyses we used linear mixed models, which were fit using the lme4 and analyzed using the lmerTest package in R (Kuznetsova, Brockhoff, & Christensen, 2014). 

Results

To summarize, we set out to investigate the effects of saccade landing point deviations after saccadic adaptation on transsaccadic perception. Participants (N = 32, after exclusion N = 26 due to the lack of an adaptation effect in six participants) completed transsaccadic integration trials whereas their saccades were unadapted or adapted. 

Saccade landing position

First, we examined whether our manipulation of adapting saccades was successful. Density plots of saccade landing points for all trial types are shown in Figure 3. The X-coordinates of the landing points over the course of the Adaptation trials are shown in Figure 4. Note that, at this point in the analysis, we have already excluded participants based on the presence of an adaptation effect, so this analysis serves more as a quantification of landing distances rather than evidence for an adaptation effect. However, we also investigated whether there were differences in landing points across the stimulus conditions, because these conditions differ between, e.g., the presaccadic and postsaccadic presentation. 

Figure 3.

 

Density plots of saccade landing points across the three trial blocks. The center of the screen was at point (0,0). Saccades started at point (‒8,0), saccade targets were presented at (8,0), and in adapted trials jumped to (2,0). The solid circle indicates the saccade target, the dashed circle indicates the adapted saccade location (only behaviorally relevant in the middle and rightmost plot). Note that in the experiment, the targets were circular, not ellipsoidal, but the y-axis was scaled (×2) to make variance in vertical landing point visible for graphing purposes.

Figure 4.

 

Landing position across 100 adaptation trials in the adaptation block. Each point is one landing point of a participant. The bold line is the average landing point. The solid red line indicates the saccade target, at position (8, 0), and the dashed red line indicates the updated location after a saccade was detected, at position (2,0).

A linear mixed model, with the X-coordinate of the landing point for each individual trial as the dependent variable, and Trialblock (Unadapted vs. Adapted) and Condition (Pre Only, Post Only, Color Constant, Color Change) as fixed effects, and a random intercept per participant revealed the following. There was a statistically significant main effect of Trialblock (which makes sense because we selected participants for this difference), F(1, 9234) = 3662.4, p < 0.001), but no effect of Condition F(3, 9234) = 1.2, p = 0.29, and no interaction between Trialblock and Condition [F(3, 9234) = 0.6, p = 0.64]. Before adaptation, the average X-coordinate of the landing point in the Pre Only trials was 7.31° (SD = 1.83). In the Post Only trials this was 7.30° (SD = 1.86), in the Color Constant trials this was 7.30° (SD = 1.86), and in the Color Change trials this was 7.33° (SD = 1.88). In the Mixed (Adaptation + Integration) block, we expected the value of the X-coordinate on integration trials to be lower (i.e. shorter saccades, landing points close to the shifted target position) due to saccadic adaptation. The average X-coordinate was 5.39° (SD = 2.10) in the Pre Only, 5.35° (SD = 2.13) in the Post Only, 5.42° (SD = 2.10) in the Color Constant, and 5.48° (SD = 2.09) in the Color Change condition. 

These results indicate that, on average, saccades were landing further from the saccade target and closer to the adapted saccade target after saccadic adaptation. This adaptation effect did not differ between the different conditions (Pre Only, Post Only, Color Constant, Color Change). 

Saccade landing point and transsaccadic color integration

Mean response hyperparameters (µ)

Note that the values for the Color Change condition were recoded to represent biases towards the postsaccadic color hue, with the presaccadic color having a value of 0° and the postsaccadic color hue having a value of 30°. For the other conditions, these values represent biases on the color wheel (as only one color was presented in a given trial) with 0° being perfectly accurate. 

First, we expected that, if feature integration occurred, the mean in the Color Change condition was different from the presaccadic color (0°), and the postsaccadic color (30°). Indeed, we found evidence for a difference of the responses in the Color Change from 0° in both the unadapted, BF₁₀ = 1.06 * 1043, and adapted trials, BF₁₀ = 1.71 * 1037. We also found evidence for a difference of the responses from 30°, in the unadapted trials, BF₁₀ = 1.2 * 106, and the adapted trials BF₁₀ = 60. This indicates that participants’ responses lay in between the presaccadic and postsaccadic color in the Color Change condition and were biased towards the postsaccadic color both for the adapted and unadapted trials. 

Second, the color reports differed between the unadapted and adapted trials in the Pre Only condition (M_unadapted = −0.21°, 95% confidence interval [95%CI] = −3.71 to 2.45; M_adapted = 2.60°, 95%CI = −0.71 to 7.58; BF₁₀ = 6.97), and in the Color Change condition (M_unadapted = 25.69°, 95%CI = 24.09 to 27.29; M_adapted = 27.38°, 95%CI = 25.53 to 29.21; BF₁₀ = 4.52, see Figure 5). In the Pre Only condition (in which there was no postsaccadic color), color reports of the presaccadic color were less accurate in the adapted trials compared with the unadapted trials. In the Color Change condition, color reports in the adapted trials were more strongly biased toward the postsaccadic color than in the unadapted trials. In contrast to the Pre Only and Color Change condition, we found no evidence for differences between the unadapted and adapted trials for the Post Only (M_unadapted = −0.2°, 95%CI = −2.17 to 2.21; M_adapted = −0.71°, 95%CI = −2.36 to 1.91; BF₀₁ = 5.12), and the Color Constant trials (M_unadapted = 1.46°, 95%CI = −0.12 to 3.22; M_adapted = −0.10°, 95%CI = −1.88 to 1.67; BF₀₁ = 4.75). Thus we found a difference in the Color Change condition, congruent with our hypothesis that transsaccadic perception is affected by saccadic adaptation. In the adapted trials, participants give a response that is closer to the postsaccadic color compared with the unadapted trials. This suggests that there is less influence of the presaccadic color when saccades are adapted. We would expect these differences to be expressed in the standard deviation hyperparameters or in the response rate parameters as well. If there is less influence of the presaccadic color on transsaccadic perception, we would expect participants to either report the presaccadic stimulus less accurately or report the postsaccadic stimulus more accurately. 

Figure 5.

 

Parameter estimates across adapted and unadapted trials for the hyperparameter µ (population mean) of the Color Change condition. The bars show the average hyperparameter estimate. The error bars show the 95% credible interval on the estimates. The red dashed line indicates the postsaccadic hue. The overlaid symbol indicates the outcome of the Savage-Dickey density ratio tests where: (<) BF₁₀ between 3 and 10.

Standard deviation hyperparameters (σ)

Next, we investigated the standard deviation hyperparameters. The results of our analyses are shown in Figure 6. In the unadapted trials, a standard deviation of 21.39 was found for the Pre Only condition (95%CI = 19.09–24.03). In the Post Only condition a standard deviation of 18.72 was found (95%CI = 17.39–19.32), and in the Color Constant condition the standard deviation was 17.34 (95%CI = 15.95–18.81). In the Color Change condition, we observed a standard deviation of 16.23 (95%CI 14.94–17.90). 

Figure 6.

 

Parameter estimates across adapted and unadapted trials for the hyperparameter σ (population standard deviation). The bars show the average hyperparameter estimate per Condition and Trialblock. The error bars show the 95% credible interval on the estimates. The bars are overlaid with symbols. The symbols indicate the outcome of the Savage-Dickey density ratio tests where: (<<) BF₁₀ between 10 and 100; (<<<) BF₁₀ 100 or higher; (=) BF₀₁ between 3 and 10.

For the adapted saccade Pre Only trials, we found a standard deviation of 27.93 (95%CI = 24.89–31.11), finding strong evidence for a decrease in precision in adapted relative to unadapted trials, BF₁₀ = 152.26. For the Post Only trials, we found evidence against a different standard deviation in the adapted compared to the unadapted trials (95%CI = 14.86 to 17.76, BF₀₁ = 3.38). For the adapted Color Constant trials, we found evidence for no difference between the standard deviation of unadapted and adapted trials (95%CI = 15.21 to 19.06, BF₀₁ = 4.46). Lastly, we found evidence for a decrease in precision after adaptation in the Color Change trials (estimate = 18.58, 95%CI = 16.98 to 20.34, BF₁₀ = 11.87). 

A hallmark of optimal integration of two sources is an increase in precision (a decrease in variability) relative to the highest precision of the single sources (Ernst & Banks, 2002). As a second measure of integration, we therefore investigated whether the standard deviation was lower in the Color Constant condition than in the Post Only condition. Much like in a prior study using the same stimulus set-up (Schut et al., 2018), we find inconclusive evidence for integration in terms of standard deviation reduction for both the adapted and unadapted] trials, BF₁₀ = 1.22 and BF10 = 1.59. 

In the prior section on the population mean, we found that participants were reporting a stimulus that was closer to the postsaccadic stimulus when their eye movements were adapted. Here, we found that participants were reporting the presaccadic stimulus less accurately in those adapted trials. These two results seem congruent, where participants are reporting presaccadic stimuli less accurately and use less presaccadic information for transsaccadic hue perception. Both these findings point to a presaccadic process that seem perturbed by saccadic adaptation. 

Response rate hyperparameters (1-γ)

Next, we analyzed the response rate (1-γ), which reflects the proportion of trials that participants were reporting and not guessing (γ) the color. The results are shown in Figure 7. We found no difference in response rate for any of the conditions within the unadapted or adapted trials. In unadapted trials, in the Pre Only condition we found a response rate of 0.65 (95%CI = 0.54–0.75), in the Post condition and Color Change condition we found a response rate of 0.99 (95%CI = 0.97 to 0.99 for both), and in the Color Change condition the response rate was 0.98 (95%CI = 0.95 to 0.99). Compared with the adapted trials, in the Pre Only condition we found a response rate of 0.66 (95%CI = 0.54 to 0.76, BF₀₁ = 11.51 as compared with the unadapted trials¹), and in the Post Only and Color Constant condition we found a response rate of 0.97 (95%CI = 0.94–0.99, BF₀₁ = 17.61 and 95%CI = 0.93–0.99, BF₀₁ = 18.86, respectively), and in the Color Constant condition we found a response rate of 0.98 (95%CI = 0.96 to 0.99, BF₀₁ = 56.33). Thus there were no differences between the adapted and unadapted trials for any of the conditions. In sum, participants guessed more during Pre Only trials than on other trial types that included postsaccadic color information, indicating the less precise color perception in the periphery before saccade execution.

Figure 7.

 

Parameter estimates across adapted and unadapted trials for the hyperparameter γ (population response rate). The bars show the average hyperparameter estimate per Condition and Trialblock. The error bars show the 95% credible interval on the estimates. The bars are overlaid with symbols. The symbols indicate the outcome of the Savage-Dickey density ratio tests where: (= =), BF₀₁ between 10 and 100.

 

Discussion

In the current study, we used saccadic adaptation to investigate the relation between transsaccadic hue perception and variations in saccadic endpoints. In a previous study, we observed that transsaccadic feature integration is landing point invariant, even in situations in which the endpoint errors were quite dramatic due to the presentation of a distractor (Schut et al., 2018). Importantly, in that study, the presaccadic shift of attention was always deployed to the saccade target, even though the saccade landed remote from the target. In case of an endpoint error due to either general noise in the oculomotor system or due to the presence of multiple active oculomotor programs, the presaccadic shift of attention is still deployed to the intended saccade target (Van der Stigchel & de Vries, 2015). This was different in the current study due to the utilization of a saccadic adaptation paradigm. That is, during saccadic adaption, the presaccadic shift of attention is adapted along with the saccade vector (Collins & Doré-Mazars, 2006; Doré-Mazars & Collins, 2005). If transsaccadic perception is affected by the locus of spatial attention during presaccadic information processing, one would expect transsaccadic hue perception to be affected by saccadic adaptation. 

Our current results are in line with this hypothesis and reveal that a presaccadic hue has a decreased influence on the transsaccadic perception of a target color when the saccade endpoint is altered by saccadic adaptation. In trials in which the color of the target was changed during the saccade, the reported hue was closer to the postsaccadic hue in adaptation trials compared to non-adaptation trials. Because presaccadic attention does not shift to the saccade target location when the saccade is adapted, information acquisition at that target location will be decreased before the saccade. This results in an increase in the variability of the presaccadic color estimate and a decreased influence of presaccadically acquired color information on transsaccadic hue perception. 

The standard deviation of the color report responses was higher (and thus the reliability lower) on adapted trials than in unadapted trials when the color changed during the saccade. There are two potential explanations for this increase in color report variability. First, the variability of color reports could be increased because the presaccadic shift of attention was allocated to a location remote from the presaccadic color target, potentially affecting the quality of color information. Second, although the presaccadic shift of attention on adapted trials was modulated compared to unadapted trials, on a portion of the adapted trials attention could still have been shifted towards the saccade target before the saccade. It is therefore possible that attention shifted presaccadically towards the target location on a subset of trials, resulting in more variability in the responses on adapted compared to unadapted trials. This line of reasoning can be applied to the results of all conditions in which a color was presented before the saccade. 

Previous studies have demonstrated that the outcome of transsaccadic integration is indeed dependent on the reliability of both presaccadic and postsaccadic acquired information in line with an optimal observed model (Demeyer, De Graef, Wagemans, & Verfaillie, 2009; Ganmor, Landy, & Simoncelli, 2015; Oostwoud Wijdenes, Marshall, & Bays, 2015; Wolf & Schütz, 2015). The decreased influence on hue perception of presaccadic acquired hue information due to saccadic adaptation suggests that saccadic adaptation influences which weights are attributed to both sources of information. When the presaccadic shift of attention is shifted due to saccadic adaptation, less weight is attributed to the presaccadic visual input and therefore relatively more weight on the postsaccadic visual input. 

The notion that visual attention may be one of the crucial mechanisms driving transsaccadic perception was recently supported by Stewart and Schütz (2018a). In their study, a salient distractor was presented at variable intervals before the execution of a saccade. Due to the attention-grabbing nature of the salient distractor, the presaccadic shift of attention intended for the target was shifted toward the distractor impairing saccade motor performance and the outcome of transsaccadic integration. Due to the unpredictable presentation of the distractor in their study, attentional resources were involuntarily redirected to the distractor. The timing of the distractor in their study was different compared to our previous study in which the target and the distractor were presented simultaneously (i.e., a global effect paradigm; Schut et al., 2018). When the target and distractor are presented at the same time, it is likely that attentional resources are distributed across both the target and distractor, and not necessarily redirected away from the saccade target. This difference in the setup of the experiments might explain why we observed no disturbance of transsaccadic integration when target and distractor were presented simultaneously (Schut et al., 2018) in a global effect paradigm. Irrespective of these differences, however, both studies observed that the perceptual error was dissociated from the saccade endpoint given that the presaccadic shift of attention did not follow the saccade endpoint. This is consistent with a recent study that showed that integration also occurs at locations other than the saccade endpoint in case attention is allocated to these locations (Stewart & Schütz, 2019). 

The difference in the influence on transsaccadic perception induced by saccadic adaptation and those induced by the presentation of a distractor (Schut et al., 2018) cannot be explained by the fact that endpoint errors might be larger for adapted saccades then those evoked by a distractor. The average landing point due to saccadic adaptation in the current study was actually closer to the saccade target compared to when we presented the target and distractor simultaneously (Schut et al., 2018). Despite the finding that saccades landed closer to the target on average than in the global effect paradigm study, we still observed an effect of saccadic adaptation on transsaccadic perception. 

Importantly, we did not observe a reduction in the standard deviation in the Color Constant condition, which is a hallmark of optimal integration of two sources (Ernst & Banks, 2002). Therefore it is difficult to interpret the current results as reflecting transsaccadic “integration” or any other mechanism that underlies transsaccadic perception. The lack of observing optimal integration in terms of the standard deviation is most likely due to a mismatch in the reliability of the presaccadic and postsaccadic cues. The postsaccadic cue is presented at the fovea where color perception is much more precise than the in the presaccadic peripheral condition. Similarly to our previous study (Schut et al., 2018), we decided not to degrade the postsaccadic cue in this study for two reasons: (1) to avoid participants becoming aware of the changing color in color change trials, and (2) by not increasing or decreasing the stimulus reliability between presaccadic and postsaccadic information, we were able to investigate how differences in the reliability of color information due to peripheral and foveal stimulation interact with saccadic adaptation. 

Therefore not having tested optimal cue integration for different stimulus reliabilities limits the conclusions we can draw about whether the results reflect changes in transsaccadic integration or more generally transsaccadic perception. However, not finding smaller standard deviations does not necessarily mean that participants were not integrating the presaccadic and postsaccadic color. For instance, on average, the systematic bias in color reports in the Color Change condition was more in line with a transsaccadic integration account than with a transsaccadic cue-switching account. If participants were using a cue-switching strategy on color change trials, this should have resulted in a bimodal distribution of color reports as each trial the color report would depend either on presaccadic or postsaccadic information only. Closer inspection of the color report distributions clearly show that most individuals have a prominent single peak in the distributions, suggestive of participants reporting a weighted average of the presaccadic and postsaccadic colors. However, as can be seen in Supplementary Figures S1 and S2, some participants had bimodal distributions, indicative of a cue switching strategy. 

It is interesting to speculate why the influence of saccadic adaptation on transsaccadic perception is so different from the influence of modifications of saccade endpoint due to the presentation of a distractor. Although we now know that the driving factor is a change in the presaccadic shift of attention, this does not explain the functional relevance of this modification of the attention shift along with the saccade vector in adaptation trials. It might be that modifications of saccade endpoint due to competing distractors are more common in daily life compared to those elicited by saccadic adaptation and that stability of visual perception in the face of motor variance is therefore more crucial for modifications induced by distractors (although it should be noted that this is rather speculative). 

One may wonder why the presaccadic shift of attention does not always follow adaptation of the saccade endpoint. The saccadic adaptation paradigm that was used in the current study essentially provided feedback about the displacement of the target relative to the saccade landing point which may provide important information about the relation between the intended saccade target and the generated saccade vector. In the case of saccade endpoint error and adaption based on this error, the saccade endpoint and the presaccadic shift of attention may either adapt together or not. Whether they adapt together or not potentially leads to different outcomes: (1) Adapting the presaccadic shift of attention together with the saccade vector may facilitate postsaccadic processing of the displaced target as no shift of attention is required after the saccade to attend the postsaccadic target. (2) Only adapting the saccade vector and not the presaccadic shift of attention may facilitate presaccadic target processing at the cost of an additional attention shift that is required after the saccade to attend the postsaccadic target. Given the low reliability of presaccadic, more peripheral, information, we speculate that it is optimal to facilitate postsaccadic processing and adapt the presaccadic shift of attention together with the saccade endpoint. 

Our line of reasoning with regard to the differing effects of saccadic adaptation and distractor presentation on transsaccadic perception is heavily dependent on the idea that the presaccadic shift of attention is modulated by saccadic adaptation. Although we did not measure the locus of attention in the current paradigm, there is quite strong evidence from the literature that the deployment of attention follows the adapted/shifted saccadic endpoints (Collins & Doré-Mazars, 2006; Doré-Mazars & Collins, 2005). It should be noted, however, that in one study opposite results were found: Attention was allocated toward the goal of the adapted saccade (i.e., the location of the saccade target before it shifts to a new location during the saccade) rather than its displaced endpoint (i.e., the new location of the target after having been displaced during the saccade; (Ditterich, Eggert & Straube, 2000). According to Collins and Doré-Mazars (2006), this inconsistency is likely explained by the weak saccadic adaptation observed in the study by Ditterich and colleagues. 

Even though the evidence that the locus of attention is modulated by saccadic adaptation is quite convincing, it does not automatically follow that the effects of saccadic adaptation on transsaccadic perception are due to these modulations in attentional allocation, even if we would have measured the locus of attention. For instance, it is known that there are also perceptual changes after saccadic adaptation causing stimuli to be mislocalized in the direction of the adaptive shift (Awater, Burr, Lappe, Morrone, & Goldberg, 2005; Zimmermann & Lappe, 2009, 2010; reviewed in Zimmermann & Lappe, 2016; Kosovicheva & Bex, 2020). This mislocalization could also result in a decrease of the influence on the presaccadic color. However, given the known role of visual attention in transsaccadic perception (e.g., Stewart & Schütz, 2018a), we think our findings reflect modulations of hue perception by a change in the locus of presaccadic attention. Due to the presaccadic shift of attention, the presaccadically acquired information can be temporarily stored in visual working memory, allowing for quick transsaccadic integration after the saccade (Schut, Van der Stoep, Postma, & Van der Stigchel, 2017; Steward & Schütz, 2018b). When this presaccadic shift of attention shifts with the adapted oculomotor program, the influence of the presaccadic color is decreased. Here, we report one (perhaps less frequent) situation in which an experimental manipulation might influence transsaccadic perception and potentially the perceived stability of the visual world. The previously reported dissociation between transsaccadic perception and modifications of the saccade endpoint is therefore not a general property of the visual system (Schut et al., 2018): in case the presaccadic shift of attention simply follows the modifications of saccade endpoints (as during saccadic adaptation), transsaccadic perception is modulated. 

Acknowledgments

The authors thank E. Smans and B.C.M.P. Dobbelsteen for contributing their time to assist with the data collection as part of their master thesis. 

Supported by a VIDI Grant 452-13-008 from the Netherlands Organization for Scientific Research to SVdS. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. 

Commercial relationships: none. 

Corresponding author: Stefan Van der Stigchel. 

Email: [email protected]. 

Address: Langeveld building, Room H0.30, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands. 

Footnotes

1  Note that BF₀₁ indicates evidence in favor of no difference between conditions.