Individual differences reveal similarities in serial dependence effects across perceptual tasks, but not to oculomotor tasks

Abstract

Serial dependence effects have been observed across a wide range of perceptual and oculomotor tasks. This opens up the question of whether these effects observed share underlying mechanisms. Here we measured serial dependence effects in a semipredictable environment for the same group of observers across four different tasks, two perceptual (color and orientation judgments) and two oculomotor (tracking moving targets and the pupil light reflex). By leveraging individual differences, we searched for links in the magnitude of serial dependence effects across the different tasks. On the group level, we observed significant attractive serial dependence effects for all tasks, except the pupil response. The rare absence of a serial dependence effect for the reflex-like pupil light response suggests that sequential effects require cortical processing or even higher-level cognition. For the tasks with significant serial dependence effects, there was substantial and reliable variance in the magnitude of the sequential effects. We observed a significant relationship in the strength of serial dependence for the two perceptual tasks, but no relation between the perceptual tasks and oculomotor tracking. This emphasizes differences in processing between perception and oculomotor control. The lack of a correlation across all tasks indicates that it is unlikely that the relation between the individual differences in the magnitude of serial dependence is driven by more general mechanisms related to for example working memory. It suggests that there are other shared perceptual or decisional mechanisms for serial dependence effects across different low-level perceptual tasks.

Introduction

A common experimental assumption is that behavioral responses across trials are independent of each other and are only driven by the current sensory input. However, there is converging evidence demonstrating that previous trials systematically influence current responses. Previous experiences often bias current perceptual judgments, a phenomenon known as serial dependence (see (Cicchini, Mikellidou, & Burr, 2024; Pascucci et al., 2023) for recent overviews). For example, seeing a leftward-oriented grating in the previous trial will lead to a more leftward response in the current trial (Fischer & Whitney, 2014), suggesting an attractive serial dependence effect. Similar sequential effects have been reported not only across the whole perceptual hierarchy, ranging from simple orientation or color judgments to judgments of complex motion patterns or facial identity (see Manassi, Murai, & Whitney, 2023 for an overview), but also in attention and memory research (Kiyonaga, Scimeca, Bliss, & Whitney, 2017) or oculomotor control (Cont & Zimmermann, 2021; Goettker & Stewart, 2022). 

One idea to explain why these attractive serial dependence effects occur is the idea of a continuity field (Manassi & Whitney, 2024). A continuity field is a spatiotemporal integration mechanism that continuously biases perception, decisions, and memory toward previously encountered information and so helps to smooth otherwise noisy representations. This idea can be explained by taking the example of visual information: while the incoming visual information at every moment in time might be affected by noise, natural sensory input contains strong autocorrelations over time and therefore integrating previous information by attractive serial dependence can be beneficial to reduce noise and stabilize perception. A good illustration for that is that serial dependence is stronger for more stable factors (e.g., the gender of a face), and weaker for more variable factors (e.g., facial expression) (Taubert, Alais, & Burr, 2016). However, beyond the fact that serial effects can have an direct influence on visual perception (Cicchini, Mikellidou, & Burr, 2017; Manassi & Whitney, 2022), the observed serial dependence effects also have clear influences of decisional processes (Ceylan, Herzog, & Pascucci, 2021; Fritsche, Mostert, & de Lange, 2017; Houborg, Kristjánsson, Tanrıkulu, & Pascucci, 2023; Tanrikulu, Pascucci, & Kristjánsson, 2023), and also working memory can play a role (Bliss, Sun, & D'Esposito, 2017; Markov, Tiurina, & Pascucci, 2024; Mei, Chen, & Dong, 2019). 

Interestingly, although serial dependence has been observed across a range of tasks, the relation in the strength of serial dependence across tasks so far has not been investigated. Although sequential effects on the group level seem to be consistent within a given task, the weight given to past information reliably differs across individuals (Kondo, Murai, & Whitney, 2022; Zhang & Alais, 2020). The goal of the current study is to leverage these individual differences to address an interesting question: Do individuals showing stronger attractive serial dependence in one task also show stronger attractive serial dependence in another task? If these similarities across tasks exist this could point to a similar reliance on previous information, or other shared factors like the use of similar decision-making processes or constraints in working memory. 

To answer this, we tested serial dependence effects in four different tasks, including two perceptual tasks (color and orientation judgments) and two oculomotor (tracking moving targets and the pupil light reflex). To get a quick assessment of the serial dependence effect for each observer in each task, we used a counterbalanced trial sequence (Fischer & Whitney, 2014). Trials in each task were presented in pairs, and the second stimulus was always constant, so we measured serial dependence in a semi-predictable environment. By harnessing individual differences across different tasks, we show that the strength of serial dependence effects is correlated across different perceptual tasks, but not with the strength of sequential effects for oculomotor control. These results provide evidence for similarities of perceptual sequential effects and highlight differences in sensory processing between perception and oculomotor control. 

Methods

Observers

Data were collected from 40 observers (29 females, 25.4 ± 4.7 years old). Observers were students or employees of Giessen University, Germany. All observers were naïve to the study. They had normal or corrected-to-normal vision and passed the Ishihara color blindness test (Ishihara, 1997). Informed consent was obtained from each observer before data collection. All the tasks complied with the Helsinki Declaration and were approved by the local ethics committee (Giessen LEK 2020-0015). 

Setup

A 32-inch LCD monitor (Display++; Cambridge Research Systems, Ltd., Rochester, UK) with a resolution of 1920 × 1080 and a refresh rate of 120 Hz, was used. The chromaticity and luminance values (xyY) for the monitor primaries are as follows: R = [0.642, 0.34, 27.03], G = [0.30, 0.60, 111.6], and B = [0.15, 0.07, 15.67], and white = [0.29, 0.302, 155.2]. The monitor is characterized by linear gammas (R: 1.02, G: 1.04, B: 1.03). Observers were sitting comfortably in a dark room, with their chin and forehead positioned on a rest to ensure stability. Their eyes were aligned with the screen center at a viewing distance of 85 cm. Eye movements from the right eye were tracked using a desk-mounted EyeLink 1000 Plus eye tracker (SR Research, Kanata, ON, Canada) at a sampling rate of 1000 Hz. The tasks were programmed in MATLAB (MathWorks, Natick MA) using the Psychtoolbox (Kleiner, Brainard, & Pelli, 2007). A nine-point calibration procedure was conducted before each block to precisely align gaze data with the screen coordinates. 

Experimental paradigm

Four different tasks were measured for each observer (Figure 1): color judgment, orientation judgment, oculomotor tracking, and the pupil light response. The sequence of these tasks was randomized across observers. Each observer completed two sessions, with one block dedicated to each task in every session. The sessions were conducted on separate days, each lasting between 1 to 1.5 hours. All tasks shared a similar structure; in each trial, two stimuli were presented: the prior, which was presented first and differed from the probe stimulus in two opposite directions. This was followed by the probe stimulus, which remained constant. By examining the effect of the prior stimulus on the observers’ responses to the probe stimulus, we quantified the serial dependence effect. 

Figure 1.

 

Methods and behavioral data. Depiction of the different tasks for color judgments, orientation judgments, oculomotor tracking, and the pupil light response. The top row shows depictions of the paradigms: In the prior always one of two different stimuli was presented. Stimuli are adjusted for visualization. Please note that although both prior possibilities are depicted, observers in the tasks just saw one per trial. The probe stimulus was always identical and between the two prior stimuli in the relevant dimension. This design allowed us to look at responses in the probe stimulus with respect to the different priors. The average responses for the probe stimulus separated by the two different priors are shown in the bottom row.

Color judgment

Each trial consisted of a prior and a probe stimulus, following identical procedures. The stimuli were all defined within the DKL color space (Derrington, Krauskopf, & Lennie, 1984), and presented on a gray (at the isoluminant plane of the DKL space, with xyY = [0.29, 0.31, 80.56]) background. The hues of both the prior and probe stimuli were set at the isoluminant plane. For the prior stimulus, two hues were used: an azimuth of 275° (+5° difference from −[S − (L + M)] axis, close to purple, xyY = [0.26, 0.22, 80.36]) and 355° (−5° difference from [L − M] axis, close to pink, xyY = [0.33, 0.28, 80.60]), whereas the probe stimulus's hue was set to the midpoint between the two prior hues, with an azimuth of 315° (close to magenta, xyY = [0.29, 0.23,79.97]). 

The trial started with a white fixation dot (0.5 deg in diameter) displayed at the center of the screen for two seconds. Subsequently, a colored circle (4 deg in diameter) appeared in the right visual field at 6.5 deg eccentricity for 300 ms. Observers were instructed to maintain their gaze on the dot while using their peripheral vision to perceive the color of the circle. After the circle disappeared, a color mask of the same size emerged at the same location for one second to erase any potential after-effect. This color mask was a pixelized pattern, with each pixel randomly selected from a range of 0° to 360° hue on the isoluminant plane. The mask then disappeared, and after a short interval (250 ms), a color square (size: 1 deg × 1 deg) appeared at the center of the screen, surrounded by a full-range color ring (at 7.5 deg eccentricity, width: 1 deg), covering 0°–360° hue at the isoluminant plane. The color ring was added to help observers navigating the circular color space. Observers were required to adjust the color of the square to match the previously seen color of the target circle. The initial color of the square was randomly selected out of all possibilities. They could make either a big-step change (using Left/Right arrow keys, each step amounting to a 3° change on the isoluminant plane) or a small-step change (using Up/Down arrow keys, each step was 1°), without any time limitation. Following the response period, a blank screen was shown for 200 ms. In each block, every prior stimulus was repeated 20 times, leading observers to complete a total of 40 trials (2 priors × 20 repeats), resulting in 80 judgments (each prior followed by a probe) and observers completed two blocks for a total of 160 judgments. 

Orientation judgment

The orientation judgment procedure paralleled that of the color judgment. Initially, a fixation circle was displayed for two seconds, followed by an oriented Gabor patch (spatial frequency: 1/3 cycles/deg, peak contrast: 20%, Gaussian envelope: 1.5 deg s.d.) positioned in the right visual field at 6.5 deg eccentricity for 300 ms. Observers were required to use their peripheral vision to determine the orientation of the Gabor. Subsequently, a 1/f noise mask embedded in the same Gaussian envelope was presented for one second. During the response period, an adjustable white line (length: 3.6 deg, width: 3 pixels) appeared at the same location. The initial orientation of the square was randomly selected out of all possibilities. Similar to the color task, observers used the arrow keys to adjust the line's orientation to match their perceived orientation of the Gabor patch. On completing the adjustment, they pressed the space bar to proceed, which either started the same sequence for the probe stimulus or, if the response was made for the probe stimulus, started the next trial. 

All stimuli were displayed on a mid-gray background (82.68 cd/m²). The probe Gabor was set to a fixed orientation of 36°. The prior was presented with a 15° shift either clockwise (21°) or counter-clockwise (51°). In each block, every prior orientation was repeated 20 times, culminating in a total of 40 trials and yielding 80 judgments. Observers completed two blocks. 

Oculomotor tracking

For the oculomotor task, a single trial consisted of two movements that needed to be tracked by the observer: The prior movement, which could move with either 5 deg/s or 15 deg/s and the probe movement which always moved at 10 deg/s. The direction of the movement was randomly either to the left or right, but the prior and probe movement always moved in the same direction. Before the start of a trial, participants saw a red fixation cross and started the trial by looking at it and pressing the space bar. This was step was used as a drift correction. Then a red fixation circle (diameter 0.2 deg) was presented for a random time between 1 and 1.5 seconds. Then the dot disappeared and the target, a Gaussian blob (SD = 0.4 deg) appeared and immediately moved across the screen. The contrast for the prior stimulus was set to 1, whereas the contrast for the probe stimulus was set to 0.1 to maximize a potential serial dependence effect. The target always appeared with a slight offset into the opposite direction of the target movement; the offset was scaled in a way that the target always crossed the initial fixation location after 200 ms (e.g., for a speed of 10 deg/s, the step was 2 deg). This step-ramp paradigm (Rashbass, 1961) was used to reduce the need for an initial corrective saccade. The target then kept moving for one second and disappeared. Then a new fixation circle appeared to indicate the start of the probe movement. After a new random time between 1 and 1.5 seconds, the probe movement was presented with the same temporal characteristics as the prior. When the probe movement disappeared, the appearance of a new fixation cross indicated the start of the next trial. 

Each of the prior speeds (5 deg/s and 15 deg/s) were presented 15 times per block, leading to the tracking of a total of 60 movements (30 priors and 30 probe movements) per block. Again, observers completed a total of two blocks. 

Pupil light reflex task

At the onset, observers underwent a one-minute adaptation period for a mid-gray blank screen (50% intensity, 81.68 cd/m²) within a dark room. After this initial adaptation phase, pupillary data recording began. Before each trial, observers were presented with a red fixation cross, and they initiated the trial by fixating on the cross and pressing the spacebar. This step was again used as a drift correction. Once the trial began, a black fixation dot (0.3 deg in diameter) appeared for one second. Subsequently, a darker/brighter grayish prior disk (10 deg in diameter) appeared alongside the fixation dot for one second, followed by a blank period with fixation lasting seven seconds for pupillary measurement. Following this, a probe disk (10 deg in diameter) appeared for one second and transitioned back to a blank screen with fixation for seven seconds. There were two prior conditions: one with a high luminance disk (100% intensity, 156.2 cd/m²), brighter than the probe as a medium luminance disk (80% intensity, 128.4 cd/m²), and a low luminance disk (60% intensity, 96.90 cd/m²), darker than the probe. 

In the pupillary task, observers were instructed to minimize blinking during the trial, with the flexibility to blink before initiating each trial. All frames within the trials, except for the initial one-minute adaptation, featured a black fixation dot at the center of the screen to mitigate potential location bias in pupil size measurement. Each prior condition was repeated 10 times, resulting in a total of 20 trials for one block. Again, observers completed two blocks. 

Data analysis and preprocessing

Color task

The magnitude of the serial dependence effect was quantified by analyzing the hue reproduction errors for the probe (azimuth of 315°) in each trial, calculated as the difference between the reproduced hue azimuth and the probe's azimuth. These errors were compared across two conditions, corresponding to the presentation of two different priors (with azimuths of 275° and 355°). 

For each observer, the median probe reproduction error was calculated for all trials within each prior condition. The difference between these median errors for the two conditions was then calculated to establish a metric for the serial dependence effect. We identified whether an observer exhibited an attractive serial dependence (where probe reproduction errors were more negative with 275° priors than with 355° priors), a repulsive serial dependence (where probe reproduction errors were more positive with 275° priors than with 355° priors), or no effect. Please note that the difference metric used here in all tasks is robust against individual biases in for example always making more positive errors. 

Orientation task

The analysis of the orientation task paralleled that of the color task. We calculated the orientation estimation errors for the probe Gabor (36°) for each trial, defined as the difference between the reproduced orientation and the probe's orientation. These errors were then compared across trials that featured two different prior Gabors (with orientations of 21° and 51°). An attractive serial dependence was identified if the reproduction errors for the probe Gabor were more negative with priors set at −15° than with 15° priors. Conversely, a repulsive serial dependence was suggested if the reproduction errors were less negative or more positive with −15° priors compared to 15° priors. Because we had one observer who showed very poor accuracy in the orientation judgment task (average error of around 79°) and was more than three times the standard deviation away from the mean of the other observers, we excluded him from this part of the analysis. 

Oculomotor task

Eye movement data saved and analyzed off-line using custom Matlab scripts. First, blinks were linearly interpolated and the eye position was filtered with a second-order Butterworth filter, with a cutoff frequency of 30 Hz. Then eye velocity was calculated as the first derivative of the filtered position traces. Saccades were identified based on the EyeLink criteria with a speed and acceleration threshold of 30 deg/s and 4000 deg/s², respectively. After the detection of saccades, a linear interpolation of the eye movement velocity around the time of the saccade (from 35 ms before the detected saccade onset to 35 ms after the detected saccade offset) was performed. Eye movement velocity was filtered with an additional low-pass Butterworth filter with a cutoff frequency of 20 Hz. 

Because all targets moved horizontally, we took the horizontal velocity of the eye and aligned it to the target movement onset. To ensure that there was a valid tracking response, we detected pursuit onset in the prior and probe trials as the first point where the horizontal eye velocity reached 30% of the target speed and stayed there for at least 100 ms. Then, for each observer, across all relevant trials, the median eye velocity trace for the probe stimulus was computed, and this was done separately depending on the prior velocities. The serial dependence effect was then computed as the average difference between the two velocity traces from 100 to 350 ms after target motion onset. A positive effect indicated a faster eye velocity for the faster prior. 

To ensure our data quality, we excluded trials from the analysis that had more than 500 ms of missing data in a single trial, where the computed velocity after the interpolation of the saccades still was larger than 30 deg/s (indicating some overlooked saccades), and trials where we couldn't identify a valid pursuit response with a latency below 400 ms. This was done both for the prior and probe movement and only trials where both movements fulfilled all criteria were used for the analysis. With these criteria, we included a total of 4347 out of 4800 trials (90.5%) in the analysis. 

Pupil task

The eye movement data were saved during the task and analyzed offline with custom MATLAB scripts. First, the pupil size during blinks was linearly interpolated (−100 ms before blink to 100 ms after blink) and then was filtered with a second-order Butterworth filter, with a cutoff frequency of 30 Hz. Then we aligned the pupil response to the stimulus onset of the probe movement. To account for individual differences in pupil size, the pupil size was normalized based on the average pupil size 1000 ms before a stimulus was presented (the fixation period before the prior and the probe stimulus was presented). As for the eye velocity, we then computed the median pupil size response to the probe stimulus, separately for each of the prior stimuli. To compute the serial dependence effect, for each observer we computed the mean difference in pupil size in the one second where the probe stimulus was presented. A positive value indicated a stronger pupil light reflex for the brighter prior. 

Trials were excluded from the analysis when we had missing data for more than 500 ms. Based on these criteria we included a total of 1555 out of 1600 trials (97.2%) in the analysis. 

Statistical procedures

To test for serial dependence effects on the group level for each task, we took the respective metric and computed one-sampled Bayesian t-tests. We tested whether the distribution was significantly different from zero. The strength of the Bayesian tests is that they also allow the interpretation of evidence for the absence of an effect. 

To ensure that we observed meaningful individual differences, we estimated the reliability of the individual differences in each of the tasks via a split-half correlation. For this, we used a bootstrapping procedure, where instead of taking all trials to compute the average response for the probe stimulus with a respective prior trial, we randomly split the trials into two groups. We did this for both priors and then computed the serial dependence effect in the same way as before. To estimate a distribution for each observer, we repeated this procedure 1000 times. This gave us for each observer 1000 estimates of the serial dependence effect for one half of the trials and another 1000 estimates for the second half of the trials. We repeated this procedure for each of the observers and then for each of our 1000 iterations computed the Pearson correlation between the serial dependence effects estimated for one half and the serial dependence effects for the other half across observers. This distribution of 1000 correlations gave us an estimate of the reliability of the individual differences for the respective task. 

To look into the potential origins of the individual differences we conducted two control analyses. First, in the color and orientation judgment tasks, the trial durations were variable and dependent on the time it took observers to give a response to the prior stimulus. To assess whether the time between the presentation of the prior and probe stimulus influenced the serial dependence effect, we conducted Pearson correlations between the individual strength of the serial dependence effect and the median duration between the two. Second, we wanted to relate overall differences in task performance to the strength of the serial dependence effect. We hypothesized that more accurate observers might rely more on the prior stimulus. For that, we computed a proxy for behavioral accuracy in the prior: for the orientation and color judgment tasks, the median of absolute response errors relative to prior stimuli was used as the accuracy metric for each observer. To compute the accuracy of tracking performance, we computed the average pursuit gain for the prior movement in the interval of 100 to 300 ms after pursuit onset. Because, as a result of the lack of a ground truth, we could not compute an accuracy measurement for the pupil response, we did not perform this analysis for the pupil task. 

In the last step of the analysis, we correlated the strength of the serial dependence effects across tasks. We did so for each of the tasks that showed a significant serial dependence effect on the group level (color, orientation, and tracking). To correct for testing multiple correlations, we adjusted the p-value from the original p = 0.05 to p = 0.017 (0.05 divided by the three possible correlations). 

Results

To search for common underlying mechanisms for sequential effects, we measured serial dependence effects in four different tasks (Figure 1) for the same group of 40 observers. We used modified versions of three already established tasks: color judgments, orientation judgments, and oculomotor tracking. Additionally, we included a novel domain for oculomotor behavior: the pupil light reflex (Hall & Chilcott, 2018). All tasks had a similar structure. Each trial consisted of two intervals. During the first interval, the prior, observers saw one of two stimuli that differed across the relevant dimension of the task (e.g., two different orientations for the orientation task or two different speeds for the tracking task). After a response (either a judgement, or an oculomotor response), the prior interval was then followed by the second interval, the probe, in which the stimulus was always identical (see Methods for details). Thus the stimuli observers saw were semi-predictable: Although the prior stimulus was randomized, the probe stimulus was always predictable. This allowed us to make a quick estimate of the strength of serial dependence effects, by looking at the differences in response to the probe depending on which prior was presented (see Figure 1 bottom row). 

Serial dependence effects for each task

For the already established tasks, we replicated previous results with our short tasks (Figure 2A). Across observers, we found significant attractive serial dependence effects for color judgments (BF₁₀ = 11.160), orientation judgments (BF₁₀ = 138.961), and tracking behavior (BF₁₀ = 3.125 × 10¹⁷). Despite the effect on the group level, there was substantial variability between observers. For perceptual judgments, some observers even showed repulsive effects and for all tasks, the magnitude of serial dependence effects differed by multiple orders of magnitude across observers. These variations were not just measurement noise but were reliable differences between observers (see Figure 2B, estimated via split-half Pearson correlation: Color: r = 0.627; Orientation: r = 0.493; Oculomotor: r = 0.344, see Methods for details). 

Figure 2.

 

Serial dependence effects across different tasks. (A) The strength of serial dependence effects is quantified as the differences observed in the probe responses across different priors. Dots depict individual observers. The solid line depicts the median and the shaded area in the background depicts a violin plot based on a distribution estimate. ** Depict a significant effect on the .01 level. (B) Depicted is the split-half reliability of the serial dependence effects across the four tasks. Dots show boot-strapped samples, the solid lines represent the median of the distribution. (C–D) Correlations between the time intervals between the presentation of the prior and the probe stimulus and the strength of the serial dependence effects for the color (C) or the orientation (D) task. Please note that the timing between stimuli was always consistent for the tracking and pupil tasks.

In contrast to the overwhelming literature on sequential effects across all kinds of tasks, for the pupil response, we did not observe a serial dependence effect. There was even significant evidence for the absence of an effect (BF₀₁ = 5.861, Figure 2A). The lack of a systematic effect was also supported by the absence of reliable individual differences (estimated via split-half Pearson correlation: r = −0.065, Figure 2B). Therefore the pupil task provides a rare exception as one task that does not show a serial dependence effect. 

The relation of serial dependence and differences in sensory processing

Since we observed substantial variability in the magnitude of the serial dependence effects across observers, we wondered whether this could be explained by individual differences in sensory processing. While for the perceptual tasks the timing between the prior and probe differed due to different response durations, this was not related to the strength of the serial dependence effects (see Figures 2C–2D, the timing between stimuli was fixed in the oculomotor task). Although the timing between the prior and probe varied substantially between observers for the perceptual tasks due to differences in response time when judging the prior stimulus, there was no relation to the strength of the serial dependence effect of the observers. In addition, previous work has shown that serial dependence effects are stronger for more reliable priors or under situations of more uncertainty in the current sensory input (Ceylan et al., 2021; Darlington, Beck, & Lisberger, 2018; Gallagher & Benton, 2022; Goettker, 2021). Therefore we tested whether there is a relationship between the accuracy of responses for the prior stimulus as a proxy for its reliability and the strength of serial dependence effects for different observers. The median of absolute response errors for the prior stimuli was calculated to determine the accuracy of each observer in the perceptual tasks, and the average pursuit gain for the prior movement in the interval of 100 to 300 ms after pursuit onset was calculated to determine accuracy in the oculomotor task. However, neither for perceptual tasks (Color: r(38) = −0.281, p = 0.079; Orientation: r(37) = 0.189, p = 0.250, Pearson correlation) nor for the oculomotor tracking task (r(38) = 0.153, p = 0.347), a systematic relationship was present. This highlights that the variability between observers in the magnitude of the serial dependence effects cannot be simply explained by differences in basic sensory processes. 

Linking the strength of serial dependence across tasks

For our main aim of searching for similarities in serial dependence effects across tasks, we harnessed the reliable individual differences observed for color and orientation judgments and oculomotor tracking (Figure 3). We observed that the magnitude of the serial effects was correlated between the two perceptual tasks (r(37) = 0.439, p = 0.005, Figure 3A) across observers. This result complements recent work showing that individual differences within a single task can be predicted by a different weighting of a positive choice and a repulsive motor bias (Zhang & Alais, 2020). However, they extend our current knowledge by showing that this mapping generalizes across different perceptual tasks. This indicates that there is a substantial amount of shared variability across our two perceptual tasks (∼20% of the variance). 

Figure 3.

 

Relationship between strength of serial dependence effects across tasks. Correlations between individual serial dependence effects across tasks. The correlation between (A) the color and orientation task, (B) orientation and tracking task, and (C) the color and tracking task. Dots represent individual observers, the gray bold line in (A) represents a linear regression model fitted to the data, and the gray dashed lines represent the 95% CI of the fit.

In contrast to the two perceptual tasks, there was no systematic relationship between the sequential effects for the two perceptual tasks and oculomotor tracking (r(38) = −0.234, p = 0.147 for color; r(37) = −0.208, p = 0.205 for orientation). Because we did not observe a significant serial dependence effect for the pupil task and also no reliable individual differences, it is not surprising that there were also no significant correlations between the pupil task and any of the other tasks (all rs < 0.131, all ps > 0.419). 

Discussion

We observed attractive serial dependence effects for color and orientation judgements, as well as for oculomotor tracking. In addition, although serial dependence effects have been observed across a wide range of tasks, we did not observe it for the pupil-light reflex. For tasks where we found consistent effects on the group level, there were still large and reliable individual differences between observers for all tasks. These differences were not related to basic differences in visual processing, but showed an interesting pattern across tasks: The strength of the serial dependence effect for the two perceptual tasks was correlated across observers, but there was no relationship to the strength of the serial dependence for the oculomotor tasks. 

Serial dependence shows similarities across different perceptual tasks

The observed relation in the strength of the sequential effects across different perceptual tasks provide a new perspective: They point to the potential importance of common factors determining the individual weight of past information across different perceptual tasks. However, it is important to note that this only a moderate correlation indicating a similarity of roughly 20% between the two perceptual tasks and there are some limitations in the generalizability of the results (see Limitations and Future Research section). In the following we will speculate about factors that could lead to these similarities:

The similarity in the magnitude of the serial dependence effects could be explained by common underlying factors such as the role of visual working memory (Bliss et al., 2017; Markov et al., 2024; Mei et al., 2019). The strength of the persistence of a previous memory trace might differ across individuals and this could directly affect the strength of the serial dependence effect and lead to a correlation between the two tasks. In addition, since we tested serial dependence in a semi-predictable environment as the probe stimulus was always predictable (for a more detailed discussion of the potential influence of this see also the next section), observers could also differ in how much they relied on this predictability, and therefore reduced the influence of the previous stimulus, when responding to the probe stimulus. Similarly, differences in task engagement or attention could lead to a relation across tasks. However, one could expect that these more general, task-independent factors or strategies should lead to a correlation of the serial dependence effects across all tasks, but we did not observe a correlation between the magnitudes of serial dependence for perceptual tasks and for oculomotor tracking. This suggests that the exclusive correlation between the two perceptual tasks might be based on more specific mechanisms.

 

One related topic to the observed partial generalizability across the different perceptual tasks is that even within the same modalities there might be differences between observers how the magnitude of serial dependence varies over space. Although Kondo and colleagues (2022) observed reliable individual differences across sessions between the strength of serial dependence for orientation judgements when the stimuli were presented at the same location in the visual field (e.g., both times measured in the right periphery), this correlation was significantly reduced when judgements were measured at different visual field locations (once in the right periphery, once in the fovea). Thus suggesting that serial dependence within the same modality can vary across visual space in an observer-specific manner. That we still observed a correlation between our two perceptual tasks could make an interesting prediction: If there are observer specific profiles how the magnitude of serial dependence differs across space, it would be interesting to test whether these profiles also share similarities across tasks. In our experiments, we presented the two perceptual tasks at the same peripheral location, and therefore can only say that there is a similarity for the one spatial location (6.5° in the periphery) we tested in both tasks, but it would be interesting to see that although observer show space-specific profiles for the magnitude of serial dependence within a certain task, whether these profiles are consistent across task by testing both tasks at different locations. 

An alternative potential explanation for the similarities could be that differences in the quality of the incoming sensory information might play a critical role. For perceptual tasks (Ceylan et al., 2021; Gallagher & Benton 2022), as well as for oculomotor tracking (Darlington et al., 2018; Deravet, Blohm, Orban de Xivry, & Lefèvre, 2018; Goettker, 2021), suggestions have been made that the magnitude of sequential effects increases when the current sensory input is more uncertain. Thus observers with lower levels of sensory uncertainty might be more likely to show weaker serial dependence effects. We tried to address this point by looking into individual differences in the timing between the prior and the probe stimulus, with longer times leading to more noise in the representation of the stimulus, and the accuracy in the report, serving as a proxy for the overall uncertainty level of the sensory representation of the stimulus. However, for both of these factors we did not observe a correlation with the individual differences in the magnitude of serial dependence effect. This suggests, that although differences in uncertainty clearly have an effect on the overall magnitude of the effect on the group level (Ceylan & Pascucci, 2023; Darlington et al., 2018; Deravet et al., 2018; Gallagher & Benton, 2022; Goettker, 2021), individual differences in these factors seem not to be directly related to individual differences in the magnitude of serial dependence. 

Another potential factor could be individual differences in the reliance of previous information. It was suggested that serial dependence effects for perceptual judgements are be driven by top-down signals that reflect previous experience that influence the incoming sensory information of the next trial (Cicchini, Benedetto, & Burr, 2021). Animal work with rats has shown that the posterior parietal cortex encodes perceptual history and mediates its effect on behavior (Akrami, Kopec, Diamond, & Brody, 2018). In line with that we also see that prior expectations bias the representation of stimuli in early visual cortex (Kok, Failing, & de Lange, 2014; Summerfield & de Lange, 2014), and even can preactivate expected stimulus templates before the next trial is presented (Kok, Mostert, & de Lange, 2017). Our results suggest that the strength of this positive perceptual choice bias (Zhang & Alais, 2020) could consistently vary across different perceptual judgement tasks: some people show a stronger influence of this positive choice bias, and some people show a weaker influence. This different weighting then could introduce the similarities in serial dependence effects observed across our different perceptual tasks. 

The difference between serial dependence in random and semi-predictable environments

One important topic to address is how the present results testing serial dependence in a semi-predictable environment are related to more classical serial dependence studies that use a random trial structure where stimuli cannot be predicted. We chose to use a counterbalanced trial structure where we only used two different stimulus levels for the prior and the probe was always constant to get a quick assessment of the magnitude of the serial dependence effect for each of our tasks in comparable paradigms. This allowed us to focus on our major point, the relation of the strength of the serial dependence effect across tasks, in a reasonable amount of time. However, it is not clear whether the mechanisms involved in the serial dependence effects observed in more predictable environments are comparable to the mechanisms described in random environments and therefore how our results would generalize to such studies. 

Fischer & Whitney (2014) directly compared serial dependence between a counterbalanced trial order (as in the current experiment) and a random sequence and observed attractive serial dependence in both cases: the estimated magnitude of the effect was qualitatively larger in the counterbalanced blocks, but there was no statistically significant difference between the two conditions providing no conclusion about whether the same or different mechanisms were leading to the observed serial effects. However, there are studies showing that perceptual judgments can be influenced by predictable trial sequences: When observers see a sequence of gratings that either change in a predictable (for example a constant rotation offset) or random order and have to judge the orientation of the final grating, observers were more certain when they were able to predict the final stimulus and showed a repulsive effect with respect to the penultimate stimulus (Abreo, Gergen, Gupta, & Samaha, 2023). In contrast, for the random order, the penultimate stimulus led to an attractive serial effect. This difference seems to be related to a phenomenon called representational momentum, an effect that biases perceptual reports towards future states of predictable changes (Pascucci & Plomp, 2021). However, in our design such a representational momentum was not present since the prior stimulus (although it only had two levels) was randomized and therefore could not be predicted and no consistent patterns were available. 

Although at the end it is an empirical question whether the mechanisms involved in the serial dependence effects observed for a counterbalanced or random trial structure are the same (for example by relating individual differences between estimates of the serial dependence effects between the two trial orders), we argue we can still learn a lot from our results. In our semi-predictable environment, we replicated the attractive serial dependence effects observed for orientation, color and oculomotor tracking in previous studies. This is noteworthy because in general the magnitude of attractive serial dependence effects seems to be larger when the stimulus that needs to be judged is uncertain (Ceylan et al., 2021; Cicchini, Mikellidou, & Burr, 2018; Gallagher & Benton, 2022). Although in our design the sensory stimulus was uncertain, the probe stimulus was 100% predictable and constant across the experiment. Nevertheless, there was still an attractive serial dependence effect suggesting that on this trial-by-trial level there seems to be an almost automatic integration of previous information. In fact, for oculomotor tracking this adaptation on a trial-by-trial level seems to be even maladaptive (Goettker, 2021): Despite the probe being predictable, tracking the probe stimulus after the slow prior stimulus leads to larger errors in tracking performance than tracking the probe after the fast stimulus. Similarly, when systematically varying the previous information also perceptional judgements show short-term biases, which can reduce the overall accuracy. However, the cost of such short-term biased might be outweighed in terms of more general mechanisms that help to smooth our representation of the world (Manassi & Whitney, 2024). 

The absence of an effect for the pupil-light reflex

The absence of any serial dependence effect for the pupil task came as a surprise, since serial dependence was already found for a large range of different stimuli (Manassi et al., 2023) and tasks (Kiyonaga et al., 2017). Please note here that this is now only our single experiment showing the absence of serial effects for the pupil-light reflex, which does not rule out that the pupil response cannot show serial effects under different conditions or given a different task (Pomè, Binda, Cicchini, & Burr, 2020). However, given our result, in the following we try to speculate about potential explanations for the observed absence of serial effect in our experiment. 

First, one could argue that this null effect could be explained by the relatively long time between the prior and probe stimulus (7s) for the pupil task, which was introduced to give the pupil time to go back to baseline size before the probe occurred. While there is clear evidence that serial effects are stronger for more recent information (see Manassi et al., 2023 for an overview), the temporal tuning of sequential effects can be influenced by different factors (Bilacchi, Sirius, Cravo, & Neto, 2022; Bliss et al., 2017; Ceylan & Pascucci, 2023) and might even depend on the type of attribute to be judged (Ortega, Chen, & Whitney, 2023; Taubert, et al., 2016). However, there are studies still reporting serial dependence effects for time intervals longer than our interstimulus interval (see Manassi et al., 2023): This is also exemplified by the fact that within our own study, the time between the prior and probe stimulus for the perceptual tasks was comparable, as observers also first gave a perceptual response for the prior stimulus (average time: 10.63 ± 1.98 seconds for Color; 7.38 ± 2.20 seconds for Orientation). Therefore we think it is unlikely that the just the long interstimulus interval by itself can explain the absence of an effect. Second, serial effects introduce small biases in perceptual responses, however here they are not really harmful and in the sense of a continuity field might be even helpful to smooth out noise in perception (Manassi & Whitney, 2024). In contrast, an overdilation of the pupil despite bright lighting conditions due to a sequential effect could potentially damage photoreceptors. Third, pupil size can be modulated by very different factors ranging from a low-level light response to the influence of higher cognitive factors. The pupil light response task investigated in this experiment is mostly controlled in a reflex-like manner by the brain stem (Hall & Chilcott, 2018). Previous work showing that sequential effects for oculomotor tracking are controlled by frontal brain areas (Darlington et al., 2018). In line with that, there is evidence that the pupil is reflecting sequential effects when there is a cognitive signal that indicates that the new stimulus is unpredictable and pops-out (Pomè et al., 2020). Therefore the absence of a serial dependence effect to the mostly luminance driven pupil-light reflex suggests that there is the need for higher-level cortical processing to find sequential effects. 

Different mechanisms for perceptual tasks and oculomotor control

The lack of a link between perceptual judgments and oculomotor control highlights that information processing for perception and oculomotor control can differ (Spering & Carrasco, 2015). Thus there might also be differences in how top-down information is used: although mediating signals for perceptual serial dependence effects originate at higher-level visual processing (Cicchini et al., 2021), serial dependence effects for tracking eye movements seem to rely on low-level retinal information (Goettker & Stewart, 2022). This most likely reflects the different goals of the two systems: Although perception needs to integrate information to create a valid percept, for oculomotor tracking mostly low-level retinal information plays a role across trials to be able to foveate objects of interest. In addition, while most perceptual tasks require on discrete motor response that can be influenced by visual, decisional or working memory influences (Pascucci et al., 2023), oculomotor tracking requires an immediate continuous response. However, a recent preprint even suggests, that even the attractive serial dependence effect for oculomotor tracking seems to have multiple components: one retinotopic, directionally-tuned component, and more general direction unselective component (Goettker & Stewart, 2024). Especially the latter component might also be related to more general aspects like working memory or attentional processes and could therefore in isolation actually be still related to similar mechanisms at play for the perceptual tasks. 

Limitations and future research

Although our results provide a first glimpse into how serial dependence effects are linked across tasks, it remains unclear which specific factors—such as working memory, decision-making, or more visual top-down processes—are the determinants of individual differences or individual weighting. Therefore it will be critical to include, for example, measurements of working memory performance as an additional control variable for future studies. In addition, it will be critical to investigate how far this generalization holds across different and more complex tasks: our two perceptual tasks were both related to low-level sensory signals that share some processing resources (Garg, Li, Rashid, & Callaway, 2019; Gegenfurtner, Kiper, & Fenstemaker, 1996) and required a similar judgment response. Especially the latter might be something to consider when comparing serial dependence effects across tasks: When people made the judgment for the color task by picking a color from a hue circle, they might have even also encoded their color response in terms of an orientation judgement on the hue circle. This could also lead to a close relation between the two tasks. Therefore our two tasks might be inherently more similar to each other than to a task requiring the judgment of other features, such as higher cognitive level features like facial emotion where no discrete judgment about a low-level visual feature is needed and different response types are required. Moreover, all tasks showing serial dependence effects in this study exhibited an attractive effect. It would be important to include tasks with repulsive serial dependence effects to determine whether these opposite effects share the same mechanisms. If this generalization indeed holds across the visual processing hierarchy, it will allow important insights into the shared origins and mechanisms that can explain sequential effects across tasks. 

In addition, it is important to note, that while we observed reliable individual differences in the strength of serial dependence in line with previous work (Kondo et al., 2022; Zhang & Alais, 2020), the split-half correlations are not 1. Especially, the lower reliability for oculomotor tracking could have a significant impact on the estimated correlation across tasks and potentially even lead to an underestimation of the correlations (Murphy & Davidshofer, 1988). However, please note that the correlation between oculomotor tracking and the perceptual tasks were even negative. So even when we would correct for the low reliability of the individual metrics, the correlation would still show an opposite pattern: People with strong serial dependence for oculomotor control, are more likely to show weaker serial dependence for the perceptual tasks. This would match well with previous work that shows that the sequential effect for oculomotor control can be in the opposite direction of the sequential effects for perceptual judgments (Maus, Potapchuk, Watamaniuk, & Heinen, 2015; Wu, Rothwell, Spering, & Montagnini, 2021). This happens even on the same trials, presumably because of a different temporal integration for the two systems. 

Conclusions

Together, our results highlight the potential of individual differences to gain insights into sensorimotor processing. Leveraging variations between observers revealed similarities in perceptual serial dependence tasks that could unify parts of the vast literature on serial dependence effects across the whole visual processing hierarchy. In addition, we found differences between how sensory information is processed for perception and oculomotor control and a rare exception: one of the most basic reflexes of sensorimotor control, the pupil light reflex, did not show serial dependence effects. This indicates that sequential effects require cortical processing. 

Acknowledgments

The authors thank Lorena Klöckner for her help with the data collection. 

S.G. was supported by European Research Council (ERC Advanced Grant Color 3.0, 884116). A.G. was supported by the Deutsche Forschungsgemeinschaft (SFB/TRR 135 Project A1, 222641018). 

Commercial relationships: none. 

Corresponding author: Shuchen Guan. 

Email: [e-mail]. 

Address: Justus Liebig Universität, Giessen 35390, Germany.