Serial dependence promotes object stability during occlusion

Abstract

Object identities somehow appear stable and continuous over time despite eye movements, disruptions in visibility, and constantly changing visual input. Recent results have demonstrated that the perception of orientation, numerosity, and facial identity is systematically biased (i.e., pulled) toward visual input from the recent past. The spatial region over which current orientations or face identities are pulled by previous orientations or identities, respectively, is known as the continuity field, which is temporally tuned over the past several seconds (Fischer & Whitney, 2014). This perceptual pull could contribute to the visual stability of objects over short time periods, but does it also address how perceptual stability occurs during visual discontinuities? Here, we tested whether the continuity field helps maintain perceived object identity during occlusion. Specifically, we found that the perception of an oriented Gabor that emerged from behind an occluder was significantly pulled toward the random (and unrelated) orientation of the Gabor that was seen entering the occluder. Importantly, this serial dependence was stronger for predictable, continuously moving trajectories, compared to unpredictable ones or static displacements. This result suggests that our visual system takes advantage of expectations about a stable world, helping to maintain perceived object continuity despite interrupted visibility.

Introduction

From moment to moment, we perceive the identities of objects and people as stable even though their image properties frequently fluctuate due to factors such as occlusion, changes in viewpoint, eye movements, and other sources of noise. Whether and how the visual system promotes perceived object continuity over time remains an important question. Recent experiments have demonstrated a serial dependence in perception: a bias in the perceived identity of objects toward similar objects seen in the last few seconds (Cicchini, Anobile, & Burr, 2014; Fischer & Whitney, 2014; Kondo, Takahashi, & Watanabe, 2012, 2013; Liberman, Fischer, & Whitney, 2014; Taubert, Van der Burg, & Alais, 2016b). Intriguingly, visual processing may mirror the temporal regularity of objects in the physical world through a perceptual bias that promotes stability of identity perception. 

Previous studies revealed that the perception of orientation, numerosity, and other low-level features is indeed serially dependent—systematically biased toward similar visual input from the recent past (Cicchini et al., 2014; Corbett, Fischer, & Whitney, 2011; Fischer & Whitney, 2014). Beyond these basic features, face identity perception is also systematically biased toward identities seen up to several seconds prior even across viewpoint changes (Liberman et al., 2014). To test for serial dependence in orientation perception, Fischer and Whitney (2014) had subjects report the perceived orientation of a series of randomly oriented Gabors using a continuous response wheel in a method of adjustment matching task (they and others have also used two-alternative forced choice [2AFC] tasks; Cicchini et al., 2014; Fischer & Whitney, 2014; Liberman et al., 2014; Taubert et al., 2016b). Fischer and Whitney found that reported Gabor orientation was consistently biased toward the orientation of the Gabor on the previous trial. This is revealed by a positive relationship between subject error in the current trial and the difference between the orientations of the current and previously seen Gabor patches. This serial dependence is tuned to the similarity between successive orientations, such that if the current and previous Gabors have radically different orientations, no bias in perception occurs. Essentially, similar but distinct orientations are perceived as more similar than they actually are. This serial dependence effect is also tuned in space and time, and the spatiotemporal region over which current object features, such as orientation, are pulled by previously seen features is known as the continuity field (CF). For orientation, the spatial tuning of the CF extends over 20° or more of visual space and has a spatiotopic component (Fischer & Whitney, 2014). The CF is temporally tuned over the past 5–15 s; moreover, the CF is gated by attention, meaning that the perception of orientation is only serially dependent on previously attended orientations (Fischer & Whitney, 2014). 

Although serially dependent perception manifests as a misperception, it could be adaptive: Visual processing echoes the stability of objects in the world to create perceptual continuity. This would be especially helpful in cases in which there is noise, occlusion, or discontinuities in the retinal image. For example, how similar does an object look after it is temporarily obscured and then reappears (e.g., because it moves behind an occluder, because of motion parallax, etc.)? If the CF facilitates the stability of an object's appearance in the presence of noise or occlusion, then we should find serial dependence in the perception of an object that reemerges from an occluder. 

A great deal of research has investigated dynamic occlusion and the processes that support tracking moving objects behind occluders, often referred to by names such as object permanence, the tunnel effect, amodal integration, etc. (Baillargeon, Spelke, & Wasserman, 1985; Burke, 1952; Flombaum & Scholl, 2006; Michotte, Thines, & Crabbé, 1991). However, previous work does not make a clear prediction about whether there is serial dependence in the perception of objects that enter and reemerge from behind occluders. It is well known that observers perceive objects moving continuously behind an occluder as following a single trajectory even if those objects look different before entering and after exiting from behind the occluder (Burke, 1952). Additionally, detecting the change in an object's color after occlusion improves when the object is perceived as moving along a continuous trajectory (Flombaum & Scholl, 2006), indicating that we perceive objects as persisting even when we cannot see them. Furthermore, object-based attention increases for occluded objects compared to when those same objects are visible, suggesting that attentional resources increase during object occlusion (Flombaum, Scholl, & Pylyshyn, 2008). This knowledge of object continuity and persistence during occlusion is learned at a young age and is present early in infancy (Carey & Xu, 2001; Spelke, Kestenbaum, Simons, & Wein, 1995). Despite a great deal of research, it still remains to be explored how an object appears as it reemerges from behind an occluder. 

If the CF helps maintain perceived object identities during visual discontinuities, then we should be able to measure serial dependence through occlusion: Sequential objects should look more similar than they actually are. Moreover, we should find stronger serial dependence for continuous trajectories that are expected and natural as opposed to trajectories that are unpredictable or unexpected. Note that the change detection experiments mentioned earlier (Flombaum & Scholl, 2006) do not anticipate these results: Serial dependence makes similar objects look identical, which should weaken change detection and increase change blindness. The seeming contradiction in these predictions is easily reconciled because the change detection experiments tested sequential objects that were easily discriminable (Flombaum & Scholl, 2006). Thus, no serial dependence would be expected in those cases because serially dependent perception is tuned to feature and object similarity (Fischer & Whitney, 2014; Liberman et al., 2014). In the following series of experiments, we examined whether perceptual serial dependence facilitates the appearance of stable object identities despite occlusion. 

Experiment 1: Does serial dependence stabilize perceived object identity during interrupted visibility?

If the object-selective CF facilitates the stability of an object's appearance in the presence of noise or occlusion, then we should find serial dependence when that object emerges from behind an occluder. The object's identity should appear to be pulled by the identity of the object that first entered the occluder, especially when it travels behind the occluder in a predictable or expected trajectory. To test this hypothesis, we presented a series of randomly oriented Gabor patches that traveled behind an occluder along either a continuous or discontinuous trajectory (Figure 1). Our question was whether the perceived orientation of the Gabor that emerged from the occluder was serially dependent on the orientation of the Gabor that entered the occluder. 

Figure 1

 

Experiment 1 stimuli and procedure. (a) Example continuous moving trajectory trial sequence for a Gabor with a starting position at the top left corner of the screen. Subjects responded by adjusting a rectangular bar (0.24° × 4°) at fixation to match the perceived orientation of the second, exiting Gabor patch. (b) Example continuous moving trajectory path for a Gabor with a starting position in the middle of the screen. (c) Example discontinuous moving trajectory trial for a Gabor with a starting position at the bottom left corner of the screen.

Methods

Stimuli

Experiments were conducted in a darkened experimental booth. Subjects viewed stimuli on a CRT monitor (1024 × 768, 100 Hz, Dell Trinitron) at a distance of 56 cm. All experiments were programmed in MATLAB (The MathWorks, Natick, MA) using Psychophysics Toolbox (Brainard, 1997). The stimulus consisted of drifting Gabor patches. Each Gabor patch was a sine wave grating (carrier) 4 c/°, 29% Michelson contrast with a Gaussian contrast envelope (SD = 4°) and Brownian noise (1/f² spatial noise). In Experiment 1, subjects saw a Gabor patch traveling behind an occluder (26° high, 13.25° wide; Figure 1a). The Gabor always appeared on the left side of the screen and traveled toward the right side of the screen. The orientation of each Gabor that appeared (both before entering the occluder as well as after exiting the occluder) was randomly selected from a uniform distribution with possible orientations of 1° to 180°. Thus, the orientation of the Gabor patch could be very similar or very different when it reemerged from the occluder. The Gabor patch traveled across the screen at a speed of 19.9°/s. 

Participants

All experimental procedures were approved by the UC Berkeley Institutional Review Board and were in accordance with the Declaration of Helsinki. Participants were affiliates of UC Berkeley and provided written informed consent before participation. A total of 13 subjects (six female) participated in this experiment, ranging in age from 19 to 37 years (M = 27.5, SD = 5.4). The partially collected data from two participants was discarded because they were not available to complete the full experiment, resulting in a total of 11 subjects in Experiment 1. All participants had normal or corrected-to-normal vision, and all except one were naïve to the purpose of the experiment. To determine our sample sizes, we estimated the anticipated effect size from previously reported data (Fischer & Whitney, 2014) for spatially and temporally separated Gabors. For these experiments, our anticipated effect size (Cohen's d) for our Experiment 1 trial conditions was d = 3, which required a minimum of six subjects for a power of 0.8 and probability level of p = 0.05 (one-tailed). We stopped data collection once we reached twice the minimum number of subjects (12) but collected data from one additional subject due to subject exclusions. 

Procedures

In this experiment, subjects were experienced psychophysical observers and were explicitly encouraged to maintain fixation on a central point while their head was stabilized in a chin rest as they viewed Gabor stimuli. They saw a Gabor patch appear either in the upper left-hand corner, lower left-hand corner, or middle left edge of the screen (Figure 1). This first Gabor patch (entering Gabor) had a random orientation (1°–180°) and traveled at a fixed speed across the screen for 570 ms until 50% of the Gabor was occluded. The Gabor then traveled behind the occluder for 871 ms. A second Gabor patch (exiting Gabor) with a new, random orientation then exited from the opposite side of the occluder and traveled for 320 ms (beginning when 50% of the Gabor was again visible) to the right edge of the screen, where it stopped and disappeared. It was replaced by a 1000-ms random noise patch to reduce any afterimages. The exiting Gabor moved away from the occluder in a trajectory that was either consistent with the path of the entering Gabor (continuous straight moving trajectory, Figure 1b) or along a path that was not consistent with the entering Gabor (discontinuous moving trajectory, Figure 1c). Subjects saw a total of three possible continuous trajectories: The Gabor moved horizontally across the screen and behind an occluder along the top, middle, or bottom of the screen. We generated the discontinuous trajectory by randomizing the location of the exiting Gabor to be one of the possible alternate trajectories that were not continuous with the initial entering Gabor. For example, a (discontinuous) Gabor whose initial position on the screen was in the top left corner would exit the occluder from either the middle or bottom of the screen. 

There were three randomized trial types in this experiment: continuous moving trials (40% of trials), discontinuous moving trials (40% of trials), and catch moving trials (20% of trials). For each noncatch trial, subjects responded by using the keyboard to adjust a randomly oriented rectangular bar (0.24° by 4°, 10.2 cd/m²) at fixation to match the perceived orientation of the exiting Gabor patch. After adjustment, the rectangular bar was replaced by a 1500-ms noise patch to reduce any afterimages. Participants then saw a fixation dot for 1000 ms before beginning the next trial. During the catch moving trials, subjects only saw the entering Gabor and had to respond to its orientation without ever seeing an exiting Gabor. The purpose of these surprise catch trials was to make sure subjects attended to the orientation of both the entering and exiting Gabors because serial dependence has been found to require and be gated by attention (Fischer & Whitney, 2014). The entering Gabor was on the screen longer than the exiting Gabor to further facilitate subjects' attention to that Gabor. Subjects completed two sessions of 350 trials each. We predicted that the amplitude of the serial dependence effect should be higher for continuous versus discontinuous trials. 

Data analysis

For both continuous and discontinuous moving trials, response error was computed as the difference between the response orientation and the physical orientation of the exiting Gabor. Response error was then compared to the difference between the exiting and entering Gabor orientations. Positive values on the abscissa indicate that the entering Gabor was more clockwise than the exiting Gabor, and positive errors on the ordinate indicate that the reported orientation was more clockwise than the actual orientation of the exiting Gabor. For each subject's data, trials were considered lapses and excluded if error exceeded 3 SD from the mean (less than 2% of data excluded on average) or if the response time (RT) was longer than 10 s. 

Previous papers have characterized the strength and tuning of serial dependence using a derivative of a Gaussian function (Fischer & Whitney, 2014; Liberman et al., 2014), and we have used the same fitting procedures here. We fit a simplified Gaussian derivative using constrained nonlinear minimization (Nelder-Mead) to each subject's data for continuous and discontinuous trials separately using the following equation:  

  .

 

Parameter y is response error in each trial (response Gabor orientation − exiting Gabor orientation); x is the difference between the entering and exiting Gabor orientation; a is half the peak-to-trough amplitude of the derivative-of-Gaussian (DoG); b scales the width of the Gaussian derivative; and the constant, 2.33, scales the curve to make the a parameter equal to half the peak-to-trough amplitude. We used the a parameter in the above equation from each subject's data as a metric of serial dependence: the degree to which subjects' reports of orientation of the exiting Gabor were pulled in the direction of the entering Gabor (Figure 2a). If a subject's perception of orientation were repelled by the entering Gabor (e.g., because of a negative tilt aftereffect; Campbell & Maffei, 1971) or not influenced by the entering Gabor, then the half-amplitude of the best-fitting Gaussian derivative should be negative or close to zero, respectively. A positive value for the a parameter indicates a perceptual bias toward the orientation of the entering Gabor. The amplitude (a parameter) and tuning (b parameter) of the DoG curve also indicate that serial dependence is tuned to the similarity between successive orientations such that if the current and previous Gabor have radically different orientations, no bias in perception occurs. We then ran a two-tailed paired Student t test to compare the amplitude of serial dependence for continuous (Figure 2a) and discontinuous (Figure 2b) moving trials across all subjects. Effect sizes for within-subject comparisons were calculated using Cohen's d_av (Lakens, 2013). We used a false discovery rate (FDR) procedure to correct for multiple comparisons (Benjamini & Hochberg, 1995). 

Figure 2

 

Experiment 1 results. (a) Example data from a representative subject for all continuous moving trajectory trials. The DoG (solid black line; ) was fit to the entire range of the data. For this DoG function, a = 2.93 (half the peak-to-trough amplitude) and b = 0.043, which scales the width of the Gaussian derivative. Smaller values of b result in a wider Gaussian derivative. (b) Example data from the same representative subject for all discontinuous moving trajectory trials. For this DoG function, a = −0.37 and b = 0.015. (c) Average amplitude of serial dependence across 11 subjects for continuous and discontinuous trials. Error bars are SEM. Serial dependence was significantly stronger when the object moved along a continuous versus discontinuous trajectory behind the occluder, t(10) = 4.69, p = 0.003, d_av = 0.92 (FDR-corrected, two-tailed, paired t test). (d) Subject performance (response error in degrees) in catch trials and noncatch trials. Subjects were more accurate in responding to the orientation of the noncatch trial Gabors yet still showed a high level of accuracy in catch trials even though these trials accounted for a random and surprise 20% of responses.

Results

The half-amplitude of serial dependence was significant for continuous motion trials, t(10) = 4.69, p = 0.003, d = 1.41 (continuous trials, FDR-corrected, two-tailed t test), despite the fact that the sequential Gabor patches were separated in space, which normally reduces serial dependence (Fischer & Whitney, 2014). There was a positive but not a significant serial dependence for discontinuous trials, t(10) = 2.05, p = 0.11, d = 0.62 (discontinuous trials, FDR-corrected, two-tailed t test), as predicted by the temporal and spatial tuning of the CF (Fischer & Whitney, 2014). Most importantly, there was a significantly larger serial dependence amplitude for the continuous compared to the discontinuous moving trials (Figure 2c), t(10) = 4.69, p = 0.003, d_av = 0.92 (FDR-corrected, two-tailed, paired t test). The perceived orientation of the exiting Gabor was pulled toward the orientation of the entering Gabor, and this effect was significantly stronger when the object moved along a continuous trajectory behind the occluder. This result suggests that object continuity modulates serial dependence in orientation perception: The perceptual pull from previous stimuli was enhanced when the drifting Gabor was perceived as traveling along a continuous, predictable path. 

Average subject RT across all trials was 1828 ms (SD = 910 ms). There was no correlation between the average RT for each subject and the amplitude of serial dependence for the continuous (r = −0.05, p = 0.89) or discontinuous trials (r = −0.33, p = 0.33). 

Experiment 2: Is motion necessary for perceived perceptual continuity?

In Experiment 1, we found that serial dependence helps promote spatiotemporal continuity by making objects that enter and exit from behind an occluder in a predictable way appear more similar than they truly are. Because serial dependence also pulls the apparent orientation of sequentially presented static objects (Fischer & Whitney, 2014), we asked whether motion that helps subjects infer continuity was, in fact, important for the exiting Gabor's orientation to be perceptually pulled toward the entering Gabor. To test this, we presented subjects with a series of static, randomly oriented Gabor patches on either side of an occluder (Figure 3). Because there were fewer continuity cues for the static Gabors, we predicted that there would not be significant serial dependence when objects did not move along a trajectory. 

Figure 3

 

Experiment 2 stimuli and procedure. (a) Example aligned static trajectory trial sequence for a Gabor with a starting position at the top left of the screen. Subjects responded by adjusting a rectangular bar (0.24° × 4°) at fixation to match the perceived orientation of the exiting Gabor patch. (b) Example aligned static trial for a Gabor with a starting position in the middle of the screen. (c) Example misaligned static trial for a Gabor with a starting position in the bottom left of the screen.

Methods

Stimuli

In Experiment 2, subjects saw two stationary Gabors sequentially appear on either side of an occluder (they did not move). The Gabor stimuli had all of the same properties as the Gabor stimuli in Experiment 1, but they were stationary. The occluder was the same size and in the same location as in Experiment 1 (Figure 3a). Subjects reported the perceived orientation of the last Gabor they saw by adjusting the orientation of a response bar shown at a central fixation point (as in Experiment 1). Catch trials were also included: In 80% of the trials, the last seen Gabor was the nominally “exiting” Gabor (although it did not move), and in the other 20% of trials, randomly interleaved, the last seen Gabor was the nominally “entering” Gabor (although it did not move). As in Experiment 1, the orientation of the response bar was initially random in each trial, and subjects used a keyboard for all responses. 

Participants

All experimental procedures were approved by the UC Berkeley Institutional Review Board and were in accordance with the Declaration of Helsinki. Participants were affiliates of UC Berkeley and provided written informed consent before participation. A total of 12 subjects (six female) participated in this experiment, ranging in age from 19 to 37 years (M = 25.9, SD = 4.9). Eleven of the subjects also participated in Experiment 1, and one subject's data was discarded because the subject was not available to complete the full experiment. All participants had normal or corrected-to-normal vision, and all except one were naïve to the purpose of the experiment. 

Procedures

The stimuli and procedures were identical to those in Experiment 1 except that the Gabors remained stationary throughout all trials. The Gabor first appeared centered halfway between the left side of the screen and the closest edge of the occluder to try to match the average location of the moving Gabor. It stayed in the same initial location without moving for 610 ms (for consistency, we will still refer to this as the “entering” Gabor although it did not actually move or display any accretion cues). It then disappeared for an 885-ms interstimulus interval. Another static Gabor (akin to the “exiting” Gabor) then appeared centered between the right edge of the occluder and the right side of the screen for 360 ms before being replaced by a 1000-ms noise patch (Figure 3a). The possible locations for the Gabor patches were based on the locations used for the drifting patches in Experiment 1; the “entering” Gabor could be in any one of the three horizontal locations as the entering Gabor in Experiment 1, and the “exiting” Gabor could be in any one of three horizontal locations of the exiting Gabor in Experiment 1. The “exiting” Gabor patch in this experiment was shown in a location that was either analogous to a continuous trajectory in Experiment 1 (aligned static trials; Figure 3b) or analogous to a discontinuous trajectory in Experiment 1 (misaligned static trials; Figure 3c). Subjects completed two sessions of 350 trials each. 

Data analysis

All data analysis procedures for Experiment 2 were identical to Experiment 1. We compared response error to the difference between the exiting and entering Gabor orientations. We used the same curve fitting method as in Experiment 1 to determine whether the exiting Gabor was perceptually pulled toward the entering Gabor's orientation while they both remained stationary. 

Results

We compared the half-amplitude of serial dependence for trials in which subjects saw the second static Gabor in a horizontally aligned versus misaligned location on the screen (Figure 4). Neither the aligned nor misaligned trials showed a significant serial dependence effect, t(10) = 1.93, p = 0.11, d = 0.58, aligned trials; t(10) = 1.33, p = 0.24, d = 0.4, misaligned trials (FDR-corrected, two-tailed t test), which is not surprising given their substantial spatial separation (Fischer & Whitney, 2014). However, they did show an obviously positive trend. The half-amplitude of serial dependence was not significantly different for the aligned compared to the misaligned static trials (Figure 4c), t(10) = 0.36, p > 0.250, d_av = 0.17 (FDR-corrected, two-tailed, paired t test). More importantly, the amplitude of serial dependence was significantly stronger in the continuous moving trials (Experiment 1) compared to either the aligned static or misaligned static trials (Figure 4c), t(10) = 2.68, p = 0.046, d_av = 1.07, continuous versus aligned static; t(10) = 2.88, p = 0.044, d_av = 1.2, continuous versus misaligned static (FDR-corrected, two-tailed, paired t test). 

Figure 4

 

Experiment 2 results. (a) Example data from a representative subject for all aligned static trajectory trials. (b) Example data from the same representative subject for all misaligned static trajectory trials. (c) Amplitude of serial dependence compared across both experiments. Error bars are SEM. Serial dependence was significantly stronger when the object moved along a perceived continuous trajectory versus a discontinuous trajectory or static presentation, t(10) = 4.69, p = 0.003, d_av = 0.92, continuous versus discontinuous; t(10) = 2.68, p = 0.046, d_av = 1.07, continuous versus aligned static; t(10) = 2.88, p = 0.044, d_av = 1.2, continuous versus misaligned static (FDR-corrected, two-tailed, paired t test).

Average subject RT across all trials was 1802 ms (SD = 959 ms) for this experiment. There was no significant correlation between subject RT and the amplitude of serial dependence for the aligned static (r = −0.13, p = 0.7) or misaligned static trials (r = 0.44, p = 0.18). Furthermore, we can infer that subjects were maintaining fixation throughout all the trials of both experiments. If subjects were tracking or making saccades to the Gabor stimuli within a trial, then we would predict equally high amplitudes of serial dependence across all conditions in both experiments because the Gabors within a trial would be retinotopically overlapping (Fischer & Whitney, 2014). 

General discussion

The CF is a mechanism proposed to facilitate the perceptual stability of objects (Fischer & Whitney, 2014; Liberman et al., 2014). It does so through serial dependence, which is a systematic bias in the appearance of an object's features and identity toward similar objects seen in the last several seconds. Our results demonstrate that serial dependence in orientation perception operates across interruptions in visibility. Importantly, this serial dependence was stronger for continuously moving trajectories compared to unexpected trajectories or static displacements, reflecting expectations of a stable and predictable world. 

Several alternative explanations for our results can be ruled out. A generalized response bias or motor serial dependence would not predict the spatiotemporal tuning of the serial dependence we report (Luce & Green, 1974; Tanner, Rauk, & Atkinson, 1970; Wiegersma, 1982a, 1982b) because subjects only responded to one of the two Gabors shown in each trial. Additionally, static displacements produced little serial dependence in orientation perception; continuous and predictably moving objects resulted in significantly stronger serial dependence. The relatively small serial dependence in the static conditions is consistent with previous results that characterized the spatial tuning of the CF (Fischer & Whitney, 2014); orientation serial dependence decreases as the spatial distance between Gabors increases. Our results are therefore consistent with previous findings and inconsistent with motor and generalized response biases. 

Adaptation and associated negative aftereffects, priming, and other phenomena show a type of perceptual dependence on the recent past yet remain distinct from serial dependence and the CF. Adaptation studies show that prior exposure to a variety of stimulus features (Anstis, Verstraten, & Mather, 1998; Campbell & Maffei, 1971; M. A. Webster & Mollon, 1991; M. A. Webster, Kaping, Mizokami, & Duhamel, 2004) results in a stimulus-specific negative aftereffect or perceptual repulsion away from the adapting stimulus (for reviews, see Thompson & Burr, 2009; M. Webster, 2012). However, our experiments show a positive perceptual pull toward the recent past and are therefore not a result of known forms of adaptation. Of course, positive serial dependence and negative orientation aftereffects (e.g., the tilt aftereffect) do occur at the same time (Fischer & Whitney, 2014, supplemental material), but the brevity of the oriented Gabors and the long temporal interval between stimuli in our experiments may minimize the negative aftereffects. With different exposure duration and timing, the negative (tilt aftereffect) and positive (serial dependence) might combine in different ways. 

Our proposed experiments may be related to perceptual priming effects (Faivre & Kouider, 2011; Kristjánsson, Ingvarsdottir, & Teitsdottir, 2008; Kristjánsson, Bjarnason, Hjaltason, & Stefánsdóttir, 2009; Maljkovic & Nakayama, 1994, 1996), but there are important differences. Priming generally manifests in reaction time (Maljkovic & Nakayama, 1994, 1996) and, when relevant, can improve discriminability of primed stimuli (Sigurdardottir, Kristjánsson, & Driver, 2008); serial dependence does not impact reaction time and is a reduction in the discriminability of objects for perceptual stability (Fischer & Whitney, 2014; Liberman et al., 2014). The CF is a spatiotemporal operator that affects appearance: It makes (even slightly different) objects look the same over time. The CF is one mechanism (of potentially many) that could generate effects that may appear similar to priming or fall under the umbrella of “priming,” but it is distinct from previously reported priming effects. This is not to say that priming, adaptation, and serial dependence are unrelated, but they are distinguishable, and the complementary roles they play in perceptual stability remain a rich area of investigation. 

Previous researchers have suggested that object files (Kahneman, Treisman, & Gibbs, 1992; Treisman, 1988) or object tokens (Chun & Cavanagh, 1997; N. Kanwisher & Driver, 1992; N. G. Kanwisher, 1987) maintain perceptual stability by using the perceived spatiotemporal continuity of objects as they move, change, or have disruptions in visibility. However, object tokens or files are a conceptual and not an algorithmic or mechanistic description of the ability to track objects during occlusion. They may contribute to object permanence (Kahneman et al., 1992), but they do not make a clear prediction about what objects should look like after emerging from dynamic occlusion or whether there should or should not be serial dependence. 

Furthermore, object file effects are mainly found in RT improvements, similar to priming (Kahneman et al., 1992). We did not find any significant RT effects in our data, which we tested by fitting both a Gaussian and a straight line to normalized trial RTs (y-axis) relative to the difference between the first and second Gabor orientation (x-axis) for continuous moving trajectory trials. We tested model significance using the Akaike information criterion (AIC; Akaike, 1974) and found that the straight line had the lowest AIC in 9/11 subjects, indicating no RT improvements for trials with the largest serial dependence effects. 

Object tokens are also often invoked in studies of change detection through occlusion (Flombaum & Scholl, 2006; Flombaum et al., 2008). Our measured serial dependence appears to be inconsistent with those predictions (or findings) of change detection experiments (Flombaum & Scholl, 2006; Flombaum et al., 2008) at least until one considers the feature tuning of the CF (Fischer & Whitney, 2014; Liberman et al., 2014). Experiments that employ change blindness as a tool typically use sequential objects that are very distinct (Flombaum & Scholl, 2006) with which little or no serial dependence would be expected. In contrast, serial dependence is strongest (represented as a ratio of the difference between the sequential objects) for similar objects. Serial dependence should therefore increase change blindness but only for similar objects. Perhaps counterintuitively, the change detection results (Flombaum & Scholl, 2006; Flombaum et al., 2008) and our serial dependence through occlusion are complementary findings. By introducing serial dependence only for similar objects and improved change detection of very different objects, the visual system simultaneously maximizes the appearance of consistency for similar objects and minimizes the likelihood of confusion between very different objects. 

Our results rely on subjects attending to and encoding the stimuli, but explicit memory or recall is not required for serial dependence (Fischer & Whitney, 2014). Unlike change detection tasks and other working memory experiments, our task does not require an explicit comparison between the first and second display; it does not tax memory and thus may not involve the same working memory demands. In our task, the catch trials ensured that subjects attended to each Gabor patch, but explicitly reporting the orientation of those catch trials was not necessary for serial dependence. Furthermore, Liberman et al. (2014) demonstrated that memory confusion or subjects mistakenly reporting the previous stimulus rather than the current stimulus occurs infrequently and cannot fully explain the magnitude of serial dependence effects. We are not ruling out the contribution of attention and implicit memory to perceiving and recognizing the oriented Gabors, but explicit memory or memory confusion cannot fully explain our results. 

Our results do not speak to the question of whether serial dependence is a “perceptual” or a “decision”-level effect. However, there are reasons to be cautious about the question itself and the validity of the distinction between perception and decision in the context of serial dependence. Serial dependence is tuned to feature similarity, space, and time, and it does not require a sequential comparison or explicit memory (e.g., the 2AFC experiments of Cicchini et al., 2014; Fischer & Whitney, 2014; Liberman et al., 2014; Taubert, Alais, & Burr, 2016a; Taubert et al., 2016b), and it can generate a visual illusion (Fischer & Whitney, 2014). It also does not require an explicit or overt response (Fischer & Whitney, 2014; Liberman et al., 2014). A decision-level effect would therefore need to be tuned in all of these respects as well and not require an overt response. This is entirely possible; perhaps decisions are implicit and tuned to feature, object, and spatial dimensions. However, what counts as a “decision” would then be tantamount to what we would normally consider “perception,” and the distinction holds little value. Future work will no doubt attempt to reconcile this issue. 

There are existing frameworks and models for characterizing serially dependent perception and the CF. Bayesian accounts can, of course, provide an algorithmic description of perceptual serial dependence with great accuracy (Cicchini et al., 2014; Jazayeri & Shadlen, 2010). Testing or falsifying these models proves more difficult, however, as a Bayesian model would easily accommodate our results regardless of the role of spatiotemporal continuity. Perhaps a better test of this class of models will involve manipulating noise, but that is beyond the scope of the present experiments. With flexible and dynamic criterion settings, models based on signal detection theory (SDT) should, in principle, also be able to characterize our findings. Other models, such as a population code model (Fischer & Whitney, 2014), are complementary to the Bayesian and SDT accounts although at a different level of description. Our results do not address these models but do suggest that any model of the CF and the associated serially dependent perception should take into account the important role that spatiotemporal continuity and expectation play. 

Conclusion

Our results suggest that the CF is tuned to the spatiotemporal predictability of dynamic objects for the purpose of perceptual stability. The CF is highly tuned to continuous and expected object trajectories, mirroring the predictability of objects in the world and reflecting expectations of stability. The CF therefore provides a key mechanism that could support perceptual stability during object tracking and while objects are occluded. 

Acknowledgments

This work was supported by NIH EY018216 to DW, a Kirschstein-NRSA under Grant No. EY025942 and NSF GRFP under Grant No. 1106400 to AL, and by the Berkeley Research Impact Initiative (BRII) sponsored by the UC Berkeley Library to KZ. 

Commercial relationships: none. 

Corresponding author: Alina Liberman. 

Email: [e-mail]. 

Address: Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA, USA.