An object-mediated updating account of insensitivity to transsaccadic change

Abstract

Abstract

Abstract:

Abstract  Recent evidence has suggested that relatively precise information about the location and visual form of a saccade target object is retained across a saccade. However, this information appears to be available for report only when the target is removed briefly, so that the display is blank when the eyes land. We hypothesized that the availability of precise target information is dependent on whether a post-saccade object is mapped to the same object representation established for the presaccade target. If so, then the post-saccade features of the target overwrite the presaccade features, a process of object mediated updating in which visual masking is governed by object continuity. In two experiments, participants' sensitivity to the spatial displacement of a saccade target was improved when that object changed surface feature properties across the saccade, consistent with the prediction of the object-mediating updating account. Transsaccadic perception appears to depend on a mechanism of object-based masking that is observed across multiple domains of vision. In addition, the results demonstrate that surface-feature continuity contributes to visual stability across saccades.

Introduction

When exploring a visual scene, we frequently move our eyes to obtain high-resolution information from individual objects. Each saccadic eye movement introduces a disruption in perceptual input, as visual perception is suppressed during saccades (Matin, 1974). In addition, the retinal locations of objects shift as the eyes rotate to change the point of regard. These natural consequences of a foveal visual system and a moveable eye create a problem of object correspondence and continuity that has been one of the central areas of research in vision science. How are objects perceived as continuous across the retinal shifts and perceptual disruptions introduced by saccades? More generally, how is the external world perceived as stable despite the changes and discontinuity in perceptual input generated during natural vision? 

In the present study, we focused on a particular instantiation of this general problem: How is an object at one retinal location perceived as the same object when it appears at a different retinal location following an eye movement? Establishing object correspondence across saccades is a process that requires close integration of perceptual-motor processes and memory; the gap in perceptual input requires a brief form of memory to retain features of the scene across the saccade. Several researchers have proposed that memory across saccades, or transsaccadic memory, depends on visual working memory (Henderson, 1994; Hollingworth & Luck, 2009; Hollingworth, Richard, & Luck, 2008; Irwin, 1992a, 1992b; Irwin & Andrews, 1996). In particular, visual properties of the saccade target object are thought to be encoded into visual working memory (VWM) before the saccade. When the eyes land, the remembered properties are used to establish that the object now at (or near) the fovea is indeed the intended saccade target (Bridgeman, Van der Heijden, & Velichkovsky, 1994; Currie, McConkie, Carlson-Radvansky, & Irwin, 2000; Deubel, Schneider, & Bridgeman, 1996; Hollingworth et al., 2008; McConkie & Currie, 1996). 

The content and precision of transsaccadic memory has generated considerable debate, however. Several studies have demonstrated that observers fail to detect surprisingly large spatial displacements of the saccade target during eye movements (Bridgeman, Hendry, & Stark, 1975; Bridgeman & Stark, 1979), suggesting that transsaccadic memory for the location of the saccade target is highly imprecise. Evidence that sensory-level shape features are not integrated across saccades (Bridgeman & Mayer, 1983; Irwin, 1991; Irwin, Yantis, & Jonides, 1983; O'Regan & Lévy-Schoen, 1983) has also been taken to indicate that transsaccadic memory is highly abstract (Irwin, 1992a), is more conceptual than perceptual (Mathot & Theeuwes, 2011), or is nonexistent (O'Regan, 1992). 

After initial evidence of poor sensitivity to spatial displacement across saccades, subsequent studies have indicated substantially more precise memory than had been previously assumed. The bulk of this evidence comes from a striking finding by Deubel et al. (1996). They modified the target displacement paradigm of Bridgeman et al. (1975) to include a blank period immediately following the saccade to the target. A small target dot was moved horizontally from central fixation to the periphery, and participants executed a saccade to the dot. As in the standard displacement paradigm, the dot was shifted a second time during the saccade itself (when vision is suppressed). Participants reported the horizontal direction of the second shift. In the blank condition, the target was removed during the saccade and was absent for a short period of time when the eyes landed (blank period). The eyes landed on an empty display before the onset of the displaced target. 

With no blank, displacement discrimination was poor even for displacements as large as 2°, replicating Bridgeman et al. (1975). On blank trials, however, participants were quite sensitive to shifts as small as 0.3°, despite the fact that the functional retention interval (delay between saccade initiation and presentation of the displaced dot) was longer in the blank condition than in the no-blank condition. Thus, precise spatial information was preserved across saccades, but it was available for comparison and report only when the screen was blank upon landing. 

Work on transsaccadic memory for object form has followed a similar path. A growing body of evidence indicates that the form information retained across a saccade can be quite precise. Perceptual processing of the target after the saccade is impaired if it is replaced by another object from the same basic-level category (Henderson & Siefert, 2001; Pollatsek, Rayner, & Collins, 1984) or if it is mirror reversed (Henderson & Siefert, 1999); visual changes to saccade targets can be detected quite accurately under some conditions (Henderson & Hollingworth, 1999, 2003); and feature-specific priming (Wittenberg, Bremmer, & Wachtler, 2008) and adaptation (Melcher, 2007; Zirnsak, Gerhards, Kiani, Lappe, & Hamker, 2011) are transferred across saccades. Most strikingly, recent evidence using the Deubel et al. (1996) blanking paradigm has reestablished the possibility that low-level sensory persistence is retained across saccades (Germeys, De Graef, Van Eccelpoel, & Verfaillie, 2010b). On blanking trials of the Germeys et al. (2010) experiments, transsaccadic memory for object form exhibited high capacity, rapid decay, and sensitivity to pattern masking, hallmark properties of sensory persistence. 

The remarkable increase in visual sensitivity generated by the blanking method reveals that precise information about location and form can be retained across saccades. But this raises an important question that has not been resolved since the original findings by Deubel et al. (1996). Why, under normal conditions, do people fail to access this information when making discriminations across saccades? Deubel et al. (1996) argued that the visual system is strongly biased to assume that the world remains stable across saccades because transsaccadic discrepancies are much more likely to be generated by motor error (e.g., by an inaccurate saccade) than by an abrupt change in the world during the 20–40s ms that the eyes happen to be in motion. Therefore, discrepancies are typically attributed to internal error rather than to external change. Screen blanking provides unambiguous evidence of external change, contradicting this assumption of stability, and once contradicted, participants can access precise information about target location. 

This account provides an explanation for general insensitivity to change, but it does not provide an explanation of how the precise information about the target is lost or becomes inaccessible. We propose that the inability to access precise target information under normal conditions can be explained by object-mediated updating (Enns, Lleras, & Moore, 2009; C. M. Moore & Enns, 2004; C. M. Moore & Lleras, 2005; C. M. Moore, Mordkoff, & Enns, 2007), a form of object-based masking. Object-mediated updating is the process by which, in the face of perceptual change or discontinuity, the visual system either updates an existing object representation or establishes a new object representation. Specifically, if newly sampled information from a stimulus allows an object to be considered as the continuation of a previously visible object, the two states will be mapped to a single persisting object representation. That representation will be updated to reflect the newly observed properties, and the original properties will be overwritten, at least to some extent. However, if newly sampled information is not consistent with the interpretation of a single persisting object, a second object representation will be established, the newly observed features will be assigned to that second object, and the features of the original object will not be replaced. This approach accounts for several object-mediated masking phenomena, including motion de-blurring, the flash-lag illusion, and object substitution masking. (We discuss these phenomena and their relationship to transsaccadic persistence in the General discussion). 

In the case of insensitivity to transsaccadic displacement (e.g., Bridgeman et al., 1975), we assume that there is indeed a high threshold for attributing a transsaccadic discrepancy to a change in the world (Deubel et al., 1996). The target shift typically is not sufficient for the post-saccade object to be perceived as a different object than the presaccade object (unless the shift exceeds about 1/3 of the distance of the saccade itself). The pre and post-saccade objects will be mapped to the same object representation, and the post-saccade location will overwrite memory for the presaccade location, limiting comparison of the two and generating poor sensitivity to displacement. In the blank condition, however, the visual system has unambiguous evidence for a discontinuity in objecthood, as no object is visible when the eyes land. The post-saccade target object is represented as a new object, and memory for the location of the presaccade object is not overwritten. Consequently, the spatial location of the presaccade object can be accurately compared with the location of the post-saccade object. 

A key assumption of the object-mediated updating hypothesis is that multiple sources of information are consulted in the interpretation of object continuity (C. M. Moore et al., 2007), including spatiotemporal properties of the object (position over time), surface features of the object (such as luminance, color, or texture), visual context, and the history of the object. In particular, object continuity can be disrupted by introducing a change in the object's surface feature properties even when the spatial properties of an object are consistent with the interpretation of continuity (Feldman & Tremoulet, 2006; Hollingworth & Franconeri, 2009; Hollingworth et al., 2008; C. M. Moore & Enns, 2004; C. M. Moore et al., 2007; C. M. Moore, Stephens, & Hein, 2010; Richard, Luck, & Hollingworth, 2008; Tas, Dodd, & Hollingworth, 2012). This contrasts with prominent theories of object correspondence (Flombaum, Scholl, & Santos, 2009; Kahneman, Treisman, & Gibbs, 1992; Pylyshyn, 2000), which claim that spatiotemporal features of an object are the dominant, or even the sole, form of information used to establish continuity across perceptual disruption. 

In the present study, we tested the hypothesis that the blanking effect reflects object-mediated updating. According to this account, any manipulation that causes the post-saccade target to be treated as a new object should limit overwriting of the presaccade object and increase sensitivity to displacement. Such an effect should not be limited to blanking. Evidence consistent with this prediction was recently reported by Demeyer, De Graef, Wagemans, and Verfaillie (2010), although they did not interpret it in these terms. In their experiments, changing the shape of the target object improved sensitivity to spatial displacement relative to a no-change control condition. From the current perspective of examining surface feature correspondence, however, a limitation of the Demeyer et al. (2010) study is that changes in shape features necessarily created changes in the spatial properties of the objects. This makes their method less than ideal for isolating the role of surface features. In addition, shape changes were quite subtle and generated an increase in sensitivity that was relatively small compared with a blanking condition. 

In Experiment 1 of the present study, we kept the shape of the target constant while manipulating only one surface feature property of that object: contrast polarity. To maximize the potential effect of surface feature change, in Experiment 2 we kept the global shape of the target constant while replacing one round, natural object (e.g., basketball) with another (e.g., medal). This manipulation modified multiple surface features of the target as well as its identity. 

To preview the results, we found that changes limited to the surface features of the saccade target significantly increased sensitivity to spatial displacement, consistent with the prediction of the object-mediated updating hypothesis. In addition, changing multiple surface features of the object in Experiment 2 led to an increase in sensitivity that rivaled the increase found in a blanking condition. We discuss these results within the larger theoretical context of object correspondence and object-mediated updating, providing a general account of updating effects that spans multiple domains of vision. In addition, we discuss the implications of the present results for theories of transsaccadic stability. 

Experiment 1

Experiment 1 employed a transsaccadic displacement task modeled on Deubel et al. (1996). A target dot was presented at fixation. The first displacement varied between 6° and 8° on the horizontal axis, either to the left or right, and participants executed a saccade to the target dot. The target dot was displaced horizontally a second time, either in the same direction (forward) or in the opposite direction (backward) as the first displacement. In the no-blank condition, this second displacement occurred during the saccade to the target, so that the displaced target was visible when the eyes landed. In the blank condition, the target was removed for 250 ms (blank) before the displaced target was presented so that no object was visible when the eyes landed. Finally, the polarity-change condition was identical to the no-blank condition, except the contrast polarity of the displaced target was reversed. Any difference in sensitivity to displacement in the no-blank and polarity change conditions can be attributed to the surface feature change. The inclusion of all three conditions allowed us to assess the effect of polarity change relative to the no-blank condition and the magnitude of the polarity change effect relative to the blank condition. 

Method

Participants

Seven participants from the University of Iowa community, 18–30 years of age, completed the experiment for monetary compensation. They completed three 1-hr sessions which were separated by at least one day. All participants reported normal or corrected-to-normal vision. 

Stimuli, apparatus, and procedure

Stimuli were displayed on a 17-inch CRT monitor at a refresh rate of 120 Hz. The position of the right eye was monitored by an SR Research Eyelink 1000 eye tracker sampling at 1000 Hz. A chin and forehead rest was used to ensure a 70-cm viewing distance and to minimize head movements. Stimulus presentation was controlled by E-prime software (Schneider, Eschmann, & Zuccolotto, 2002). 

Stimuli were presented against a gray background. The target stimulus was a disk subtending 0.33°. At the beginning of the trial, the target appeared at central fixation for a randomly determined duration between 500 and 1000 ms (See Figure 1). On half of the trials, the initial disk was white, and on the other half it was black. Next, the target disk was displaced to the left or right. The magnitude of the displacement varied randomly within a range between 6° and 8°. Participants were instructed to execute a saccade to the shifted disk immediately. The screen was changed during the saccade when the eye tracker detected that the eye crossed a 1° boundary defined to the left and right of central fixation. Mean time from the beginning of the saccade to the beginning of the screen change was 15 ms. The screen change was completed within a maximum of 8 ms. Mean time from initiation of the screen change to the end of the saccade was 27 ms. Thus, the change was completed well before the saccade ended. In the no-blank condition, the screen change introduced a second target displacement that was either in the same (forward) or in the opposite (backward) direction as the initial displacement. The size of the second displacement varied between 0° and 1.5° in steps of 0.5°. In the polarity-change condition, the transsaccadic displacement was identical, except the contrast polarity of the target disk was also changed (black to white or white to black). In the blank condition, the target was removed during the saccade, and the screen remained blank for 250 ms, after which the displaced target was displayed. Contrast polarity did not change. 

Figure 1

 

The sequence of events for the three main conditions in Experiment 1. Stimuli are not drawn to scale. The eye icon illustrates horizontal eye position.

In each case, the participant's task was to indicate the second shift direction (forward or backward) via a button response on a serial response pad. To avoid motor-compatibility effects, the two buttons were arranged perpendicular to the stimulus plane. The button closer to the participant was mapped to “backward,” and the further button was mapped as “forward.” Participants used the index finger and thumb of the right hand to respond. 

Upon arriving for the experiment, each participant provided informed consent and was instructed about the requirements of the task. The eye tracker was calibrated. Calibration was checked every 15–20 trials during the experiment, and the participant was recalibrated if position error exceeded approximately 1°. The experimenter initiated each trial. In each of the three sessions, participants first completed 12 practice trials drawn randomly from the full design. They then completed 630 experimental trials, 15 in each of the 42 conditions created by the 3 (no-blank, blank, polarity-change) × 7 (second displacement distance) × 2 (initial target black, white) design. Trials from all conditions were randomly intermixed. Across three sessions, participants completed a total of 1,890 experimental trials. 

Results

Eye-tracking data were analyzed offline. A combined velocity (>30°/s) and acceleration (>8000°/s²) threshold was used to define saccades. For all of the analyses, we first eliminated trials on which participants failed to maintain fixation on the screen center before the first displacement (11% of the data), trials on which the transsaccadic screen change was not completed before the eyes landed (1% of the data), and trials on which the latency of the saccade to the first displacement was less than 100 ms (<1% of the data) or longer than 400 ms (2.5% of the data). A total of 15% of the trials was eliminated from the analyses. 

Individual participant data are presented in Figure 2. Proportion forward response is plotted for each participant in each of the three conditions as a function of displacement. The proportion forward data were modeled as a two-parameter sigmoid function: 

 

where displ is the displacement size, X is the slope of the transition between “backward” and “forward” responses, reflecting sensitivity, and Y is the bias to respond either “backward” or “forward.” Parameter values for each participant were estimated by the maximum likelihood estimation procedure within a nonlinear mixed effects model of the data set (Pinheiro & Bates, 2000), which converged on the individual and group fits simultaneously. 

Figure 2

 

Proportion of forward responses as a function of displacement size for blank (A), no-blank (B), and polarity-change (C) conditions of Experiment 1. Positive displacement sizes represent displacements in the same direction as the initial displacement. Negative values represent displacements in the opposite direction as the initial displacement. Each line represents one participant.

The model fits for both sensitivity and bias parameters were significant, F(2, 121) = 19.9, p < 0.001 and F(2, 121) = 8.1, p = 0.005, respectively. Participants were more sensitive to position shifts in the blank condition (X = 3.42) than in both the polarity-change condition (X = 2.29) and the no-blank condition (X = 1.94), F(1, 81) = 20.2, p < 0.001 and F(1, 81) = 29.1, p < 0.001, respectively. Importantly, participants were significantly more sensitive to position shifts when the target's contrast polarity changed during the saccade compared when it did not (i.e., in the no-blank condition), F(1, 81) = 4.1, p = 0.04. 

Participants showed similar biases in the blank and polarity-change conditions (F < 1). The bias parameter estimate in the no-blank condition was significantly different from those in both the blank and the polarity-change conditions, F(1, 81) = 9.3, p = 0.003 and F(1, 81) = 8.8, p = 0.004, respectively. Thus, overall, participants showed a similar backward bias for the blank and polarity-change conditions, but they showed a forward bias for the no-blank condition. The source of this difference in bias is not entirely clear, although it could arise simply from the fact that the lower sensitivity to actual displacement in the no-blank condition provided greater opportunity for responses to be influenced by bias. 

In summary, the results of Experiment 1 showed that the introduction of a surface feature change during a saccade improved sensitivity to spatial displacements of that object, which is consistent with the object-mediated updating hypothesis. As in Germeys et al. (2010), the magnitude of the sensitivity increase was not as large as that produced by blanking of the target object. This is not particularly surprising, however, because blanking disrupts all features of the saccade target, whereas a contrast polarity change disrupts only one. 

Experiment 2

In Experiment 2, we used a set of photographs of round, natural objects as the saccade targets. A replacement of one object for another changed multiple surface features of the object as well as its identity. The object-change condition was compared with no-blank and blank conditions. 

Method

Participants

Seven new participants from the University of Iowa community, 18–30 years of age, completed the experiment for monetary compensation. They completed two 1-hr sessions which were separated by at least one day. All participants reported normal or corrected-to-normal vision. 

Stimuli and procedure

The procedure was the same as used in Experiment 1, except for the differences noted below. Instead of presenting a dot, photographs of five real-world objects were used as the stimuli (see Figure 3). All of the objects (basketball, button, clock, globe, and medal) were round, but they had different colors and surface patterns. The objects subtended 0.65° of visual angle. In order to eliminate an overlap between pre and post-saccadic object when there was a second displacement, we increased the size of the second displacement. Specifically, the target object was displaced 0°, 0.6°, 1.2°, or 1.8° either forward or backward during the saccade. 

Figure 3

 

The five object stimuli used in Experiment 2.

As in Experiment 1, there were three within-subject conditions: no-blank, blank, and object-change. Except for the use of photographs, the no-blank and blank conditions were the same as in Experiment 1. In the object-change condition, the target object was replaced with a different object selected randomly from the remaining four. Each object was presented as the target object equally often. Participants completed a total of 1260 experimental trials over two sessions. 

Results

Trials in which participants were not fixating the central target object at the time of first displacement (18% of the data), in which the second displacement occurred after the saccade (<1% of the data), or with saccade latencies shorter than 100 ms (<1% of the data) or longer than 400 ms (7% of the data) were eliminated from the analyses, resulting in exclusion of 26% of the trials. 

Figure 4 shows the proportion of forward responses for each displacement size and condition. The model fits for both the sensitivity and bias parameters were significant, F(2, 121) = 8.2, p < 0.001 and F(2, 121) = 5.6, p = 0.005, respectively. The blank condition produced higher sensitivity (X = 2.35) than the no-blank condition (X = 1.67), F(1, 81) = 10.5, p = 0.002. Replicating the results of Experiment 1, participants were significantly more sensitive to displacements for the object-change trials (X = 2.14) than for the no-blank trials, F(1, 81) = 14.1, p < 0.001. Importantly, there was no significant difference between blank and object-change conditions, F(1, 81) = 1.4, p = 0.25. 

Figure 4

 

Proportion of forward responses as a function of displacement size for blank (A), no-blank (B), and object-change (C) conditions of Experiment 2.

As in Experiment 1, participants showed similar backward biases in the blank and object-change conditions, F (1, 81) = 1.1, p = 0.30. The bias parameter estimate for the no-blank condition was significantly different from those in both the blank and object-change conditions, F(1, 81) = 6.1, p = 0.02 and F(1, 81) = 9.1, p = 0.003, respectively. 

In summary, Experiment 2 replicated and extended the results of Experiment 1. Participants were significantly more sensitive to position displacements when the saccade target object changed surface features during the saccade than when it did not. Importantly, changing multiple surface features of the object and its identity generated sensitivity to displacement that was similar to the sensitivity found when blanking the target object. We cannot conclude from the data that there was no difference at all in displacement sensitivity between the object-change and blank conditions. However, we can conclude that surface feature discontinuity generated an effect on object persistence that was similar to the magnitude of the effect generated when removing the object entirely for 250 ms. 

General discussion

The blanking method of Deubel et al. (1996) has shown that precise spatial and visual information is preserved across saccades and is available for report when the screen is blanked. However, such information does not appear to be available under normal circumstances, when the saccade target is visible after the saccade. Why is precise information about the saccade target available only when there is a disruption in the presence of that object across the saccade? We proposed that access to precise target information is dependent on whether the pre and post-saccade displays give rise to the interpretation of one persisting object or two different objects. 

There are several components to this account. At a general level, we adopt an object-based approach to transsaccadic correspondence and visual stability (Currie et al., 2000; Deubel et al., 1996; Deubel, Schneider, & Bridgeman, 2002; Hollingworth et al., 2008; Irwin, McConkie, Carlson-Radvansky, & Currie, 1994; McConkie & Currie, 1996). The object-based approach contrasts with image-based accounts of stability that depend on global remapping and spatiotopic integration (Duhamel, Bremmer, BenHamed, & Graf, 1997; Duhamel, Colby, & Goldberg, 1992; McConkie & Rayner, 1976; Melcher, 2005). Object-based theories assume that stability depends on a local evaluation of saccade target information encoded before the saccade and visual information available near the landing position after the saccade. Before a saccade, visuospatial attention shifts covertly to the location of the saccade target object (Deubel & Schneider, 1996; Hoffman & Subramaniam, 1995; Kowler, Anderson, Dosher, & Blaser, 1995) as observed in the frontal eye fields (Schall, 2002, 2004) and the superior colliculus (McPeek & Keller, 2002). Attentional selection in oculomotor regions feeds back into visual sensory regions (T. Moore & Armstrong, 2003), generating enhanced processing of visual features from the saccade target object (T. Moore & Armstrong, 2003; T. Moore, Tolias, & Schiller, 1998; Sheinberg & Logothetis, 2001), possibly by the dynamic convergence of visual receptive fields on the target location (Hamker, Zirnsak, Calow, & Lappe, 2008; Zirnsak et al., 2011). The shift of attention also enables the consolidation of these features into VWM (Irwin & Gordon, 1998; Schmidt, Vogel, Woodman, & Luck, 2002), and VWM supports the active maintenance of feature information across the disruption caused by the saccade (Hollingworth et al., 2008). Because attention controls the consolidation of items into visual memory and attention is directed to the saccade target location before the saccade, the information maintained across the saccade will be dominated by objects at or near the target location (Irwin, 1992a; Irwin & Andrews, 1996). 

When the eyes land, remembered properties of the saccade target are compared with objects near the landing position (Currie et al., 2000; Deubel et al., 1996; McConkie & Currie, 1996). If there is an object present with sufficient similarity to the remembered features, then correspondence will be established. If not, then the correspondence operation will fail, and the observer will perceive instability in the visual field. In the spatial domain, considerable discrepancy is tolerated. Saccade programming and execution generate substantial spatial error (see Bridgeman et al., 1994), and stability typically is preserved for target displacements that are less than one-third of the distance of the saccade (Bridgeman et al., 1975). The present results and those of Demeyer et al. (2010) indicate that certain surface-feature changes exceed tolerances for transsaccadic correspondence. Precise estimates of the thresholds for establishing correspondence on the basis of surface features will require systematic manipulation of surface feature similarity. 

The present study focused on the consequences of establishing object correspondence for the availability of precise information about target location. We proposed that that insensitivity to displacement across saccades is caused by an object-mediated updating mechanism (Enns et al., 2009; C. M. Moore et al., 2007). If correspondence is established, then the pre and post-saccade objects are mapped to the same object representation and the new state of the object overwrites (at least to some extent) memory for the initial state of the object. When the target is displaced across the saccade, the two states are often treated as the same object and participants perceive stability, but the new location of the target overwrites information about the original location of the target which impairs the detection of position changes. 

Poor sensitivity to displacement across saccades therefore can be considered a case of object-based masking, similar in mechanism to other phenomena in which visual masking is modulated by object continuity (Enns & Di Lollo, 2000; Lleras & Moore, 2003; C. M. Moore & Lleras, 2005). An example that is particularly relevant to the blanking effect involves apparent motion displays consisting of multiple noncontiguous locations (e.g., as an object travels on a circular path). Participants perceive a single, continuously moving object that persists visually at each presented location for a much shorter duration than is observed when apparent motion is absent. This motion deblurring effect suggests that the availability of low-level, sensory persistence is modulated by object continuity (C. M. Moore et al., 2007). When each state of the object is mapped to the same object representation (and apparent motion is perceived), visible persistence of the earlier states of the object is overwritten by the features of the object at the present location, thereby limiting the duration of visible persistence and eliminating blurring in the perception of motion. If, however, the object undergoes a salient change between motion frames (e.g., its size or color changes), participants perceive two objects simultaneously (C. M. Moore et al., 2007). In this case, object continuity is disrupted by the salient discrepancy in feature information, two object representations are formed, visible persistence of the previous state is not overwritten, and participants perceive two objects simultaneously: the currently presented stimulus and the visible persistence of the previously presented stimulus. Figure 5 illustrates this phenomenon, which the authors refer to as change-related persistence. A similar explanation applies to object substitution masking (Lleras & Moore, 2003; C. M. Moore & Lleras, 2005) and the misperception, relative spatial position in the flash-lag effect (C. M. Moore & Enns, 2004). 

Figure 5

 

Illustration of change-related persistence. A disc is presented in adjacent locations along a circular path, giving rise to the perception of a single disc moving along a circular path (apparent motion). The gray discs indicate previous locations. In the second-to-last frame of the display, a smaller disc is presented. The final frame of the display includes only a single (large) disc (shown in the bottom-left panel), but the perception is of two discs, the small one in the second-to-last position and a large one in the last position (shown in the bottom-right panel). This doubling does not occur when the disc does not change size.

In the present experiments, we tested whether a manipulation that has influenced object-based masking in previous studies—surface feature change—would generalize to object processing across a saccade. Such a finding would indicate that the failure to access precise information about target location is caused by a mechanism equivalent with, or functionally similar to, that proposed in masking studies: The two states of the target are mapped to the same object representation and the precise representation of the original location is overwritten. In Experiment 1, a change in the contrast polarity of the saccade target led to an increase in sensitivity to target displacement. In Experiment 2, changing multiple surface features of the target led to displacement sensitivity that rivaled the blanking condition. 

These findings suggest a resolution to the apparent discrepancy between theories claiming that transsaccadic memory is limited to VWM representations (abstracted away from precise sensory persistence) (e.g., Hollingworth et al., 2008; Irwin, 1992a) and evidence that sensory-level information is retained across the saccade (Deubel et al., 1996; Germeys et al., 2010). It seems quite likely that sensory-level information is retained across saccades. However, sensory persistence is highly susceptible to masking (Irwin, 1992b), whereas VWM is far less susceptible to masking (Pashler, 1988; Phillips, 1974). If object correspondence is established—and the visual system has a strong bias to establish correspondence—sensory persistence is masked by the new properties of the object to which it has been mapped, just as in the motion de-blurring phenomenon. Thus, under normal circumstances, when correspondence has been established, the only information available after the saccade is the more abstract information in VWM. According to this view, sensory persistence could play role in the immediate comparison operation after the saccade, but sensory persistence does not play any direct role in the accumulation of information across saccades. 

Likewise, it is important to stress that sensory persistence is not integrated across saccades to form a unified, spatiotopically organized percept: Direct tests have failed to find any evidence of such integration (Bridgeman & Mayer, 1983; Irwin, 1991; Irwin et al., 1983; O'Regan & Lévy-Schoen, 1983). At the delays used by Germeys et al. (2010) and Deubel et al. (1996), sensory representations almost certainly consisted of informational persistence rather than visible persistence (Coltheart, 1980). Informational persistence does not support visual phenomenology and does not integrate with new sensory input (Irwin, 1992b), providing no plausible means to generate a spatiotopically organized, integrated percept. In this respect, our position differs from recent accounts holding that low-level form information is transferred across saccades and aligned in a spatiotopic register (Melcher, 2005, 2007). Although our data are consistent with those of Melcher and colleagues in suggesting that visual form information survives saccades, we do not assume that this information is integrated or brought into precise spatiotopic register. Instead, we see the primary role of transsaccadic memory as supporting a comparison operation, not integration. Of course, when we view a scene, visual information from objects is acquired during fixations and accumulates over the course of viewing (e.g., Hollingworth & Henderson, 2002). However, such accumulation does not depend on integrating low-level sensory representations. Instead, higher-level object representations are retained in VWM and LTM from previously fixated objects during the course of extended scene viewing (Hollingworth, 2004; for reviews, see Hollingworth 2006, 2008). 

An important aspect of the present results is that they demonstrate a direct role for surface feature representations in the computation of object correspondence and the experience of continuity across saccades. This finding is consistent with the assumption of object-based theories (Currie et al., 2000; Deubel et al., 2002; Hollingworth et al., 2008) and related approaches (Fracasso, Caramazza, & Melcher, 2010; Melcher, 2011) that multiple properties of an object (including surface features and spatial location) are functional in correspondence operations. Converging evidence comes from the related work of Demeyer et al. (2010). Further converging evidence comes from Hollingworth et al. (2008). In this study, object continuity across saccades was examined in the context of gaze correction. Participants executed a saccade to a colored disk within a circular array of colored disks. During the saccade, the array was rotated one-half of the distance between two objects so that the eyes landed between the target and distractor, requiring a corrective saccade. This manipulation simulated an inaccurate saccade that causes the eye to miss the intended object. Such errors are extremely common, and it is precisely this variability in the landing position of saccades that necessitates a high threshold for attributing spatial discrepancy to a change in the world (Deubel et al., 1996). In the Hollingworth et al. (2008) method, the target could be discriminated from the distractor only on the basis of a surface feature: color. Spatial information was noninformative. Gaze correction was accurate, extremely rapid, and automatic, demonstrating that memory for object color was used to establish correspondence between the saccade target before and after the saccade, supporting selection of the target as the goal of the corrective saccade. Subsequent work has established that memory-based correction depends specifically on the VWM system (Hollingworth & Luck, 2009) and that both surface feature properties and spatial properties of the target can be used to solve the correspondence problem introduced by saccades (Richard et al., 2008). 

This evidence contrasts with traditional theories of object correspondence, often termed “object-file” theories, claiming that object continuity is established solely on the basis of an object's spatiotemporal properties (Flombaum et al., 2009; Kahneman et al., 1992). In the object-file approach, correspondence operations depend on attentional pointers (alternatively termed spatial indexes or FINSTs) that mark an object's location but do not represent the object's perceptual features or identity. For example, Pylyshyn (2000) proposed that a set of pointers is established before a saccade, marking the locations of a handful of salient objects. The pointers are retained across the saccade and are mapped to the locations of objects visible after the saccade, providing continuity in object representation across saccadic disruption and change. This basic proposal has been extended by Cavanagh, Hunt, Afraz, and Rolfs (2010; see also Wurtz, 2008) to include the claim that the pointers correspond to peaks of activation in neural maps implementing visual attention and saccade target selection. Before a saccade, activation is remapped to the predicted locations of the indexed objects after the saccade. This type of mechanism is certainly a plausible means to establish transsaccadic correspondence and stability. However, our data indicate that such a mechanism cannot be a complete explanation. An account that is limited to attention pointers would have no means to solve the correspondence problem in the Hollingworth et al. (2008) gaze correction task, which participants solved efficiently on the basis of color. In addition, an account in terms of attentional pointers cannot explain the surface feature effects on displacement sensitivity observed here, as spatiotemporal properties were controlled between the no-blank and polarity/object-change conditions. 

A growing body of evidence suggests that object correspondence operations consult surface feature information across multiple perceptual domains. Surface feature effects have been observed in the original Kahneman et al. (1992) object reviewing task (Hollingworth & Franconeri, 2009; C. M. Moore et al., 2010), in object mapping across brief occlusion (Feldman & Tremoulet, 2006; Hollingworth & Franconeri, 2009), in apparent motion (Hein & Cavanagh, 2012; Hein & Moore, 2012; C. M. Moore & Enns, 2004; C. M. Moore et al., 2007), in multiple-object tracking (Makovski & Jiang, 2009), and in object-based inhibition of return (Tas et al., 2012). Thus, the general idea that object correspondence is computed solely on the basis of spatiotemporal information is no longer tenable. The visual system consults multiple forms of perceptual information when faced with the difficult task of mapping objects across perceptual disruption and change. 

Acknowledgments

This work was supported by National Institutes of Health (R01-EY017356 to A. H.). 

Corresponding author: A. Caglar Tas. 

Email: [e-mail] 

[e-mail] 

Address: Department of Psychology, University of Iowa, Iowa City, IA. 

Commercial relationships: none. 

Footnotes

¹  Although Cavanagh et al. (2010) adopt many of the features of object-file theories, they do not necessarily claim that transsaccadic correspondence and stability are established solely on the basis of spatial information.