Blindness to a simultaneous change of all elements in a scene, unless there is a change in summary statistics

Abstract

Sudden change of every object in a display is typically conspicuous. We find however that in the presence of a secondary task, with a display of moving dots, it can be difficult to detect a sudden change in color of all the dots. A field of 200 dots, half red and half green, half moving rightward and half moving leftward, gave the appearance of two surfaces. When all 200 dots simultaneously switched color between red and green, performance in detecting the switch was very poor. A key display characteristic was that the color proportions on each surface (summary statistics) were not affected by the color switch. When the color switch is accompanied by a change in these summary statistics, people perform well in detecting the switch, suggesting that the secondary task does not disrupt the availability of this statistical information. These findings suggest that when the change is missed, the old and new colors were represented, but the color–location pattern (binding of colors to locations) was not represented or not compared. Even after extended viewing, changes to the individual color–location pattern are not available, suggesting that the feeling of seeing these details is misleading.

Introduction

Important information is often signaled by changes to a visual scene. For example, the color change of a traffic signal tells us to go or to stop, and a change in the movement pattern of a lion on the hunt suggests she may have noticed us. Despite this importance, under certain circumstances, changes to a scene are usually missed. This is called “change blindness.” 

Change blindness demonstrations typically include two potentially critical characteristics. First, the change to the display does not create a unique transient such as flicker or motion. Unique transients are avoided when designing change blindness experiments because a transient in an otherwise static field attracts attention to its location and that of the change. In the paradigm developed by Rensink, O'Regan, and Clark (1997), a scene and a changed version of the scene alternate with a blank field between that causes flicker to appear in all locations rather than just the location of the changed element. Simons, Franconeri, and Reimer (2000) reported that blindness to the change of an element of a scene could also occur without a blank field, provided that the change was gradual rather than sudden, presumably because then the transient evoked by the change is much weaker. 

A second aspect of classic change blindness displays is that only one or at most a few elements of the pictured scene are changed. The reason is that if participants know or are told the location of a potential change, typically they detect it very quickly (Levin & Simons, 1997; O'Regan, Rensink, & Clark, 1999; Rensink et al., 1997; Simons et al., 2000). This suggests that with typical displays change blindness occurs because the participant, although processing selected elements for change, usually selects unchanging elements rather than the few changing elements. 

When all elements of a display change simultaneously, there is no possibility of selecting the wrong elements. One might expect, then, that people would easily detect that a change had occurred. In some circumstances however, this is not true. 

Investigations of “structure-from-motion” displays established that for moving dots consistent with a larger structure such as a surface, participants often fail to detect local changes inconsistent with the continuing larger structure (Petersik & Dannemiller, 2004; Treue, Anderson, Ando, & Hildreth, 1995). This literature confined itself to investigating changes in dot motion direction however, and it is not clear whether the results would generalize to color changes. In addition, it is not clear whether a strong overriding global structure is necessary for this form of change blindness, although Petersik and Dannemiller briefly mentioned change blindness with two transparent surfaces composed of dots moving in the opposite directions. 

Motion transparency displays also produce a phenomenon termed “blindness to oscillating dots,” in which observers fail to notice the direction reversal of individual dots when the phases of dots' oscillations are random (Kanai, Paffen, Gerbino, & Verstraten, 2004). The blindness to oscillating dots can be interpreted as a phenomenon where the global simpler organization of two surfaces sliding over each other captures the individual motions (Kanai et al., 2004). The continuous motion may also play a part, as it creates continuous luminance transients so there is no unique transient to signal the location of the change. In one case of a motion transparency display, it was suggested that color changes were not conspicuous, although this was not explicitly measured. In one case of a motion transparency display, it was suggested that color changes were not conspicuous, although this was not explicitly measured (Clifford, Spehar, & Pearson, 2004; Moradi & Shimojo, 2004). Rather, these studies investigated reports of the color–motion pairing of dots. In Clifford et al. (2004) for example, each dot changed its color from red to green simultaneously as its direction reversal. When the phase coherence of direction change across dots is low, observers see motion transparency, that is, a global percept of two surfaces sliding over one another, suggesting that the visual system reassigns dots to the other surface when they change direction and color, yielding the impression that both the direction and the color of dots are unchanging. These findings of blindness to change of moving dots share some common characteristics. First, they use stimuli that did not have a stable configuration because the dots had limited lifetime and/or involved motion oscillations with different dots out of phase. The fast motion speed (4 deg/s or faster) used may further impair any configuration perception. Second, these displays involved simultaneous changes of color and motion direction, making local dot information and global surface organization incompatible, which has been suggested to be key to the effect (Clifford et al., 2004; Kanai et al., 2004). As detailed below, the current study utilized stimuli without these characteristics. Instead we used slow motion, unlimited lifetime, and no direction reversals yet nevertheless found blindness to color changes. 

After our initial conference report (Saiki & Holcombe, 2008) of the phenomenon that will be documented formally below, Suchow and Alvarez (2011) also discovered a situation in which color change of every element is not conspicuous. Unlike the studies cited above, our stimuli and those of Suchow and Alvarez used displays with unlimited lifetime and no motion reversal. Although Suchow and Alvarez did not show that observers actually fail to detect the change, they did show that the change is not salient, using a dramatic demonstration. 

In Suchow and Alvarez's (2011) demonstration, every element changes color continuously, gradually rather than with a sudden change, which helps reduce the attention cuing power of the transient (Simons et al., 2000). A second factor that reduces the attention cuing effect of the change is that the elements are always in motion, which yields luminance transients that may mask any remaining transient associated with the gradual change. 

It appears that in Suchow and Alvarez's (2011) display, as in the displays we used, if one attentionally tracks an individual dot, then the change can be reliably detected. Suchow and Alvarez acknowledge this possibility and, for this reason, asked participants to attend to the entire set of dots simultaneously and tightly spaced the dots to reduce the ability to attend to an individual dot. 

Here, in addition to using a primary task of change detection, to reduce the ability to track individual dots and to compel fixation we included a vigilance task involving the fixation point. The white fixation point changed briefly to blue either once or twice, and participants were asked to monitor fixation and report whether it became blue once or twice. 

Our experiments were designed to reveal what aspects of change, if any, would be salient when associated local transients were made inconspicuous, the configuration of elements was stable, and participants did not track an individual dot. They were also designed to document the existence of change blindness when all elements of a display change simultaneously, although this is no longer novel due to the publication of Suchow and Alvarez (2011) in the interim. 

In Suchow and Alvarez's (2011) display, the color of each element was changing at every moment, in a gradual ramp through color space. A consequence is that observers may not have had enough time to perform some necessary processing for change detection. This might involve encoding the relative locations of the colors, binding them with their individual locations, or comparing them to their previous colors. If the pattern is stable as in our experiments, the additional time available might allow observers to succeed at encoding or comparison and detect a change. 

A second issue is whether in the absence of conspicuous transients in a display and with little or no tracking of individual dots, there exists any class of color change that would be noticeable. The experiments here reveal that the answer is yes—what we call “surface color change” allows change detection. To explain this, we must first describe the basic stimulus. 

The display in our first experiments comprised 192 dots, half moving to the left and half moving to the right, giving the impression of two distinct surfaces ( Figure 1). Each dot was either red or green, equiluminant with each other (but contrasting with the black background) to reduce strength of the transient associated with the color switches. In half of trials, all the dots changed color and all did so at the same time—the red dots became green and the green dots became red. In the other half of trials, there was no change. Unlike typical dot kinematograms that involve dots with limited lifetime (Braddick, Wishert, & Curran, 2002), here the lifetime of each dot was unlimited. The use of unlimited lifetime means that the feature–location pattern remained unchanged despite the motion of the dots (except for when dots reached the edges of the display). This maximized the possibility of encoding and remembering the pattern of colors and locations. 

Figure 1

 

Experimental stimuli. (a) Illustration of the single frame of the motion transparency display. Dots were presented within an imaginary circle with 4 deg of eccentricity. The arrows and the large white circle were not displayed in the experiment. (b) The moving dots were displayed for 1920 ms, during which the central fixation changed its color once or twice.

The primary task was to judge whether there was a color change or not. The fixation vigilance task, intended to reduce attentional tracking of individual dots, was to judge whether the fixation changed color to blue once or twice during the trial. 

The dots moving in a particular direction (leftward or rightward) formed a subjective surface (e.g., Qian, Andersen, & Adelson, 1994). We define “surface color change” as each surface's change in proportion of red and green dots (Figure 2 and Table 1). For example, if each surface was half red and half green dots, then the color switch of all dots from red to green and vice versa had no effect on each surface's proportion of red and green—therefore, surface color change is zero. For a surface color change of 1, each surface has dots of only one color, so the color switch causes the surfaces to change color entirely. A surface color change of 0.5 starts with 3/4 of the dots on each surface being one color and 1/4 of the dots being the other color. The change in colors then causes a surface to go from 3/4 one color to 3/4 the other color, meaning that half (0.5) of the dots changed color. As the surface color change increases from zero, the color switch event causes a greater change in color proportion of each surface (summary representation), while the color–location pattern change remains maximal (Table 1). 

Figure 2

 

Schematic illustration of the notion of surface color change. For different proportion surface color change conditions, other cues to color switch detection remained constant, including color–location pattern, local binding, and flicker. Only the color proportion of each surface, not the whole pattern, varies as surface color changes.

Table 1

 

Examples showing the correspondence between surface color change and proportion of changed dots. Suppose each surface contains 100 dots. Each cell denotes the number of dots of one color (e.g., red) for each surface (proportion in parentheses). All the dots always change color, but depending on the color proportion of each surface, surface color change systematically varies.

Surface color change	Before change		After change
	Surface 1	Surface 2	Surface 1	Surface 2
0	50 (0.5)	50 (0.5)	50 (0.5)	50 (0.5)
0.2	60 (0.6)	40 (0.4)	40 (0.4)	60 (0.6)
0.4	70 (0.7)	30 (0.3)	30 (0.3)	70 (0.7)
0.5	75 (0.75)	25 (0.25)	25 (0.75)	75 (0.75)
0.6	80 (0.8)	20 (0.2)	20 (0.2)	80 (0.8)
1	100 (1.0)	0 (0)	0 (0)	100 (1.0)

As opposed to the surface color change that we varied, the display-wide summary representation (overall proportion of red and green dots) never changed, because the whole display was always composed of half red and half green dots. Thus, an effect on performance of surface color change implicates surface-specific summary representations rather than global summary representation. In addition, note that regardless of surface color change, the entire color–location pattern always changed in that all dots changed color. Furthermore, flicker and change in binding of color and motion for single dots were equivalent across different surface color change conditions. 

Conceivably, the hypothetical summary representation of surface color might not contribute to change detection performance and change detection might be poor in all conditions. Alternatively, if the change in surface summary representation is available, then performance should mirror the amount of surface color change. If a change in feature–location pattern is sufficient, then change detection performance should be high regardless of summary representation. 

General methods

Participants

Each experiment had five or six participants including one author (JS). All had normal or corrected-to-normal vision. 

Materials

Experiments were coded using the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997) and displayed on a 75-Hz CRT screen. Visual stimuli were random dot kinematograms (Figure 1). Each dot was 0.1 deg in visual angle across and was either red or green, with its luminance adjusted to subjective equiluminance for each participant using a minimum motion procedure (Anstis & Cavanagh, 1983). The pattern spanned a circular area 8 degrees in diameter. All dots moved either horizontally (with half of dots moving to the right and half to the left) or vertically (half upward and half downward). The dots moved at 2 deg/s, slow enough to attentively track. At the center of the pattern, a white fixation point 0.15 deg in visual angle was presented. 

Each trial began with the presentation of a fixation dot for 500 ms, followed by the moving dot display. All dots moved with constant velocity and direction throughout the trial, unless they moved out of the display region, in which case they were placed in a random new location within the region and continued moving. The fixation's visibility was preserved by an invisible occlusion zone of 0.75-deg diameter around fixation, behind which dots were invisible. The dot motion lasted for 1920 ms. In half the trials, the color of all the dots switched simultaneously halfway through the motion display. That is, all red dots became green, and all green dots became red. The primary task was to detect this color switch event. Observers' secondary task was to judge the number (one or two) of fixation color changes correctly. The fixation dot began white, but at a random time during each trial it turned blue for 147 ms either once or twice. 

The presence of dots moving in both directions yielded two subjective transparent surfaces. The main independent variable, surface color change ( Figure 2 and Table 1), was each surface's change in relative proportion of red and green. For example, for surface color change of zero, the color switch had no effect on each surface's relative proportion of red and green. 

Procedure

Observers were instructed that the primary task (color switch detection) and secondary task (counting fixation color changes) should be performed as accurately as possible, with additional emphasis on avoiding errors in the easier secondary task. Observers ran practice trials until they felt comfortable performing the dual task. One additional practice block of 18 trials was provided, followed by a series of experimental blocks. At the end of each trial, under no time pressure, participants first made a key press to indicate whether a color switch had occurred. Second, they indicated how many times the fixation had changed to blue by pressing the “1” or “2” key. 

Predictions

If observers apprehend the spatial pattern of colors, then they should be able to detect a color switch regardless of surface color change. In contrast, if only summary statistical information is available, performance of the color switch detection should be low with zero surface color change and increase as surface color change increases. Although observers are told to attend to the fixation point, if they can nonetheless monitor an individual moving dot or if they are able to pick up color flicker signals produced by the color switch, that should be sufficient to detect the color switch. 

Results

Poor change detection in the absence of summary representation change

In all experiments, accuracy in the fixation task was above 90% and showed no trade-off with the primary task, thus only the results of the color switch detection task are discussed. Casual inspection of the display revealed that it was possible to detect the change of an individual dot if one attentionally tracked it (see Movies S1– S3). 

Regarding change detection performance, first consider the situation of no (0) surface color change, when the two motion-defined surfaces were half red and half green both before and after the change. In this condition, mean accuracy in the change detection task was 65.4% correct ( d′ = 1.23) with the chance level of 50%. This is remarkably poor for a large and sudden change in color of every element in the display. Despite the stability of the color–location pattern until the change, participants apparently failed at one or more aspects of the processing required to detect the change. 

When surface color change was maximal (“1” on our metric), all the dots moving in one direction were one color (say, red) and all the dots moving the other direction were the other color (green). The change then caused each surface to change from uniformly one color to uniformly the other. Here, performance was much improved—91% correct, d′ = 3.0. The level of performance declined dramatically for lower surface color changes (ANOVA main effect F(2,8) = 18.103, p < 0.01, η ² = 0.604). This effect of surface color change was significant for each dot density condition (single-factor ANOVAs, 12 dots: F(2,8) = 19.272, p < 0.001, η ² = 0.643, 48 dots: F(2,8) = 9.012, p < 0.01, η ² = 0.606, and 96 dots: F(2,8) = 15.217, p < 0.01, η ² = 0.729), suggesting that the summary representation does not require crowded conditions to improve performance. More about the crowding issue below. 

Crowding might make it impossible to individuate dots and bind them with their locations, conceivably leaving the participant with only an estimate of the proportion of each color on each surface. Note however that this would be an advance in the understanding of the effects of crowding (for potentially related crowding literature, see Dakin, Cass, Greenwood, & Bex, 2010; Parkes, Lund, Angelucci, Solomon, & Morgan, 2001). In the present experiment, a role for crowding was tested by manipulating dot density (12, 48, or 96 dots/surface). The average nearest neighbor distances of dots for the 12, 48, and 96 dots/surface conditions were 0.78, 0.38, and 0.26 deg among all dots in the display, and 1.15, 0.54, and 0.38 among dots moving in the same direction. While 48 and 96 dots/surface conditions were highly crowded, many dots in the central 1.6 deg of the display in the 12 dots/surface condition were not crowded, according to Bouma's (1970) law that the critical spacing for crowding to occur is about half the eccentricity of an item. Note that this estimate is quite conservative, because it has been shown that in the isoeccentric direction (two dots at same eccentricity but different directions), the critical spacing is much smaller, by a factor of two or three (Toet & Levi, 1992). For the five participants, Figure 3 shows the average performance for the color switch detection task. Even when only 12 dots/surface were used so that many dots would not have been crowded based on nearest neighbor distance calculation (Pelli & Tillman, 2008), performance was still poor when there was no accompanying surface color change. 

Figure 3

 

Results of the first experiment. Mean d′ as a function of surface color change for each density condition. Error bars are standard error across subjects.

Blindness to ubiquitous change with a single surface

The change blindness documented here raises the possibility that even with extended viewing, participants do not apprehend arbitrary color–location patterns. If so, change blindness should not occur solely in situations with dots moving in different directions. We tested this with a further experiment using dots all moving in the same direction. Unlike the display with multiple surfaces, whose color–location pattern is stable only within each surface, the single-surface display had a stable color–location pattern for the entire display. Performance detecting the change was again poor. Two hundred dots were used, yielding a density similar to the 96 dots/surface condition of the previous experiment. The surface color change was 0, 0.08, or 0.32. Surface color change here was equivalent to overall color change, because there was only one surface. 

For the 0 change condition, both hypothetical summary representations (surface color change and overall color change) remained constant, even though all dots changed color. For the 0.08 change condition, the dots changed from 54% one color (say, red) to 54% the other (green). For the 0.32 change condition, the number of dots of one color was 66%, which was taken on by the other dots after the change. This single motion direction display is unlike the two-direction displays of the first experiment, for which the total proportion (including both directions or surfaces) of dots of each color was always 50%, with the change in putative summary representation occurring only within each surface. 

As shown in Figure 4a, change detection performance in the 0 surface color change condition was poor ( d′ = 1.9), although higher than the 0 change conditions in the previous experiment. As in the previous experiments, surface color change improved performance (ANOVA main effect F(2,8) = 43.426, p < 0.001, η ² = 0.559), for example, performance was significantly greater for the 0.32 surface change condition than for the 0 surface change condition (Tukey HSD test, p < 0.001). 

Figure 4

 

Single surface experiments. (a) Mean d′ as a function of surface color change in the color change experiment. (b) Mean d′ as a function of brightness proportion change in the brightness change experiment, where no switch trials include spatial jump of dots. (c) Schematic illustration of the spatial jump manipulation. Brightness switch reversed contrast of each dot, while spatial configuration was kept constant. Spatial jump changed spatial configuration, while brightness proportion was kept constant. Error bars are one standard deviation across participants.

Next, we investigated whether the change blindness phenomenon found is specific to color. Instead of distinct hues, two contrast polarities were used with half of dots bright (100 cd/m ²) and half dark (54 cd/m ²) against a gray (77 cd/m ²) background forming a single surface. Unlike in the equiluminant color display, in this contrast polarity display apparent motion accompanying the switch was immediately evident. This is consistent with the strong contribution of luminance contrast to motion perception (Anstis & Cavanagh, 1983). To prevent observers from using this motion signal to detect a switch, a spatial jump was added to the no switch condition. At some point during the display interval, when the brightness switch occurred in the switch condition, in the no switch condition all dots displaced in random directions (each by random distances between 0 and 0.75 deg) simultaneously, yielding a jumbled impression. This simultaneous displacement occurred in a single frame transition, with the normal motion resuming afterward. Note that this jump changed the spatial configuration of dots, while the brightness switch kept the spatial configuration of dots but reversed the contrast polarity of each dot. This difference is independent of the manipulation of surface brightness change, so if observers are sensitive to spatial configuration, jump and switch should be discriminable. The experiment thus tests whether the observers could discriminate the jump and switch events. The three trial types (no change, brightness switch, and spatial jump) were explained to the participants and they were asked to respond yes to only the brightness switch trials. 

Figure 4b shows d′ for the brightness switch detection task as a function of brightness proportion change. In the 0 change condition, switch detection performance was approximately as poor ( d′ = 1.16) as for the previous experiments. The mean d′ in the 0.32 change condition (2.17) was significantly higher than the 0 change condition (1.16) or the 0.08 change condition (1.13; Tukey HSD test, p < 0.025). Compared to the color single surface experiment, performance was lower, which reflects more false alarms for the brightness switch, likely caused by the spatial jumps ( M = 0.356, much higher than false alarms with no change trials, M = 0.017), suggesting that observers had difficulty in discriminating change in spatial configuration from change in attribute configuration by brightness switch, unless there was also a change in the summary representation (proportion of black and white). 

Availability of summary representations

Experiments thus far suggest that a change in the summary representation is important to detect the change of a surface when one does not monitor an individual element. However, the alternative possibility that a misbinding illusion (Wu, Kanai, & Shimojo, 2004) impaired the detection of color change should be addressed. In peripheral vision, steady-state misbinding of color and motion has been reported (Wu et al., 2004). Although the failure in global binding of color and motion documented by Wu et al. (2004) occurred farther in the periphery than our displays extend, here the misbinding might conceivably occur more centrally. To investigate this possibility, the more peripheral 2 deg of the dot field was set to maximally differ in surface color proportion from the more central area. This was consistently perceived erroneously in the circumstances used by Wu et al. but perceived very accurately in our display (Figure 5), as the following data document. Five participants judged whether surface color proportion in central and peripheral areas are the same or different. They also performed the same secondary fixation flicker task as in the other experiments. Performance of the primary task was very high whether the central region had low or high surface color proportion (mean d′ = 3.48 and 4.20, respectively). In both conditions, steady-state misbinding apparently does not occur with our display. 

Figure 5

 

Schematic illustration of the stimulus of the experiment testing for steady-state misbinding (data are described in the text). Vertical dotted lines dividing central and peripheral areas were actually presented.

We have suggested that surface color change improves change detection because it increases the degree of change of surface-specific summary statistics. Conceivably, surface color change might instead boost performance by increasing availability of one of the other cues, such as individual dots' color binding change or ability to apprehend the configuration. To investigate these possibilities, we performed an experiment involving rearranging dot locations randomly. 

Rearranging the dots disrupted the individual bindings and the color–location pattern, reducing any possibility of using color–location binding to notice the change, as well as reducing the utility of local dot tracking, while preserving the summary statistics. If the benefit of surface color change seen in previous experiments reflects summary statistics as we hypothesize, then the effect of surface color change here should be similar to previous experiments. In contrast, if higher surface color change displays somehow involve higher availability of the color–location pattern, the effect of surface color change should be substantially reduced with limited lifetime displays. 

The displays were like those of our first experiment, but each dot had a lifetime of only 280 ms, after which all the dots were relocated to random positions. This created conspicuous flicker. To approximately equate perceived flicker between the limited and unlimited lifetime conditions, every 320 ms of stimulus display a blank period of 40 ms was inserted. In the unlimited lifetime condition, the dots reappeared where they would have been had they continued moving during the 40 ms. Three surface color change conditions (0.2, 0.4, or 0.6) were used, corresponding to 60%, 70%, and 80% of one color for each surface ( Figure 6a). 

Figure 6

 

(a) Schematic illustration of the limited lifetime display, which prevents observers from tracking dots for longer than the 320-ms lifetime. Dotted circles were imaginary. (b) For the unlimited and limited lifetime conditions, mean d′ is a function of surface color change. Error bars are one standard error across participants.

Figure 6b shows d′ for the change detection task as a function of surface color change for the limited and unlimited lifetime conditions. The effect of surface color change was observed with both unlimited and limited lifetime (main effect of surface color change, F(2,8) = 10.77, p < 0.01, η ² = 0.303), whose effect sizes were comparable (interaction not significant, F(2,8) = 1.27, η ² = 0.001). The effect sizes of surface color change on performance ( d′) evaluated by coefficient of linear regression were 5.637 ( t(14) = 2.188, p < 0.05) and 7.107 ( t(14) = 2.759, p < 0.05) for unlimited and limited lifetime conditions, respectively. The existence of a similar effect of surface color change in the limited lifetime condition that had no stable color–location pattern suggests that summary representations are mainly responsible for the benefit from surface color change. 

Although the benefit of surface color change was similar regardless of lifetime, performance in the baseline unlimited lifetime condition was consistently superior to the limited lifetime condition. In the unlimited lifetime condition, this may reflect the use of some cue besides the color–location pattern that we tried to isolate. We suspect there is some residual ability to track an individual dot or detect the flicker associated with the change. Alternatively, the unlimited lifetime condition may have some advantage in encoding location information due to predictability of dot locations. Note that the comparable effect sizes of summary representations between limited and unlimited lifetime conditions indicate that any advantage in encoding spatial layout does not extend to the detection of change in color–location pattern. 

Is detection of a change done by attending to a single surface?

The experiments in the previous section suggest that the effect of surface color change in change detection performance reflects change in summary representations, not illusory misbinding, individual dot binding, or a spatial configuration representation. The next experiment investigated how summary representations are extracted for change detection. 

Can surface-based statistics be extracted for both the surfaces in parallel, or is selective processing necessary? Recall that the surface color change in our experiments affected both surfaces. However, selective processing may be necessary to make summary representations available for change detection, suggesting that a higher level top-down control mechanism is essential to make use of summary information. 

If selective processing is necessary, the summary statistics derived may be conditional on selecting a single feature value, either motion or color, which we call feature-based processing. Feature-based processing assumes that summary representations can be extracted efficiently in two ways: by selecting a particular motion direction or by selecting a particular color. Although we defined manipulation of surface color change in terms of motion directions that give rise to subjective surfaces, the statistical structure is equivalent when summary statistics are considered in terms of colors. For example, in Table 1, surface color change of 0.5 corresponds to one surface (say, rightward motion) having 75% of one color (say, red) at first and changing to 25% of that color. Considered in terms of color, 75% of dots with one color (say, red) move in one direction (say, rightward), which changes to 25% after the change. 

An alternative to the feature-based processing hypothesis is that surface-based selective processing is required. On this account, summary representations are extracted only via surface-based computation, which predicts that selective processing of motion direction is needed, not selective processing of color. Selection of a single motion amounts to selection of a single surface and possibly leads to calculation of the color proportion, which is how the change detection is done according to this theory, whereas it cannot be done through selection of a single color. Note that this is not to deny the existence of feature-selective attention generally but rather may pertain only to change detection with displays like ours. 

To address the selective processing issue, we asked participants to perform a new secondary task involving the display of dots. One dot in the display changed size suddenly and participants were to report which quadrant of the display it was in. Participants were informed that the dot size change might occur in any dot (baseline), that it could occur only in dots moving in a particular direction (motion or surface attention), or that it could only occur in dots of a particular color (color attention). 

If selective processing is not necessary to extract summary representations, then change detection (the primary task) performance should be as good or better in the baseline condition as in the other conditions. If feature-based selective processing is necessary, improvement by higher surface color change will be observed in the conditions with monitoring one color or one direction. If surface-based selective processing is necessary, only the condition with monitoring one direction will show improvement. 

This final experiment also examined an additional factor that may affect performance. In the previous experiments, although dots moved constantly with unlimited lifetime, dots that moved outside the display region were relocated to a random location, which creates sudden, unpredictable onsets of the dots. To eliminate this possible distracting factor, the final experiment used a square viewing window, and dots disappearing at one edge were relocated at the other edge. If disruption of spatial configuration by sudden onsets was a major factor in the change blindness observed, the final experiment will show a substantial improvement in performance regardless of surface color change. 

The display was composed of 200 dots forming a square region. The primary task was the same change detection task of color reversal of all dots. The secondary task was the detection of size change of a single dot, and participants were asked to report the quadrant of the size-changed dot. The size of the critical dot changed from 0.1 deg to 0.23 deg for 200 ms, and to avoid ambiguity, dots located within 1.0 deg of the vertical or horizontal meridians never changed. 

Three selective processing conditions were included. In the baseline condition, size change could occur to any dot, so all the dots should be monitored. In the motion attention condition, only dots of a particular motion direction were eligible to change size, and the direction was signaled by the letter (R for rightward and L for leftward) at the fixation point. In the color attention condition, the color to monitor was signaled by the color of the fixation point. Each attention condition was run in a separate block. At the beginning of a block of 36 trials, the name of the attention condition was displayed, and in each trial, the fixation point indicated which condition a participant was in. There were 4 blocks of each attention condition in a session, and 3 sessions were run in total for each participant. The order of selective processing conditions was randomized. 

Figure 7 shows d′ for the change detection task as a function of surface color change for the selective processing conditions. First, it is evident that the change detection performance was poor with the low surface color change in all conditions (mean d′ = 1.19 and 1.31 for surface color change 0 and 0.2, respectively), suggesting that the random relocation of some dots in the previous experiments was not the critical factor for the change blindness. Second, the effect of surface color change within the range 0 to 0.6 was modulated by selective processing conditions, such that the motion condition showed both an overall performance advantage (main effect, F(1,5) = 12.47, p < 0.05, η ² = 0.061) and a larger effect of surface color change compared with the baseline (interaction of selective processing and surface color change, F(3,15) = 4.29, p < 0.05, η ² = 0.074), whereas the color condition failed to show a significant benefit from surface color change (main effect, F(1,5) = 2.05, ns, η ² = 0.035 and interaction F(3,15) = 3.07, p = 0.06, η ² = 0.041). As Figure 7 shows, for surface color change of 0.6, only the motion condition attained the performance level comparable to the previous experiments, and the mean d′ of 2.64 was significantly higher than the baseline condition (mean d′ = 1.77, F(1,5) = 9.51, p < 0.05, η ² = 0.295). The color condition (mean d′ = 1.80) failed to show a significant difference from the baseline condition ( F(1,5) = 0.03, ns, η ² = 0.001), though a significant difference was observed at surface color change of 0.4 ( F(1,5) = 28.47, p < 0.01, η ² = 0.374). 

Figure 7

 

Mean d′ as a function of surface color change for selective processing conditions. Error bars are one standard error across participants.

These results indicate that the change blindness observed in the previous experiments is not specific to the fixation vigilance task nor to the disruption of configuration by random replacement of dots moved out of bounds. When all dots need to be monitored, the improvement from increased surface color change is substantially attenuated, suggesting that selective processing is important to utilize summary representation in change detection. Attention to a motion direction provided a significant improvement in change detection, while attention to a color did not, suggesting that surface-based selective processing is necessary. 

One possible reason that surface color change was not as beneficial in the color attention condition could be that participants cannot limit their attention to dots with a single color. However, a control experiment with the same stimulus set but without the change detection task indicates that this is not the case. To evaluate the attentional selection, the same attentional cues were used, but in half of trials the cue was valid and in half of trials it was invalid, and the instruction was that the task goal was to maximize size change detection in the valid trials, and accuracy of invalid trials did not matter. The benefit of attention, measured as difference in accuracy between valid and invalid trials, was similar in both the color and motion conditions ( M = 0.203, t(5) = 7.73, p < 0.01, for color condition and M = 0.114, t(5) = 2.82, p < 0.05, for motion condition), suggesting that selective attention to a particular color is effective and facilitates size change detection. Overall, the results of the final experiment indicate that participants are particularly sensitive to surface-based summary representation. 

Discussion

When prevented from tracking an individual element by a secondary task, people could detect a massive color change reliably only when the change yielded a large change in summary representations. The change to a completely new color–location pattern, if not accompanied by a change in surface color, was not usually noticed. 

As far as we know, no previous work investigated the contribution of summary representations and color–location pattern to detecting a change. The results here indicate that even with long exposure, color–location pattern contributes little. Even though partial information such as pairing a subset of dots or a single dot would have been sufficient, still the change was frequently not detected. The poor performance in detecting the color switch surprises many observers. When one views Movies S1– S3, the secondary fixation task does not seem to degrade the subjective feeling of a rich representation of the display, and therefore, one may expect reliable color change detection performance. 

The cause of the poor performance in detecting the color switch might be overwriting of the old colors by the new, failure to represent the new colors, or failure to compare the old colors to the new (Simons, 2000). However, the accurate performance with large surface color change suggests that the problem is not with representation of the colors per se but binding of colors and their locations (the configuration). Extended viewing was not sufficient to encode and/or prepare color–location bindings for change detection. Possibly, the color–location pattern is not represented, or if represented, it is perhaps never compared to previous representations but simply overwrites them. In contrast, the summary representation is readily available for change detection. 

The final experiment suggests that observers are sensitive to change in summary representations due to surface-based selective processing. When observers must monitor all dots, or dots of one particular color, the benefit of surface color change was substantially attenuated. The final experiment also revealed that the findings of the current study are not specific to a particular secondary task (fixation monitoring) or due to disruption of spatial configuration by sudden onsets of relocated dots. 

One account of summary representations in vision is that when summaries are extracted, the individual elements continue to be represented but with low fidelity (Alvarez, 2011; Alvarez & Oliva, 2008). At first, this theory seems consistent with our findings that the summary statistic is available while the individual elements are not. However, the color change here was very large, from red to green, and it seems unlikely that the individual dot colors would be represented with such low fidelity that red could not be discriminated from green—red and green are at the opposite ends of the color space. Furthermore, if the representations of local color change were present but noisy, the summary representation of change should be accurate because of the law of large numbers (Alvarez, 2011). 

We suggest that unlike other local properties, without transients local change signals are not explicitly represented, unless focused attention is directed to a dot, thus with distributed attention only changes in summary representation of perceptual properties can be perceived. Change detection of accumulated local properties (summary representation) is easy, while accumulation of local change signals is difficult or impossible. 

The current study is mute regarding the visual processing stage that summary representations are computed in. It may reflect low spatial frequency filtering or some more complex mechanism. Further studies are necessary to resolve this issue. 

Finally, the findings of the current study appear to have some theoretical relationship with the role of perceptual objects in attentional capture. For example, Yantis and Hilstrom (1994) reported that appearance of a new perceptual object, not luminance increment, captures attention, which is consistent with the role of surface representation in change detection, in particular, the advantage of the motion condition over color condition in the final experiment in the current study. One possible interpretation based on attentional capture is that surface color change induces the appearance of new object, which facilitates change detection by capturing attention. One should be cautious, however, as some studies suggest that surface color does not guide object persistence (Mitroff & Alvarez, 2007), so the determinants of perceptual objects warrant further studies. At least, the current study suggests that the effect of surface color change is related to surface-based attentional mechanisms. Our experimental paradigm combined with threshold measurement may further clarify the relationship between attentional mechanisms and summary representations. 

There was already ample reason to think that summary representations play an important part in visual cognition (Alvarez, 2011), and their critical role for change detection further confirms this. More work is needed to know what summary representations the visual system habitually forms, and change detection may provide a good testbed. For example, the symmetry of regions of a scene may be one summary representation, which could be tested with our paradigm. 

Supplementary Materials

Supplementary Movie - Supplementary Movie 

Movie S1 

Supplementary Movie - Supplementary Movie 

Movie S2 

Supplementary Movie - Supplementary Movie 

Movie S3 

Acknowledgments

We thank Patrick Cavanagh for helpful discussions. This work was supported by Global COE Program (D07) from JMEXT, Grants-in-Aid from JSPS, exchange scholar program from JSPS and AAS to JS, and an ARC Discovery Project and Future Fellowship Grant to AOH. 

Commercial relationships: none. 

Corresponding author: Jun Saiki. 

Email: [e-mail]. 

Address: Graduate School of Human and Environmental Studies, Yoshida-nihonmatsucho, Sakyo, Kyoto, 606-8501, Japan.