A new type of change blindness: Smooth, isoluminant color changes are monitored on a coarse spatial scale

Abstract

Abstract

Abstract:

Abstract  Attending selectively to changes in our visual environment may help filter less important, unchanging information within a scene. Here, we demonstrate that color changes can go unnoticed even when they occur throughout an otherwise static image. The novelty of this demonstration is that it does not rely upon masking by a visual disruption or stimulus motion, nor does it require the change to be very gradual and restricted to a small section of the image. Using a two-interval, forced-choice change-detection task and an odd-one-out localization task, we showed that subjects were slowest to respond and least accurate (implying that change was hardest to detect) when the color changes were isoluminant, smoothly varying, and asynchronous with one another. This profound change blindness offers new constraints for theories of visual change detection, implying that, in the absence of transient signals, changes in color are typically monitored at a coarse spatial scale.

Introduction

Changes in our visual environment are typically of greater interest and importance than unchanging information. The importance of changes to the visual system is seen in the priority given to processing unexpected events and the way in which we have evolved such that our attention is generally drawn to changing aspects of a scene. Despite this, there are situations in which changing stimuli fail to capture our attention and do not pop out from their surroundings, offering insight into the mechanisms underlying change detection. One example is inattentional blindness, when attention is diverted to another task. Robust change blindness can also occur when a change is simultaneous with an eye movement, image flicker, or other abrupt transient signal (Rensink, O'Regan, & Clark, 1997; O'Regan, Rensink, & Clark, 1999; Simons & Levin, 1997; Rensink, 2000). 

Change can also be masked from awareness by stimulus motion at the time of the change. Suchow and Alvarez (2011) used an array of circles arranged in an annular region about fixation that each made smooth, rapid transitions in color, luminance, size, or shape. When the elements of the stimulus were stationary, the changes were clearly perceived, but when they rotated about the fixation point, the smooth changes of individual elements appeared to slow dramatically or cease. 

Very gradual changes may also go unnoticed. Simons, Franconeri, and Reimer (2000) made slow (12 s), gradual changes to small sections of images of real scenes, either fading out an object from the scene or changing the color of one object. They found a similar degree of change blindness as when a typical disruption paradigm was used. Here, we demonstrate that profound change blindness to a smooth color change can occur without a simultaneous distractor transient or a masking motion signal. Unlike in Simons et al. (2000), the unnoticed changes were not restricted to a small region of the stimulus, and they were faster (0.33 Hz). This new form of change blindness demonstrates smooth color changes do not reliably attract attention even when there are no other changes in the scene and the subject is actively searching for these changes. 

Experiment 1: Detection of changing elements

We presented observers with an array of 64 colored squares that all changed color simultaneously. All color changes were isoluminant, and each square sinusoidally varied between a bluish and a yellowish version of a single color. When the changes of individual squares were in synchrony with one another (Synchronous Condition), the change in average color across the scene was apparent. However, when individual changes had random phases and were no longer in synchrony, the average color across the scene remained approximately constant, and the perception of change was dramatically reduced (Main Condition). While the change of a single attended square was clearly perceived, the global impression of change almost disappeared. The ability to perceive asynchronous changes was greatly enhanced by the introduction of a luminance change (Luminance Condition) or when the isoluminant change was abrupt rather than gradual (Abrupt Condition). 

We compared the impression of change elicited by these different conditions using two tasks. In Experiment 1, we measured subjects' change-detection performance in a two-interval forced-choice task, in which there were two brief stimulus intervals, one of which included changing elements while the other did not. We varied task difficulty within each condition by manipulating the number of elements that changed in the target interval; on any given trial, between 1 and 26 of the 64 elements were changing. 

Method

Participants

Twelve observers (five male), aged 25 to 30 years, took part; 11 were naïve to the purposes of the investigation (all observers except the author EG). All had normal or corrected-to-normal visual acuity and normal color vision as assessed using Ishihara (1990) and the Hardy-Rand-Rittler (HRR, fourth edition, published by Richmond Products) pseudoisochromatic plates. Observers provided informed consent, and the entire study was carried out in accordance with the guidelines of the Human Research Ethics Committee of the University of Sydney. 

Calibration, display system, and data collection

Stimuli were generated and displayed using Matlab (version 7) software with routines from PsychToolbox (Brainard, 1997; Pelli, 1997) on a MacBook Pro Intel Quad-Core i7 computer driving an AMD Radeon HD 6490M graphics card to draw stimuli to a 33 × 24 cm Sony Trinitron E220 cathode ray tube monitor refreshed at 60 Hz. Experiments took place in a darkened room, and the monitor was viewed from a distance of 0.57 m. The display system was calibrated using a ColorCAL colorimeter (Cambridge Research Systems Ltd., Rochester, UK). Changes in both the chromaticity and luminance of the screen with increasing R, G, and B values were taken into account when generating the experimental stimuli, using the method described in Goddard, Solomon, and Clifford (2010). 

During the experiment, subjects indicated their responses with a key press on a Cedrus RB-730 response pad, which allowed for precise reaction times to be collected along with subject responses. 

Stimuli

Stimuli consisted of 64 colored squares (each 1.9° visual angle in width) arranged in a square grid subtending a 20° × 20° visual angle. Black lines separated adjacent rows and columns of squares, and a thin grey cross divided the stimulus into quadrants, as shown in Figure 1A. Over time, individual squares changed color, alternating between a bluish and a yellowish version of a single color. These color changes were generated by simulating a flat, matte surface under illumination that sinusoidally modulated between two illuminants (CIE Standard Illuminants A and D65) that were scaled such that they had approximately equal photopic luminance. Rendered surfaces were randomly drawn from those used in Goddard et al. (2010) and included a broad range of colors. 

Figure 1

 

Experimental stimuli. A: Sample frame from Condition 1. B: Differences between the four conditions (Main, Synchronous, Luminance, and Abrupt).

In all conditions, the color of each square alternated at 0.33 Hz. In the first (Main) condition, the color changes were sinusoidal alternations with randomly chosen phases such that the changes across different squares were asynchronous. The remaining three conditions were variations on the Main Condition. In the Synchronous Condition, the squares' color changes each had the same phase as one another such that their alternations between the two illuminants were synchronous across the array of squares. Their starting phase was randomly chosen from the 60 frames in the stimulus cycle. In the Luminance Condition, a luminance change was introduced to each square by scaling one of the illuminants (CIE Standard Illuminant A) so that it had half the photopic luminance of the other. In the Abrupt Condition, each square was allocated a random starting phase from the sinusoidal modulation in the Main Condition as well as a random time in the 3 s cycle at which it would abruptly change to the color 180° further along in the cycle. Differences between the four experimental conditions are illustrated diagrammatically in Figure 1B. 

On each trial, stimuli included both changing and nonchanging squares as detailed below. Nonchanging squares were assigned the square's onset color for the duration of the stimulus. In the Synchronous Condition, nonchanging squares had colors corresponding to the same point in the stimulus cycle (the onset phase of changing squares) while in other conditions the nonchanging squares had a random distribution of phases. 

Procedure

Subjects performed a two-interval, forced-choice change-detection task, in which they reported which of two briefly presented intervals had a stimulus that contained changing elements. On each trial, the first stimulus was displayed for 500 ms, followed by a 500-ms black screen with a fixation cross, followed by the second stimulus for 500 ms, followed by a black screen with a fixation cross that remained until the subject indicated his or her response via the Cedrus response pad. Once the subject's response was detected, the screen remained black with the fixation cross for one second, after which the next trial began. 

Each subject completed two sessions of 240 trials; each session consisted of four blocks of 60 trials, corresponding to the four stimulus conditions described above. The order of the stimulus blocks was the same for both sessions and was counterbalanced across subjects. Each block of 60 trials contained 10 trials at each of six levels of difficulty, in which the number of changing elements within the test stimulus ranged from 1 to 26. The locations of the changing elements were randomly chosen for each of the 60 trials within a block. The same randomly chosen locations for the 60 trials were used across the different stimulus blocks with the order of the trials randomly interleaved for each stimulus block. 

Results and discussion

Results of Experiment 1 are shown in Figure 2. Subjects were most accurate in judging the interval containing changing elements when the changes were abrupt or included a luminance component and were least accurate for the asynchronous isoluminant color changes (Main Condition). A two-way within-subjects analysis of variance indicated a significant main effect of the condition on accuracy, F(3, 33) = 109.6, p < 0.001 and planned contrasts showed that performance was significantly worse when the changes were isoluminant, asynchronous, and smoothly varying than it was in the other conditions: Synchronous: F(1, 11) = 34.1, Luminance: F(1, 11) = 226.4, Abrupt: F(1, 11) = 284.5; p < 0.001 in each case. 

Figure 2

 

Change detection performance. Accuracy on the two-interval, forced-choice change-detection task is averaged across subjects (n = 12), and error bars indicate 95% confidence intervals of the between-subjects mean. The dashed line at 50% accuracy indicates chance performance.

The analysis of variance also revealed a weakly significant main effect of the number of elements, F(1.6, 17.7) = 3.9, p < 0.05¹, and a significant interaction between condition and the number of elements, F(15, 165) = 8.1, p < 0.001. The weaker significance of the main effect of the number of elements, coupled with the interaction between condition and number of elements, can be seen in the data in the way that increasing the number of changing elements increased detection performance in the Luminance and Abrupt Conditions but not in the other two conditions. In the Main Condition, the average performance decreased as the number of changing elements increased. This result may seem counterintuitive because, in this case, change detection performance decreased as information in the stimulus about change increased. However, in conditions in which the changes were asynchronous (all except the Synchronous Condition), increasing the number of changing elements also decreased the degree to which the average color of the stimulus changed, a characteristic which, in the absence of a robust transient signal, may account for the lower detection performance in the Main Condition as discussed in greater detail below. 

Experiment 2: Visual search for unchanging elements

In Experiment 2, subjects performed a visual search task, reporting the stimulus region containing unchanging squares. Searching for the lack of a feature is generally more difficult than searching for its presence (Wolfe, 2001), and this asymmetry is also evident in the search for unchanging elements among change (Rensink, 2007). We aimed to make the search difficult in this way in the expectation that this would best reveal differences between the conditions. 

Method

The participants, display system, and stimulus conditions were the same as in Experiment 1. 

Stimuli and procedure

In each trial, all squares changed over time in three quadrants of the stimulus while in the remaining quadrant most squares were constant: 12 of the 16 squares were randomly selected to remain a constant color throughout the trial while all other squares in the stimulus continued to alternate at 0.33 Hz until the subjects' responses. Subjects reported the quadrant containing the unchanging squares as quickly and accurately as possible, using a button press, and were given feedback on their accuracy after each trial. Feedback consisted of one second when the stimulus remained on the screen with the unchanging squares outlined, either in green or red, for correct and incorrect responses, respectively. Subjects made 64 judgments for each of the four conditions; the locations of the unchanging squares were equally distributed across the four quadrants within each condition. The entire 256 trials were completed twice in two sessions on separate days. In one session, the trial types were blocked according to condition, and in the other, the trials of different conditions were randomly interleaved. Half the subjects completed the blocked trials in the first session and the interleaved trials in the second session; for the remaining subjects, this order was reversed. Example trials from Experiment 2 are included in the online supporting information. 

Results and discussion

Results of Experiment 2 are shown in Figure 3. A two-way within-subjects analysis of variance of the time taken to indicate the location of the unchanging squares revealed a significant main effect of the condition, F(1.7, 18.7) = 12.71, p < 0.01², but no significant main effect of blocking the trial types together, F(1, 11) = 0.32, p = 0.58, and planned contrasts showed that the average response time when the changes were isoluminant, asynchronous, and smoothly varying (Main Condition) was significantly longer than for the other three conditions: Synchronous: F(1, 11) = 5.20, p < 0.05; Luminance: F(1, 11) = 43.9, p < 0.01; Abrupt: F(1, 11) = 69.4, p < 0.01. 

Figure 3

 

Response time (A) and accuracy (B) for the change localization task, averaged across observers (n = 12). Error bars indicate 95% confidence intervals of the between-subjects mean. Chance performance corresponds to an accuracy of 25%, and asterisks show where data for a condition vary significantly from those in the Main Condition (** p < 0.01; * p < 0.05).

Similarly, a two-way within-subjects analysis of variance of accuracy also revealed a main effect of the condition, F(1.4, 15.3) = 11.6, p < 0.01³, with no significant main effect of blocking trial types, F(1, 11) = 0.84, p = 0.38, and planned contrasts showed that accuracy was significantly worse when the changes were isoluminant, asynchronous, and smoothly varying than it was in the other conditions: Synchronous: F(1, 11) = 5.24, p < 0.05; Luminance: F(1, 11) = 22.8, p < 0.01; Abrupt: F(1, 11) = 21.8, p < 0.01. 

These results are consistent with the asynchronous isoluminant color changes (Main Condition) being more difficult to locate simultaneously across the scene. Locating the unchanging squares in the Main Condition was more difficult than in the other conditions as shown in the longer search times and lower accuracy. 

General discussion

What do these results imply about the mechanisms underlying visual change detection? We have demonstrated that change blindness can occur in the absence of any transients or stimulus motion to mask detection of the change. Unlike many other demonstrations of change blindness, the effect demonstrated here cannot be attributed to interrupted processing of the changing stimulus or misattribution of the transient signal. 

Detection of the changes can be restored by the introduction of luminance changes, by making the isoluminant changes abrupt rather than gradual, or by synchronizing the color changes to introduce a change in the average color across the stimulus. Each of these aspects can be used to constrain theories of the underlying mechanisms. 

Change blindness effect strongest at isoluminance

In the Luminance Condition, change detection and localization performance in the two experiments was markedly higher than in the Main Condition. These conditions were not matched in terms of change magnitude because the Luminance Condition contained a luminance change in addition to the color change, but the result is consistent with the higher salience of luminance changes (see below for a discussion of the relative detectability of luminance and isoluminant changes). Additionally, the increased detectability of the changes when they include a luminance component presents a means of integrating our results with those of Suchow and Alvarez (2011). 

Burr (2011) hypothesized that motion is a critical component of the illusions of Suchow and Alvarez (2011), arguing that their illusion occurs when dynamic signals associated with changes in the elements are subsumed by integration processes associated with motion perception. Consistent with this account is the observation that when the stimuli of Suchow and Alvarez (2011) were stationary the changes of individual elements were easily perceived. In our study, without any dynamic signals apart from the color change, the change blindness cannot be attributed to a similar integration of the color change signals by a dynamic mechanism processing a concurrent, unrelated change signal. 

What then is the relevant difference between the stimulus in our Main Condition and the stationary condition of Suchow and Alvarez (2011)? Our results imply that luminance may be the key difference. We found that adding a luminance change to the asynchronously gradually changing stimulus makes it more readily detected. Suchow and Alvarez (2011) used a color change that included a luminance component, making the stationary phase of their stimulus in which changes were apparent most similar to our Luminance Condition, consistent with our finding that such changes are more readily detected. 

Abrupt, isoluminant color changes can attract attention

We also found that isoluminant color changes were more easily detected when they were abrupt rather than gradual. Generally, this is consistent with the role of transient signals in attracting attention to changes in the scene and indicates that the isoluminant color changes were of sufficient magnitude to be easily detected when they changed abruptly. 

Previous findings on whether abrupt isoluminant color changes can attract attention have been mixed. Some previous results suggest that, unlike luminance changes, isoluminant color changes fail to attract attention (Burkell, 1986), do not preattentively pop out even when subjects are directed to search for them (Theeuwes, 1995), and isoluminant color changes fail to disrupt visual marking (Watson & Humphreys, 2002). 

Contrary to these results, Snowden (2002), using individually calibrated isoluminant color changes accompanied by a luminance noise mask to swamp any residual artifact, found that, for their stimuli, abrupt isoluminant color changes were as effective in attracting attention as large luminance changes. Similarly, Sumner, Adamjee, and Mollon (2002) reported that their precisely calibrated S-cone isolating stimulus changes elicited an exogenous cuing effect. 

In terms of whether isoluminant color changes can attract attention, our results are in agreement with this second group of findings. We cannot rule out the possibility that our stimuli included a small luminance artifact because we did not calibrate each color change for each subject but generated the isoluminant stimuli using standard observer measurements (see Methods). However, the more interesting finding in this study is that any ability of the abrupt isoluminant color changes to attract attention is dramatically reduced when the changes are gradual. 

Gradual, isoluminant color changes are detected on a coarse spatial scale

Our results indicate that changes affecting the average statistics across a scene are more likely to be detected. When the isoluminant color changes were in phase with one another (Synchronous Condition) and the average color of the scene varied over time, the changes were more apparent. Previous results suggested that color changes in a scene are more easily detected when accompanied by a change in the average color across a scene. Saiki and Holcombe (2012) showed subjects an array of dots moving left or right, colored red or green, which evoked a perception of two transparent surfaces moving across one another. At some point in time, all dots swapped color, and when the color and motion of the dots were randomly paired such that the average scene statistics did not change with the color swap, observers were virtually blind to the change. Rich and Gillam (2000) showed subjects an array of six lines of different colors rotating in depth, which could change color when they moved behind an occluder. They found that observers were far better at noticing when a new color was substituted for an old one than when the lines swapped position. In both these previous examples, the elements were moving at the time of the change, which, as for the Suchow and Alvarez (2011) study, may have resulted in any transient signals from the color change being attributed to the motion signal. In the case of Rich and Gillam (2000), the occluder also masked any transient signal that would have resulted from the change. Our results add to these previous findings by demonstrating that even when any transient signals from the color changes are not masked and cannot be attributed to stimulus motion, they are not necessarily effective in attracting attention. 

This novel result is consistent with a spatially coarse color change detection mechanism in which changes in the average color across a region are detected, but color changes on a local scale may go unnoticed when the observer's attention is not focused on the area. In this scheme, attention is guided to regions of change by transient signals or changes in the average color on a coarser scale. Such a mechanism would not be engaged by the changes in our Main Condition, in which the average color remains approximately constant, and the changes are not associated with a transient signal. 

This interpretation can also account for the finding in Experiment 1 that in the Main Condition, unlike in other conditions, change detection performance decreased as the number of changing elements increased. We think this finding is most parsimoniously explained by the fact that when the number of changing elements was small the average color of the stimulus changed over time to a greater extent than when the number of changing elements increased. This result is consistent with a spatially coarse color change detection mechanism, which, at the brief presentation time used in our experiment, averages over an area approaching the entire scene. If color changes were detected on a finer spatial scale, increasing the number of changing elements would increase the probability of a detectable change occurring within the integration area of the change detection mechanism, producing increasing, rather than decreasing, change detection performance. 

The implication that any isoluminant change detection mechanism operates on a coarse spatial scale cannot be directly linked to the spatial and/or temporal acuity for chromatically defined stimuli, both of which are lower than for luminance-defined stimuli. The rate of alternation of our stimuli (0.33 Hz) is well above the perceptual detection threshold for chromatically alternating stimuli, which is around 15–19 Hz (Wisowaty, 1981; Kelly, 1983; Holcombe & Cavanagh, 2001). Similarly, the spatial scale of our stimuli (each square 1.9° in width) was well within the spatial acuity of mechanisms underlying isoluminant discrimination, which have been estimated for sinusoidally modulating stimuli at around 22 cycles/degree in the fovea and around 12 cycles/degree at an eccentricity of 10° (Anderson, Mullen, & Hess, 1991). The fact that our stimuli are well within these limits means that change detection performance is not limited by the spatial and/or temporal acuity of these detection mechanisms, implying that there is averaging over a much coarser scale in the representation of these stimuli at the level of change-detection mechanisms. The results presented here do not allow a precise estimate of the spatial resolution at which these changes are detected, but variations on the stimulus used here could be used to further constrain the spatial resolution of change detection for isoluminant stimuli. 

Relation to color constancy

Although not essential to interpreting our results, it is also worth noting that a change in average color across a scene is associated with changing illumination. The Synchronous Condition is broadly consistent with a constant scene under changing illumination although it is not a typical natural scene because the stimulus always included some unchanging squares, which were randomly distributed throughout the stimulus. However, even if the mechanisms responsible for color or object constancy are weakly stimulated by this stimulus they cannot provide a simple account of the failure to detect change in the Main Condition. 

Traditionally, the study of color constancy has focused on how the visual system achieves a representation of surface color that is constant under changing illumination, often termed “discounting” the illuminant color. If changes in the illuminant were unimportant to the visual system, and so discounted and not noticed, there would be a decreased sensitivity to change in the Synchronous Condition compared with the Main Condition. Here, we found the opposite pattern of results, consistent with the color of the scene's illumination being perceived rather than discarded. While not the traditional focus of color constancy research, this idea is not unprecedented. The notion that the visual system transforms the retinal signal into dual representations of surface and illuminant color is supported by experimental work and has been incorporated into models of how color constancy is achieved (MacLeod, 2003; Brainard, Brunt, & Speigle, 1997; Logvinenko & Maloney, 2006; Smithson, 2005). 

Conclusions

In summary, we find that changes that occur across the entire visual scene may not be noticed even where they are not masked by transients or stimulus motion. This novel finding offers new insights into mechanisms underlying visual change detection. Change-detection mechanisms appear to be particularly tuned to transient signals and changes in the average scene statistics, and poor change-detection performance results when neither of these cues is available. Further work is needed to determine under what circumstances a lack of change in average scene statistics results in impaired detection of local changes and to find the spatial extent to which different scene statistics are averaged at the level of change detection. Our results also imply that changes in color are monitored on a much coarser spatial scale than predicted by spatial acuity for chromatically defined stimuli. 

Supplementary Materials

mov S1 

mov S2 

mov S3 

mov S4 

Supplementary materials

Supplementary online material (Movies S1–S4): Example stimuli from Experiment 2. For each condition, subjects were required to identify the stimulus quadrant in which 12 of the 16 squares do not change color over time. Subjects took longest to respond and were least accurate in the Main Condition (Movie S1), in which the color changed smoothly over time and the changes were isoluminant and out of sync with one another. This result is consistent with the subjects' accounts that in this condition the changes across all squares are not obvious at a glance but are easily perceived with serial searching. The changes are more easily detected when the changes are synchronous (Movie S2), include a luminance change (Movie S3), or are abrupt (Movie S4). Note that these videos are intended as an approximate demonstration and were calibrated for our display system. Deviations from isoluminance, and the introduction of any flicker from the compression process make the change blindness less compelling. 

Acknowledgments

This work was supported by the Australian Research Council Centre of Excellence in Vision Science. 

Commercial relationships: none. 

Corresponding author: Erin Goddard. 

Email: [e-mail]. 

Address: School of Psychology, The University of Sydney, Sydney, Australia. 

Footnotes

¹  Degrees of freedom corrected using Greenhouse-Geisser Epsilon after Mauchly's test indicated that the assumption of sphericity had been violated, χ² = 43.1, p < 0.01.

Footnotes

²  Degrees of freedom corrected using Greenhouse-Geisser Epsilon when Mauchly's test indicated that the assumption of sphericity had been violated, χ² = 18.6, p < 0.01.

Footnotes

³  Degrees of freedom corrected using Greenhouse-Geisser Epsilon when Mauchly's test indicated that the assumption of sphericity had been violated, χ² = 20.2, p < 0.01.