An Unbinding Problem? The disintegration of visible, previously attended objects does not attract attention

Abstract

In seven experiments, observers searched for a scrambled object among normal objects. The critical comparison was between repeated search in which the same set of stimuli remained present in fixed positions in the display for many (>100) trials and unrepeated conditions in which new stimuli were presented on each trial. In repeated search conditions, observers monitored an essentially stable display for the disruption of a clearly visible object. This is an extension of repeated search experiments in which subjects search a fixed set of items for different targets on each trial (Wolfe, Klempen, & Dahlen, 2000) and can be considered as a form of a “change blindness” task. The unrepeated search was very inefficient, showing that a scrambled object does not “pop-out” among intact objects (or vice versa). Interestingly, the repeated search condition was just as inefficient, as if participants had to search for the scrambled target even after extensive experience with the specific change in the specific scene. The results suggest that the attentional processes involved in searching for a target in a novel scene may be very similar to those used to confirm the presence of a target in a familiar scene.

Introduction

At any given moment, the visual world appears to be filled with a number of recognizable and actively recognized objects. You may look out the window and see a field, a tree, a cow, and a stream, all of which seem to be perceived and recognized. However, work from a number of laboratories suggests that it is not straightforward to describe the relationship between what we see and the stimulus that gives rise to that perception. The most dramatic demonstrations of this apparent poverty comes from “change blindness” experiments in which observers fail to notice substantial and clearly visible changes in natural scenes (Rensink, O’Regan, & Clark, 1997; Rensink, 2000a; Simons & Levin, 1997). As long as local transients are masked (O’Regan, Rensink, & Clark, 1999), observers can fail to notice objects moving or disappearing or even people changing identity (Simons & Levin, 1998). 

Such results seem striking because they seem to show an insensitivity or even blindness to objects that are right in front of your eyes. Other tasks suggest a comparable failure. Changes made to static stimuli go unnoticed if they are made while the observer is making an eye movement (Grimes, 1996; Irwin, 1996) (but see Carlson-Radvansky, 1999; Henderson & Hollingworth, 1999). Salient stimuli, presented at fixation, can go unreported if the subject is performing another task and is not aware that the stimuli might appear (“inattentional blindness”: Mack & Rock, 1998a, 1998b). 

Here we have performed a new series of experiments that also reveal an inability or unwillingness to use information about a stimulus to improve performance. Our basic experimental procedure is derived from the repeated search tasks of Wolfe, Klempen, and Dahlen (2000), as illustrated in Figure 1. In a standard visual search experiment, observers look for a target among a variable number of distractor items. The observer responds that a target is either present or absent, the stimulus vanishes, and the next trial consists of a new search for the same target in the midst of different distractors. A letter search of the sort shown in Figure 1 will produce reaction time (RT) × set size slopes of about 30–40 msec/item on target-present trials if the letters are large enough to be resolved without requiring eye movements. 

Figure 1

 

In standard search, a subject searches for a fixed target (here “E”) on a series of independent trials. In repeated search, the search display remains constant from trial to trial. The designated target changes on each trial. Here the designated target is shown in the center of the display.

In repeated search tasks, the visual search stimulus can remain constant over hundreds of trials. On each trial, the observer is given something new to search for. In principle, observers could develop strategies based on their accumulating knowledge of the display (e.g., the A is always on top of the display.). The question of interest is whether repeated search through a stable set of items makes subsequent search more efficient. Looking at Figure 1, on the first repeated search trial, the observer would look for and would fail to find the letter r. In order to determine that the R was not present, the observer would likely have attended to the A, binding its features together and linking that bound representation to the representation of A in memory, thus recognizing the letter. On the second trial, in this example, the designated target is a. Does the observer need to search again for the A or is that A available without another search? In previous uses of the repeated search task (Wolfe et al., 2000), we found that search efficiency failed to improve during the course of multiple searches through the same display. Even several hundred searches through the same unchanging set of 3 or 5 letters did not produce a significant improvement in search efficiency. Note that the search stimuli in these experiments simply remain visible. They do not flash. They are not masked. Nevertheless, search efficiency, as measured by the slope of RT × set size functions, does not improve. Slopes are the same as would be seen if observers simply performed a new search on each trial. Small priming effects can be seen. Observers are faster if the target letter is the same on successive trials but these effects fail to explain the unchanging slope. The same result is obtained with a variety of different sorts of stimuli: letters, novel closed curves, and objects with different sorts of probes identifying the target (direct visual match, word probes, auditory probes, etc.) (Wolfe, 1999, in press; Wolfe et al., 2000). 

In these repeated search tasks, as well as in change blindness paradigms, observers show a surprising unwillingness or inability to make use of readily available information from vision and/or memory. In each case, one can make excuses for the apparently suboptimal performance. Thus, several factors might contribute to subjects’ failures to notice change in change blindness experiments: 

•  
  Change detection requires a comparison between the current stimulus and the prior stimulus. The representation of that prior stimulus may be very fragile and short-lived (but see Rensink, 2000c).
•  
  Subjects do not usually know the nature of the change (but see Rensink, 2000c).
•  
  The location of the change is completely uncertain within the image. Once the subject knows the location of the change, there is no change blindness.
•  
  Subjects are confronted with a novel scene on each trial even though they may view the scene for some seconds.

Turning to a repeated search task, the failure to search efficiently through a well-learned display might be related to the change of target on each trial. This is a form of inconsistent mapping, a situation known to reduce search efficiency (Schneider & Shiffrin, 1977; Shiffrin & Schneider, 1977). 

Here we aim to eliminate or at least to minimize the impact of each of these factors. We present results of several versions of a repeated search task in which subjects are asked if one of a few clearly delineated objects changes drastically in an otherwise stable display. Generally, this will take the form of one object disintegrating while the subject views the scene. In these experiments, observers can search for change. In that way, they are similar to change blindness studies. However, these tasks differ from standard change blindness tasks in several ways: 

No comparison of one frame to another is needed. The task could be done as a simple search for a disintegrated object without reference to the preceding history of the scene. 

The nature of the change and the nature of the target are fixed and known throughout a block of trials. 

The relevant set size is known, the possible locations of targets are clearly specified and, consequently, stimulus uncertainty is strongly reduced. 

Because this is a version of repeated search, the display remains the same over a block of trials. 

Nevertheless, to anticipate the results, the efficiency of the search (as measured by RT × set size slopes) was the same when observers monitored a familiar display for a known change of a visible object as it was when they searched a new display for the presence of such an object. These results suggest that the phenomena of the inefficiency of repeated search and difficulty of change detection may reflect a common underlying property of visual processing. Specifically, both phenomena may arise from the tenuous connection between the visual stimulus and our representation(s) of that stimulus. We do not see change in an otherwise static image because we do not see the distal image. Our perceptual judgments are based on some proximal representation of that stimulus. Although transients in that representation may attract attention (Jonides & Yantis, 1988; Yantis & Jones, 1991), change per se is not a feature that is available to guide attention. Updating that representation in a manner that allows a change to be detected appears to require the deployment of limited capacity resources under the control of attention. This capacity-limited search through an old display seems to be very similar to capacity-limited searches through novel displays. 

Experiment 1: When “Chickens” Fall Apart

These experiments all follow a similar general plan. Observers view a stable visual stimulus over many trials. In the case of Experiment 1, the stimulus is a set of chickens (derived from Wolfe & Bennett, 1997) as shown in Figure 2. The basic question is whether attention can be directed to an object that falls apart or if that change goes unnoticed until attention happens upon it. These repeated search trials involving a stable set of items are compared with sets of unrepeated trials, displaying new objects in new positions on each trial. If the disintegrated object contained preattentive features (e.g., a different size) that attracted attention, then search for the disintegrated object would be expected to be efficient in both the repeated and unrepeated conditions. 

Figure 2

 

Sample stimuli for Experiment 1. Subjects viewed a static display of chickens. At the start of a trial, all chickens moved their feet. On target-present trials, one chicken disintegrated into a collection of line segments.

In Experiment 1, the start of a trial was indicated by a tone. When the tone sounded, the feet of each chicken moved from the standing to the running position or vice versa (compare chickens in the same positions in the two panels of Figures 2). This small change masked transients produced by the introduction of a target into the display (O’Regan et al., 1999; Rensink, O’Regan, & Clark, 2000). The leg movements of the chickens did not create new objects on each trial (Yantis & Hillstrom, 1994) any more than the moving legs of walkers or the wind-blown leaves of trees create new objects after each motion. The target was a destroyed chicken as shown in the second panel of Figure 2. The segments making up the chicken were reassembled into something that was not a chicken and was not a single, closed curve object. However, the destroyed chicken did have the same collection of local form features as the chicken. It did have additional line terminators that might have been expected to help in the detection of this target (Cheal & Lyon, 1992; Julesz & Bergen, 1983) though adding more of a feature does not necessarily lead to efficient search (for the case of line termination, see Taylor & Badcock, 1988). Thus, the second panel of Figure 2 represents a target-present trial. On target-absent trials, there would be no destroyed chicken. Note that, apart from the movement of the feet at the start of each trial, the chickens remained stationary and continuously visible on the screen. If the destruction of a clearly visible object was able to summon attention, then this task should be independent of set size. If not, then observers would need to search in a capacity-limited manner for the target. 

Methods and Apparatus

Stimuli were presented on Macintosh computers running Matlab with the Psychophysics Toolbox and VideoToolbox extensions (Brainard, 1997; Pelli, 1997). Stimuli were black on a white ground. Each chicken fit inside an invisible box that subtended 3 × 3 deg at a viewing distance of 57.4. Nine subjects, aged 18 to 55 years, were tested. All were paid volunteers who gave informed consent. All had normal or corrected-to-normal visual acuity, and all could pass the Ishihara color vision screen. 

Subjects were tested in two conditions. In the repeated search condition, described above, the objects remained visible and in fixed position throughout a block of trials. Targets were present on 50% of trials. The start of a trial was indicated by a tone. At that point, all chickens moved their legs once and, on target-present trials, one chicken was replaced by the destroyed chicken target. Observers responded with a key press to indicate if a destroyed object was present or not on that trial. The destroyed target object, if present, reverted to its undestroyed state after response. All other items remained continuously visible between trials. There was a 400-msec pause between successive trials. The location of destroyed targets was randomly chosen from trial to trial. In the unrepeated search condition, trials were independent as in a standard search task. New objects in new positions were presented on each trial. Half of the trials contained a destroyed target item. All stimuli vanished after the observer’s response, and there was a 400-msec blank period between trials. Three set sizes were tested: 4, 12, and 20. Observers were tested for 20 practice trials and 80 experimental trials at each set size in each condition. Set size was blocked in the unrepeated condition in order to match the repeated condition. Thus each observer was tested for 60 practice trials and 240 experimental trials per condition. 

Results

Figure 3 shows the average RT × set size functions for the repeated and unrepeated conditions. Error bars are plus/minus one standard error of the mean (SEM). The unrepeated condition is a near replication of experiments presented in Wolfe and Bennett (1997) showing that, when the preattentive feature information is kept similar in targets and distractors, object recognition requires attention. Here, the unrepeated search task produces RT × set size slopes that are quite inefficient and consistent with some sort of serial examination of the items (Sternberg, 1969; Treisman & Gelade, 1980). The critical question for this experiment is whether subjects perform faster and/or more efficient searches for the scrambled target when it represents the destruction of an existing object than when it simply appears at the start of a trial. The answer (see Figure 3) is repeated search is not more efficient than unrepeated search. 

Figure 3

 

Reaction time × set size functions for repeated and unrepeated conditions of Experiment 1. Note that repeated search is actually slightly slower and less efficient than unrepeated.

RTs less than 200 and greater than 4,000 msec were labeled as errors. An analysis of variance (ANOVA) reveals a significant main effect of experimental condition (F (1,8) = 6.6, p = .034), but the effect is in the wrong direction. Unrepeated RTs were somewhat faster than repeated. Unrepeated slopes were somewhat shallower than repeated, as seen in the significant interaction of condition and set size (F (2,16) = 8.13, p = .004). This effect was more pronounced for the target-absent trials, producing a significant triple interaction (F (2,16) = 5.78, p = .013). Unsurprisingly, the main effects of set size and target presence were highly significant. Error rates averaged 6%. They increased with set size and were somewhat greater in the repeated condition. Thus, the error rates mirrored the RT data. 

Discussion

This result extends the basic repeated search results of Wolfe et al (2000) in several interesting ways and makes a connection between studies of repeated search and studies of change blindness. First, this is a repeated search task in which the target remains the same from trial to trial, indicating that our previous failures to find increased search efficiency in repeated search were not due to inconsistent mapping (Shiffrin & Schneider, 1977). Second, it uses a relatively homogeneous search array, a factor that usually simplifies search (Duncan & Humphreys, 1989). Third, this is a repeated search task that, in principle, does not require object recognition. All that subjects needed to do was to detect the disintegration of an item. Nevertheless, the data suggest that detecting the scrambling of an object requires a limited capacity process similar to the processes that are needed to find an object in a novel display. The scrambling of an object is not adequate to summon attention, even when subjects know exactly what they are looking for. 

This is a repeated search task that can also be seen as a change detection task. As such, it is a task that shows a change blindness (or, at least, a failure of change pop-out) under circumstances where the nature of the change is known, where the scene is uncomplicated, and where there are only a few possible loci of change. Even with all these simplifications, subjects still behave as if they were searching the scrambled target without benefit of prior exposure to the unscrambled item at that location. 

It is interesting that performance was actually superior in the unrepeated condition. Perhaps it is harder to search for a change among otherwise stable objects than it is to search through a new set of objects. However, this result should not be over-interpreted because it does not appear reliably in subsequent versions of the experiment. Further discussion of the broader implications of this result will be deferred to the “General Discussion,” after the presentation of several replications and extensions of Experiment 1. 

Experiment 2: Bigger Chickens

Experiment 2 replicates Experiment 1 using larger stimuli and smaller set sizes. Perhaps only a limited number of objects can be simultaneously monitored. If so, a repeated search advantage might appear with smaller set sizes. The slopes of the RT × set size functions of Experiment 1 show that subjects did not need to fixate each item in order to determine if it was a chicken or not. Given about 4 eye movements/s, tasks that are limited by the need to fixate each item will yield RT × set size slopes of about 125–250 msec/trial on target-present trials, depending on one’s model of search (Horowitz & Wolfe, 1998). The slopes from Experiment 1 are in the 20–40 msec/item range typical of tasks that demand attention to each item in turn but that do not demand fixation. Still, the use of larger objects reduces the possibility that an acuity limitation or crowding effect might explain the results of Experiment 1. 

Methods

The stimuli were simple enlargements of those in the previous experiment. Each stimulus fit inside a 5.5 × 5 deg box. The entire stimulus display fit within a 29 × 30 deg field. Set sizes were 2, 5, and 8 items. Ten subjects were tested. Two were eliminated from data analysis because their RTs were markedly longer than were those of the other subjects. One of these two also had unacceptably high error rates. In all other respects, Experiment 2 was similar to Experiment 1. 

In order to determine that these stimuli were large enough to be identified at all locations while fixating centrally, we briefly presented single items at random locations and had subjects categorize them as chicken or not chicken. In this control experiment, stimuli were presented for 150 msec to prevent eye movements. Ten subjects each performed 20 practice and 100 experimental trials in this task. The average accuracy was 92%. This shows that these large chickens could be identified without eye movements at the eccentricities used in Experiment 2. 

Results and Discussion

Figure 4 shows the average RTs for the 8 remaining subjects in the repeated and unrepeated conditions of Experiment 2. As in Experiment 1, there is no evidence of any benefit due to extended exposure to the stimuli in the repeated search condition. This is supported by statistical analysis. None of the effects or interactions comparing repeated to unrepeated search were significant. The main effects of set size and target presence were highly significant. RTs less than 200 and greater than 4,000 msec were labeled as errors. Error rates, including these timing errors, for the 8 subjects included in the analysis averaged 3% and did not differ between repeated and unrepeated conditions. 

Figure 4

 

Average reaction time data for Experiment 2. Error bars show +/−1 SEM. Notice that repeated search results are very similar to unrepeated search results.

The conclusions of this experiment are essentially the same as those from Experiment 1. Even with large stimuli and small set sizes, there is no evidence that the destruction of an object attracted attention. Spatial uncertainty is reduced in this experiment because the set sizes are reduced. That makes no difference. Subjects were no faster or more efficient at finding a decomposed chicken in the repeated search condition than they were in the unrepeated search condition. The apparent superiority of the unrepeated condition in Experiment 1 was not statistically significant in this experiment and perhaps should be seen as a fluke. 

Experiment 3: Searching for the Chicken

Many visual search tasks are asymmetric, which means that a search for A among B is more efficient than a search for B among A (Frith, 1974; Treisman & Souther, 1985; Wolfe, 2001a). In one specific class of asymmetries, it is easier to find the presence of a basic feature among items that lack the feature than it is to find a target that lacks the feature among distractors that have the feature (Treisman & Gormican, 1988). A clear example comes from search for motion. It is much easier to find the moving stimuli among stationary stimuli than vice versa (Royden, Wolfe, & Klempen, 2001). Perhaps object coherence is such a feature. Perhaps attention would be attracted by the appearance of a coherent object from among collections of chicken fragments even if attention were not attracted to the destruction of a chicken among other chickens. In Experiment 3, the roles of target and distractor were reversed from Experiments 1 and 2. In this task, subjects search for the chicken among the fragmented chicken distractors. Again, we compare repeated and unrepeated search. In repeated search, the fragmented distractors remained present for a block of 100 trials. They were jiggled slightly from trial to trial in order to mask the transients produced by the appearance of the target. Methods and stimuli were identical to Experiment 2. Eight subjects were tested. 

Figure 5 shows the average RTs for Experiment 3. The results mirror the results for Experiments 1 and 2. There is no evidence that continued exposure to the distractors in the repeated search condition conveyed any benefit in the search task. No main effects or interactions with the condition variable were statistically significant. The main effects of set size and target presence were significant (p < .01 in all cases). Error rates averaged 3% and did not differ significantly between repeated and unrepeated conditions. Furthermore, the RT × set size slopes are fairly steep and comparable to those observed in Experiments 1 and 2. We conclude that the appearance of an object from a field of fragment clusters is no more likely to attract attention than is the fragmentation of an object in a field of coherent objects. 

Figure 5

 

Average reaction time data for Experiment 3. Error bars show +/− 1 SEM. As before, repeated search results are very similar to unrepeated.

Experiment 4: Detection of an Object in a Continuous Texture

In Experiment 4, we press the logic of Experiment 3 one step further. Looking again at Figure 2, it could be argued that the fragmented chicken, although not a good chicken, is still an object of sorts. It is a cluster that can still be segmented from the blank background. If you have a number of these items on the screen, they can be enumerated as objects. Perhaps attention could be guided to the appearance of an object if it were the only object present. Indeed, it has been proposed that the onset of a new object attracts attention (Yantis, 1993; Yantis & Hillstrom, 1994; Yantis & Jonides, 1996). Accordingly, in Experiment 4, subjects looked for the appearance of a chicken target in a continuous field of fragments. The stimuli are illustrated in Figure 6: 

Figure 6

 

In Experiment 4, the target was a chicken that appeared out of “chicken soup.”

Method

In both repeated and unrepeated conditions, the subjects’ task was to state if the intact chicken was present or absent. For the unrepeated search condition, the panel on the left of Figure 6 would represent a target-absent trial, whereas the panel on the right would represent a target-present trial. In the repeated search condition, a display like that on the left of Figure 6 was continuously visible during a block of trials. When a tone sounded, the entire display was shifted a few pixels to the left or right to mask transients. This did not disrupt the perceptual continuity of the stimulus. It merely appeared to be a texture moving a bit to the left or right. On target-present trials, a chicken appeared when the tone was sounded. On target-absent trials, new random fragments were presented. The transients in the present and absent trials were thus roughly equated. When the subject responded, the display reverted to the state shown on the left of Figure 6. Set size is a problematic concept in this display. Display size was used in place of traditional set size. 

Displays were composed of square tiles of chicken fragments. Each tile was 2.75 deg on a side. Display sizes were 3 × 3 tiles (8.25 × 8.25 deg), 4 × 4 (11 × 11), and 5 × 5 (13.75 × 13.75). A chicken target was composed of a 2 × 2 (5.5 × 5.5 deg)-sized region. In order to have slope values that were roughly comparable to the slopes of traditional RT × set size functions, a set size approximation was derived by dividing the area of the display by the area of the target. This yielded estimated set sizes of 2.25, 4, and 6.25. 

After 20 practice trials, subjects were tested for 100 trials at each display size (set size). Targets were present on 50% of trials. Eleven subjects were tested. 

Results

Figure 7 shows average RT data for Experiment 4. The steep slopes show that the target did not attract attention in a manner that was independent of the size of the background texture. Both repeated and unrepeated conditions were very inefficient. Introspectively, this was rather surprising because the task feels easy and the target seems very salient, once found. However, even experienced subjects (e.g., the authors) produced RTs that, like the RTs shown here, were strongly dependent on the size of the texture array. 

Figure 7

 

Average reaction time data for Experiment 4. Error bars show +/− 1 SEM. Set size is actually stimulus area, in this case (see text).

Error rates averaged 4% in this experiment and did not differ between repeated and unrepeated conditions. 

None of the main effects or interactions with set size are significant (ANOVA; p >.09 in all cases). However, the numerically large difference between the slopes for repeated and unrepeated target-present conditions reflects a confound worth mentioning. In this experiment, target eccentricity was correlated with set size because set size was defined by the area of the display. Eccentricity is known to have a substantial effect on RTs in visual search experiments (Carrasco, Evert, Chang, & Katz, 1995; Carrasco & Yeshurun, 1998; Cheal & Lyon, 1989; Wolfe, O’Neill, & Bennett, 1998). Moreover, in the repeated search conditions, subjects could, and probably did, fixate the center of the unchanging texture. Thus, targets appearing near fixation might be expected to appear in an attended location more frequently in the repeated than in the unrepeated condition. Expected changes at the locus of attention will not require search. Indeed, if we restrict the analysis to the items that would abut the presumed point of fixation, then repeated target-present slopes drop to 11 msec/item. Unrepeated slopes remain at an inefficient 66 msec/item because preferential fixation would not have been as effective. In contrast, if we restrict analysis to those locations that do not abut fixation, the slopes for repeated and unrepeated target-present conditions are 70 and 82 msec/item, respectively. 

The results of Experiment 4 would seem to be in some conflict with the results showing that new objects capture attention (Yantis, 1993; Yantis & Hillstrom, 1994; Yantis & Jonides, 1996). It may be that the chicken is not a sufficiently new object to summon attention in the way that onset objects do. The chicken is defined only by a rearrangement of existing local features. Apparently, an object onset of this sort is not adequate to attract attention. Such a target must be found by inefficient search. 

Experiment 5: The Unbinding of Color and Orientation

All of the experiments reported thus far in this work use the same set of stimuli. One would like to know if the results obtained with the chickens and their fragments apply more widely. Recognition of the chicken stimuli requires an appreciation of the spatial relationship between contours. The binding required to put a complex form together may be different from the binding required to coordinate information about two fundamentally different but spatially overlapping features such as the color and orientation of a region. Accordingly, in Experiment 5, the stimuli are defined by the conjunction of color and orientation. These stimuli, originally used by Wolfe and Bennett (1997), are shown in Figure 8. Wolfe and Bennett argued that these pluses were represented as unbound bundles of features prior to the arrival of attention. Thus, preattentively, each of these objects would be a bundle of red, green, vertical, and horizontal. Only with the binding process made possible by attention would the subject explicitly represent the item as, for example, green-vertical and red-horizontal. 

Experiment 5 asks about the fate of that color X orientation binding in postattentive vision. Suppose, as on the left side of the figure, that all of the items are of the same sort: here, green-vertical and red-horizontal. If these stimuli remained visible in a repeated search paradigm, would it become easier to search for a red-vertical, green-horizontal target? Wolfe and Bennett showed that standard unrepeated search for such a target was very inefficient. Experiment 5 compares unrepeated and repeated conditions for this task. 

Methods

The stimuli used were red and green pluses that fit in a 3.2 × 3.2 deg box. All stimuli were presented on a black background that was 29 × 30 deg at the 57.4 cm viewing distance. Distractors were green-horizontal/red-vertical pluses. The target was a red-horizontal/green-vertical plus. Thus, it would have been adequate to monitor the display for either red-vertical or green-horizontal lines. Set sizes were 4, 8, and 12. After 20 practice trials, subjects were tested for 80 trials at each set size. Targets were present on 50% of trials. In the unrepeated condition, a new display was presented on each trial. In the repeated condition, the same set of distractor pluses remained visible for the 100-trial block. A tone indicated the start of each trial. At the start of a trial, in the repeated search condition, the display was displaced a few pixels to the left or right to mask transients. This made the entire field appear to shift smoothly in one direction. It did not disrupt the subjective temporal continuity of the display. Ten subjects were tested. 

Results

As may be intuitively obvious from Figure 8, this is not an easy search task. Error rates were high, averaging 10% for the repeated condition and 12% for unrepeated. The difference is not significant by a paired-sample t test. Were we to adopt our normal criteria of no more than 10% total errors and no more than 20% errors in any cell of the experiment (set size × target present/absent × repeated/unrepeated), 7 of 10 subjects would be disqualified. Rather than do that, Figure 9 shows the average error rates as well as the average RTs for the correct target-present and target-absent trials. Both RTs and errors were analyzed for differences between repeated and unrepeated conditions. Both conditions are very inefficient. There is no main effect of condition on RT [F (1,9) < 1] but the effect of condition on slope is reliable [condition × set size interaction, F (2,18)=7.2, p = .005]. Note, however, that, as in Experiment 1, it is the repeated condition that is less efficient than the unrepeated condition, rather than the other way around. A similar pattern is seen in the errors, though none of the main effects or interactions of condition are statistically reliable. Certainly, there is no evidence for a benefit of prolonged exposure to the stimuli. If anything, the unrepeated task is somewhat easier. 

Figure 8

 

Stimuli for Experiment 5. The target would be a red-vertical/ green-horizontal item.

Figure 9

 

Search for a red-vertical / green-horizontal plus among green-vertical/ red-horizontal pluses is very inefficient in both Repeated and Unrepeated conditions. Upper panel shows the steep slopes ofthe RT × Set size functions in both conditios. In the lower panel, gray bars show errors for the Repeated condition. White bars show slightly fewer errors for the Unrepeated condition. There is no advantage to repeated search in this task.

Discussion

The results of Experiment 5 show that there is no benefit from extended exposure to specific pairings of color and orientation. If the observers are asked to monitor the display for changes in those pairings, they must search inefficiently. A change from red-horizontal to green-horizontal does not attract attention. In this experiment, as in Experiment 1, the repeated condition with its continued exposure to a set of stimuli produces less efficient search than does the unrepeated condition with its new stimuli on each trial. This might suggest some sort of masking role for the continuously present objects but, as noted before, this result is somewhat elusive and does not appear in every experiment. 

Beginning with Treisman (Treisman & Gelade, 1980), many have suggested that preattentive vision consists of an unbound soup of basic features. The present result suggests that prolonged exposure does not change this into a representation in which changes in the relationships between features can attract attention. This failure to notice a change in the relationship between two features may be unsurprising in light of Rensink’s results showing a failure to notice changes in single simple features, such as orientation and luminance polarity (Rensink, 2000c). Still, there might have been something special about repeated search for the conjunction of two features. Processing of conjunctive relationships, unlike processing of simple feature values, has been thought to require attention. As such, it might have been a more sensitive assay of a persistent effect of attention in repeated search. However, the results show that subjects must search for change in conjunctive properties just as they must search for all the other changes in these repeated search tasks. 

Experiment 6: The Disintegration of “Real” Objects

In Experiment 6, in a further effort to show some advantage of sustained exposure to stimuli in this paradigm, observers searched for scrambled versions of realistic objects. As shown in Figure 10, the items in the search display were realistically colored objects designed using Home Designer 3.0 software (Data Becker, Needham Heights, MA). Twelve objects were selected (a gift, a coffee machine, a cup on a plate, a laptop, a fruit bowl, a radio, a clock, a toaster, a hat, a candle, a TV, and a guitar). A scrambled version was made for each of these. The scrambled hat in the right panel of Figure 10 is an example. 

Figure 10

 

Sample stimuli for two frames of Experiment 6 for the repeated search condition. Subjects viewed a static display of 2, 4, or 6 objects. At the start of a trial, all the objects rotated 10 deg to the right (or to the left). On target-present trials, one object (in this case, the hat) was transformed into its scrambled version.

The normal and scrambled objects needed to meet two criteria. First, they had to be recognizable without requiring fixation. Second, the scrambled objects could not be pop-out stimuli that attracted attention because of some irrelevant attribute (e.g., It would not be interesting to find out that attention was attracted by the presence of a scrambled object that was twice as large as the other objects). To show that the objects were recognizable without the need for fixation, single objects were presented in unpredictable locations for 150 msec. Nine subjects identified these items with an average 98% accuracy. To show that scrambled objects did not pop-out, a standard search task presented scrambled target objects among various normal objects as distractors. This task, performed on 9 subjects, yielded a slope of 25 ms/object in the target-present condition, and 54 ms for the target-absent condition, comparable to standard inefficient searches, such as a search for a T among Ls (Wolfe, 1998). ANOVAs performed on the target-present and target-absent correct RTs showed significant effects of set size in both cases (F (2,16) > 49, p < .0001). A further item analysis performed on the mean slope per object revealed no significant differences between the objects (F<1). Thus, none of the scrambled objects attracted attention in a standard search task. Does anything change when subjects search through a continuously visible array of objects in a repeated search task? 

Method

To address this issue, Experiment 6 consisted of a repeated search and two versions of an unrepeated search task. Those tasks differed in their virtual set sizes. There are two sorts of set sizes that are relevant in this experiment. There is the number of items on the screen (the standard definition of set size). There is also a virtual set size consisting of all the possible items that might be on the screen in a given block of trials. For the repeated search condition, n different objects (set size 2, 4, or 6) out of 12 possible objects were chosen at random for each block for each participant. Those objects were continuously present on the screen during an entire block of 288 trials. Thus, for set size 4, one subject might see a gift, a radio, a clock, and a toaster for 288 trials. Another subject would see a different set of 4 objects drawn randomly from the set. As in previous repeated search conditions, a tone indicated the start of a trial. At that moment, one object would be replaced by its scrambled version on 50% of the trials. Subjects searched for the presence of the scrambled object. A small (10 deg) object rotation occurred at the start of the trial to mask the transient produced by the appearance of the scrambled object. 

For the unrepeated search conditions, the display changed at the start of each trial. A random selection of objects was presented. In the unrepeated-6 condition, the virtual set size was 6. This meant that only the positions of the objects were changed when the set size was 6. For the unrepeated-12 condition, the 2, 4, or 6 items on each trial were drawn from a virtual set of 12 items. Items could be in any position on the screen. 

Fourteen subjects were paid for their participation. Before the experiment began, they were familiarized with the objects and their respective scrambled versions. For the experiment, they were told to answer as quickly and as accurately as possible whether or not a scrambled object was in the display. Set size presentation was blocked (because it had to be blocked in the repeated case) with 2, 4, or 6 objects arranged on an imaginary circle that subtended a visual angle of 15 deg. The order of conditions (repeated, unrepeated-6, unrepeated-12) and of three set sizes was pseudo-randomized across subjects. Each condition per set size block was composed of 288 trials. Half of these were target-present trials. The full experiment consisted of 2,592 experimental trials plus 48 practice trials and took about 2 hours. 

Results & Discussion

RTs less than 200 ms and greater than 3,000 ms were labeled as errors. Error rates, including these timing errors, averaged 4.3% and did not differ significantly across condition or set size. Data from 2 subjects were discarded from the analysis because of a high error rate (> 15%). Even with these realistic stimuli, the repeated and unrepeated conditions did not differ in their mean RT or slopes (see Figure 11). The ANOVA shows the usual set size effect (F(2,22)=24.8, p <.0001) but neither the effect of condition (F(2,22)=1.52) nor the interaction with set size (F(4,44)=1.26) were significant. In the target-present condition, slopes were 18 ms, 25 ms, and 26 ms, respectively, for the repeated, unrepeated-6, and unrepeated-12 conditions. The same pattern of results was observed for the target-absent condition where the slopes were 40 ms, 48 ms, and 53 ms, respectively. Finally, Figure 11 shows that subjects did not show effects of uncertainty in the unrepeated conditions. It did not matter if the objects, present in the display, were drawn from a virtual set of 6 or 12 items. 

Figure 11

 

Average RT data for Experiment 6. Errors bars show +/− 1 SEM. There are no differences between repeated search results and unrepeated search results.

Experiment 7: Objects in Scenes

In a final effort to find some advantage in monitoring an old stimulus over searching through a new one, Experiment 7 used realistic objects in a realistic scene. One of the surprising aspects of the basic repeated search result is the failure to gain any apparent advantage from memory for position information. For example, the letter A is always at the top of the display in the letter search version of the repeated search task shown in Figure 1. It would seem that observers should use their memory for the letter and its position to go directly to the target when queried about an A. However, the data show the same dependence on set size for searches for a well-learned A as for an A in a new display. Perhaps embedding stimuli in a realistic scene and cueing the target object that could be destroyed would encourage observers to use this position information. 

Methods

Stimuli were similar to the realistic objects of Experiment 6. Now, however, each was placed in one of six possible locations in a room scene as shown in Figure 12. There were 10 possible objects (a parrot, a gift, a laptop computer, a coffee machine, a hat, a radio, a guitar, a fruit bowl, a clock, and a car). Each object was rendered in a variety of different viewpoints in order to fit appropriately into each of the six possible locations. Each item subtended between 3 and 4 deg on a side and existed in normal and scrambled versions. The scene was rendered using the image synthesis software Home Designer 3.0 (DataBecker) and subtended 25 deg × 19 deg of visual angle. 

Figure 12

 

Sample stimuli for Experiment 7.

There were two versions of the repeated condition. In the standard repeated condition, the observer searched for a scrambled object as in the previous experiments in this study. The upper panel of Figure 12 would be a target-absent trial for this task. Once the central cue was removed, the lower panel would be a target-present trial. 

In the cued repeated condition, a 100% reliable picture cue, presented at fixation, told the subject which object would be scrambled if any object was scrambled on that trial. Thus, in Figure 12, the cue informs the observer that the toy parrot is the only possible scrambled object on this trial. This is a target-present trial because, as can be seen in the lower panel of Figure 12, the parrot is scrambled. There are two sorts of target-absent trials in the cue-repeated case: The cue might refer to an object in the scene but not scrambled (e.g., the violin in Figure 12) or the cue might refer to an object not in the scene (e.g., the clock). 

Two unrepeated conditions were tested for comparison. The scene remained constant in these conditions but new objects, drawn from the full set of 10 objects, were presented on each trial. In the standard unrepeated condition, observers searched for a scrambled object. In the cued unrepeated condition, observers searched for a scrambled object whose identity was shown in the cue at fixation as in the lower panel of Figure 12. Note that there were no trials where an uncued item was scrambled so, in principle, the cued and standard tasks could both be performed as searches for a scrambled object. 

Observers were tested in blocks of 320 trials. Set size was fixed within a block at either 3 or 6 items. Each observer was tested in the four conditions (cued/standard × repeated/unrepeated) times two set sizes for a total of 8 blocks. Targets were present on 50% of trials. In the cued repeated and unrepeated conditions, half of the target-absent cues referred to unscrambled items in the display and the other half referred to items that were not in the display. In order to mask onset transients, the background was shifted in a manner consistent with a 10-deg viewpoint shift. In this experiment, the objects were not displaced or rotated in the repeated conditions. They remained stationary on the screen while the background moved. Sixteen observers participated in this experiment. Half began with the cue condition and the other half began with the standard condition. Repeated and unrepeated conditions for each set size (3 or 6) were blocked and counterbalanced among participants. 

Results

Outlier reaction times (200 msec < RT < 3,000 msec) were considered as errors. RT × set size functions for target-present trials are shown in Figure 13. Slopes, intercepts, and error rates for target-present and target-absent trials are shown in Table 1. 

Figure 13

 

Mean reaction times for target-present trials in Experiment 7.

Table 1

 

Slopes, intercepts, and error rates for the conditions of Experiment 7

Condition	Slope	Intercept	Error Rate %
Standard Repeated: Target Present	22	532	4.8
Standard Unrepeated: Target Present	35	504	4.1
Cued Repeated: Target Present	17	745	4.6
Cued Unrepeated: Target Present	29	746	5.8
Standard Repeated: Target Absent	64	495	1.0
Standard Unrepeated: Target Absent	98	422	1.1
Cued Repeated: Target Absent v1	13	907	6.1
Cued Repeated: Target Absent v2	43	602	1.8
Cued Unrepeated: Target Absent	57	795	2.8

 

v1 = Cued item is present but not scrambled; v2 = cued item is absent

Although one might think that it would be helpful to cue the observer to check just a single item, it is clear from Figure 13 that the cued RTs are substantially slower than the standard (F (1,15)=29.1, p < .0001). Slopes in the cued conditions are somewhat shallower than in the standard. If we pool across repeated and unrepeated conditions, there is a modest significant effect (F (1,15)=5.6, p =.032). Although the slopes also appear somewhat shallower in the repeated than in the unrepeated conditions, this apparent effect is not a significant trend in either the standard (F (1,15)=1.1) or the cued conditions (F (1,15)=1.2). Note that the repeated search slopes remain inefficient in standard and cued versions of the experiment. 

There were no significant trends in the error data (see Table 1). 

The target-absent trials roughly mirror the target-present if one averages the two types of cued-repeated absent trials. As shown in Table 1, when taken separately, those two types of target-absent trials produced quite different results. When the cue refers to an object present in the scene but not scrambled, participants took much longer to respond (968 msec) than when the cued item is absent from the scene (796 msec) (F (1,15)=36.9, p < .0001). Slopes also differed (F (1,15)=11, p < .01). When the cue refers to an object in the room, the search slopes are very similar whether the object is scrambled (cued repeated: target absent, 17 msec) or not (cued repeated: target absent, 13 msec). 

Discussion

The standard conditions of this experiment replicate the basic findings of the previous experiments. Prolonged exposure to objects, now in a naturalistic scene, does not enable observers to detect the scrambling of one of those objects without search. The slight advantage in the repeated condition might be due to the greater need in the unrepeated condition to segment objects from the background. If we had not restricted objects to a known set of locations in the unrepeated condition, it seems likely that the task would have been harder because the locations of possible objects would have been unknown. This would be roughly equivalent to increasing the set size and corresponds to the difference between searching for the scissors in your home, where you know the likely locations, and in someone else’s where you do not. 

The cued conditions are, perhaps, more of a surprise. Looking at Figure 12, it seems intuitively clear that it should be easier to determine if the parrot is scrambled than to determine if any of the 6 objects are scrambled. However, intuition fails in this case because it does not account for the time that it takes to decode the cue nor the speed with which observers can scan a known set of object locations. Apparently, the speed advantage lies with the full scan, and the only role of the cue is to slow search by forcing subjects to confirm that any scrambled item is the right scrambled item. 

General Discussion

In each of the experiments described here, we compared performance on a standard search task to performance on a repeated search task where the observers spent many minutes viewing the same set of objects. A similar pattern of results was obtained across a wide range of different types of stimuli. Repeated and unrepeated conditions produced very comparable results. Prolonged exposure to the stimuli conveyed no substantial benefit. For example, in Experiment 6 this means that subjects found a scrambled toaster just as fast in a novel display as in a display in which the toaster had been clearly visible for dozens of preceding trials. Only in Experiment 7 was there evidence of a modest advantage for repeated over unrepeated conditions. Even in that case, the slightly more efficient repeated searches still produced slopes that would be considered as evidence for a limited-capacity search process. There is no evidence that the scrambling of a visible object can serve as a cue to summon attention even in the cued object condition. If it did so, search slopes in the repeated condition should be close to zero, which they are not. 

The range of stimuli used here argues against a number of uninteresting accounts of this result. The plus and chicken tasks were searches for a target among relatively homogeneous distractors — completely homogeneous in the case of the pluses. Thus it is unlikely that the inefficiency of search can be attributed to distractor heterogeneity (Duncan & Humphreys, 1989). The stimuli run the gamut from the artificial pluses to cartoon chickens to real objects in realistic scenes, making it unlikely that the effects are dependent on specific stimuli. In the pluses and chickens experiments, all of the distractors were exemplars of the same type of object. In Experiment 6, all of the items were different object types. The chickens involved conjunctions of different spatial elements. The pluses involved conjunctions of two qualitatively different features; color and orientation. The object stimuli involve both sorts of conjunctions. It seems likely, therefore, that this is a general result. 

Obtaining these results requires that transients be masked. In fact, it may well be that we can live with our apparent insensitivity to the fate of clearly visible stimuli because, under normal circumstances, the transients that were hidden in these experiments would direct an observer to deploy attention to a changing object in the real world. It is important to note that the small, transient-masking changes in the display did not disrupt the perceptual stability of the repeated stimuli. These changes were the sorts of changes that occur naturally to objects. They do not disrupt the perceived continuity of these objects. A small part of each chicken moved. The chicken soup translated slightly. Objects rotated by a small amount and the scenes were subjected to a small viewpoint change. These manipulations are different from the sorts of manipulations that appear to create new objects in, for example, the work of Yantis and colleagues (Yantis, 1993). 

The results of the present experiments extend the repeated search results obtained in our previous work on postattentive vision (Wolfe et al., 2000). The present version of the experimental design has the virtue of being relatively disentangled from memory issues. In the previous experiments, observers viewed a set of objects and were queried about the presence or absence of a different test object on each trial. This is illustrated in Figure 10 where the observer in a standard repeated search task might be asked if a gift was present on trial 1 (yes), a violin on trial 2 (yes), a cat on trial 3 (no), and so forth. Such searches remained inefficient over many trials but, with that method, it is hard to know if a subject was performing a capacity-limited visual search or a memory search (McElree & Dosher, 1989; Sternberg, 1969). In the present experiments, the observers cannot pull an answer from memory. They must monitor the visible stimulus for a scrambled object. The data show that they were forced to perform a limited capacity search of the stimulus even when that stimulus had been present for many trials. 

As noted at the outset, these experiments link the repeated search paradigm with the work on change blindness (Rensink, 2000a; Rensink et al., 2000; Simons & Levin, 1997). These tasks can be described as a repeated search for change. The results make it clear that change blindness does not require an unknown change in an unknown location in an unfamiliar scene. Our results show that observers are insensitive to a known change occurring in one of a few possible locations in a familiar scene that remains otherwise stable over many trials. Indeed, if we consider a situation like the set size 4 block of Experiment 2, the same change will occur in the same location many times over the course of a 100-trial block. Nevertheless, an observer seems to search for the change each time it occurs. 

What do experiments of this sort tell us about the nature of what we see? Some have argued that we only see the current object of attention (e.g., O’Regan, 1992) (inattentional blindness, see Mack & Rock, 1998a). That position seems to deny the perceptual reality of a stable world filled with recognized, coherent objects. At the very least, this account requires some account of the grand illusion of perception (Noë, Pessoa, & Thompson, 2000). Alternatively, it has been proposed (e.g., Wolfe, 1999) that once attention is deployed away from a stimulus, the observer forgets the stimulus (dubbed “inattentional amnesia” in Wolfe, 1999) (see also Rensink, 2000b). However, while it may be true that some information is rapidly lost, this is not a useful account of these repeated search experiments. Even though subjects behave as if they must search in order to determine which object in a scene has been scrambled, there is no doubt that, after 100 trials, those subjects remember the objects and know their locations in the scene. 

An extension of Rensink’s (2000b) useful World Wide Web metaphor can provide one way to understand the results of the postattentive experiments described here. Rensink notes that your desktop computer does not store the contents of the Web. It simply knows how to reach out to acquire what you request when you request it. Similarly, he argues, you do not need to store a complete representation of the world, you merely need to know how to get the required information from the stimulus. However, after you request a Web page, it is represented on your desktop. Imagine, then, that you are viewing one of those Web portals that posts stock quotes and the current news. Suppose you want to know the state of the stock market. You can query the representation that is present on your screen but you know that it is out of date, even if only slightly. Things might have changed. To answer the question more accurately, it would be prudent to click on the reload button that updates the information from the Web. No matter what the capacity of your computer or the capacity of the Web, the act of reloading is limited by the capacity of your connection to the Web. 

Turning from the Web to vision, it seems clear that there is some postattentive representation of the visual world. Specifying the exact nature of that representation is difficult, subjective, and, fortunately, not critical for this argument. At a minimum, it is clear that you see something while the stimulus is present and that some visual memory of an attended scene persists after the scene is removed. If, as in the repeated search conditions presented here, a response can be based on a visual stimulus that is still present, this reloading hypothesis holds that it would be prudent to reload that visual stimulus rather than to base response on some previously created representation of that stimulus, no matter how faithful that representation might appear to be. Suppose that reloading the image is capacity-limited in the way that loading the image is capacity-limited even if the image that is being reloaded is very familiar. If that is the case, then repeated search and unrepeated search will produce comparable RT × set size slopes, as they do in the present experiments. 

Why might loading and reloading be similarly capacity-limited? We are used to the idea of “bottlenecks” between vision and memory (e.g., Sperling, 1960) and within visual processing (e.g., between “preattentive” and “attentive” processing, Neisser, 1967). We have tended to think about these bottlenecks in a feed-forward sense. Stimuli from the world must squeeze through a capacity-limited bottleneck before reaching internal representations in vision or memory. Maybe it would be better to think of the bottlenecks as operating in the other direction. Treisman’s feature integration architecture may imply this sort of reverse bottleneck if we imagine attention reaching back from the “master map” to earlier feature maps (e.g., Treisman, 1993). The idea is made more explicit in the reverse hierarchy theory by Ahissar and Hochstein (1997) and Hochstein & Ahissar (2001). They propose that feed-forward processes generate a crude, perhaps, preatttentive representation of the stimulus at relatively late stages in the hierarchy of neural processing stages in the visual system. The information for more detailed and elaborate analysis of specific stimuli is available at earlier stages in visual processing (e.g., fine orientation information available in primary visual cortex). To base a visual behavior on that information, however, the observer must reach back to these earlier stages. In this view, it is that reaching back that is the capacity-limited process. When you search as scene for the first time, you reach back and attend to objects in what appears to be a serial fashion, one object at time. When you reload a scene, you may be doing essentially the same thing. 

Conclusions

When searching repeatedly through a familiar set of stimuli, observers perform in a manner similar to the way that they perform when searching through a novel set of stimuli. Perhaps that is because the two tasks are more similar than we might have thought. When searching through a novel stimulus, observers must use attentional mechanisms to reach back to the stimulus to select object after object until they find the target or abandon the search. When asked about the presence of a target in a familiar scene, observers behave as though it would be prudent to reach back in the same way to the actual stimulus rather than to base behavior on a representation of the scene that might no longer be current. 

Acknowledgments

We thank Jennifer DiMase, Todd Horowitz, Ron Rensink and two anonymous reviewers for comments on a draft of this paper. This research was supported by grants from National Science Foundation (SBR-9710498) and National Institutes of Health (MH56020). 

Commercial relationships: None.