Visual search without attentional displacement

Abstract

The time needed to search for an object in a complex environment increases with the number of distracting stimuli, a phenomenon known as the “set-size effect.” This observation has led to the view that, during visual search, several attentional shifts are performed, suggesting that visual information is processed serially. In an attempt to find direct evidence for such attentional shifts, we implemented several dual tasks combining a covert visual search (CVS) task or a cued target detection task with a character reporting task which allowed us to determine, a posteriori, the attentional allocation. We found that, in the cueing task, subjects preferentially reported characters displayed at different locations, demonstrating that the attention spotlight actually shifted in this condition. In contrast, in both feature and conjunction CVS, subjects predominantly reported several characters flashed at the same location, whatever the delay between their presentations, indicating that, despite a clear “set-size effect,” attention remained static. These results demonstrate that CVS can be performed without shifting attention and that the “set-size effect” does not necessarily attest that visual information is processed serially. The present study therefore supports the hypothesis that parallel mechanisms are involved in visual information processing during visual search, in agreement with previous theoretical studies.

Introduction

Our everyday life is full of situations in which we have to find a particular object in a complex visual environment: a key on our desk, a signpost along the road, or a person in a crowd. The experimental counterpart of such a behavior is visual search, a task in which subjects have to detect the presence of a “target,” among a series of averting items, named “distractors.” In order to dissociate attention allocation mechanisms from orienting eye movements, visual search tasks are usually performed covertly, i.e., without any gaze displacement. 

Visual search tasks have been widely used to infer mechanisms underlying attentional allocation. Indeed, in easy (or efficient) covert visual search (CVS), the reaction time (RT) is constant, whatever the number of distractors. In contrast, in difficult (or inefficient) CVS, the RT increases proportionally to the number of distractors. This so-called “set-size effect” has been regarded as evidence for a serial processing of visual information, constrained by a step by step displacement of attention (Treisman & Gelade, 1980). However, theoretical studies have shown that interpreting this linear relationship between RT and the item number as evidence for the serial nature of the underlying processes is questionable. Indeed, both serial models and parallel models constrained by limited capacity lead to behavioral predictions about the set-size effect that are undistinguishable from each other (Townsend, 1971). More recently, it has been shown that some parallel models, based on the framework of signal detection theory (Verghese, 2001), are able to mimic the set-size effect and other common features of visual search better than serial models (Eckstein, Thomas, Palmer, & Shimozaki, 2000; Palmer, Verghese, & Pavel, 2000), suggesting that the role of parallel mechanisms, or “preattentive processes,” in inefficient CVS may have been underestimated (Wolfe, 2003). However, direct experimental evidence supporting this view is still lacking. Indeed, behavioral (Bricolo, Gianesini, Fanini, Bundesen, & Chelazzi, 2002; Carrasco & Yeshurun, 1998; Dosher, Han, & Lu, 2004; McElree & Carrasco, 1999; Townsend & Fific, 2004), neurophysiological (Bichot, Rossi, & Desimone, 2005; Woodman & Luck, 2003), and functional neuroimaging studies (Corbetta, Shulman, Miezin, & Petersen, 1995; Donner et al., 2002; Leonards, Palix, Michel, & Ibanez, 2003) which have attempted to determine the respective contribution of serial and parallel processes to CVS led to inconsistent conclusions. In addition, previous behavioral studies were all based on indirect methods, relying on comparisons between the observed data and predictions issued from either parallel or serial models. A more direct approach, consisting in measuring the occurrence of attentional shifts during the performance of CVS tasks could therefore allow us to clarify this issue. 

In the present study, we have used three variants of a dual task paradigm in order to determine the spatio-temporal characteristics of attentional allocation. This dual task consisted of (1) detecting the presence of a target in an array of visual stimuli and (2) reporting characters briefly flashed inside these visual items (see Figures 1, 6, and 7). Importantly, the report of the characters flashed inside a CVS item has already proved to be an efficient way to track spatial location of the attentional spotlight (Bichot, Cave, & Pashler, 1999; Egly & Homa, 1984; Gegenfurtner & Sperling, 1993; Kim & Cave, 1995; Kleiss & Lane, 1986; Shaw & Shaw, 1977; Shih & Sperling, 2002; Talgar, Pelli, & Carrasco, 2004; Zenon, Ben Hamed, Duhamel, & Olivier, 2008). In the present task, several characters were used, but any character was presented only once in a given trial, both at a known timing and location. Therefore, the characters reported by the subjects allowed us to determine a posteriori the deployment of attention during CVS in both space and time (Shih & Sperling, 2002) and to unveil attentional shifts. 

Figure 1

 

Procedure of Experiment 1. After the fixation point was displayed for 1500 ms, CVS items appeared. The target was the circle with an attached vertical line. Only one relevant character was displayed at a time in CVS items; it was changed and possibly displaced in each frame, i.e., every 45 ms. All other items contained irrelevant symbols. Subjects had to detect whether a target was present or not and to report as many characters as possible.

In Experiments 1 and 2, the main task was, respectively, a difficult feature (see Figure 1) and a conjunction CVS (see Figure 6). Experiment 3 was a cueing task (see Figure 7) designed to determine whether the character reporting subtask was an appropriate method to detect attentional shifts when they are known to occur. 

Experiment 1: Covert feature search

In Experiment 1, we aimed to determine whether, in inefficient CVS, the set-size effect is due to an increased number of attentional shifts, as predicted by serial models, or whether parallel mechanisms instead account for this effect. To distinguish between these two hypotheses, we used an inefficient feature visual search task, which, according to serial models, should require multiple attentional shifts. Concurrently to this task, participants had to perform a character reporting task, used to determine attentional allocation during CVS. More specifically, we tried to determine whether, when reporting multiple characters, subjects were more likely to detect characters flashed successively at the same location or at different positions. Reporting several characters flashed at the same location would indicate that the attentional spotlight remained stationary during the time interval between the presentations of the detected characters. In contrast, we predicted that, as the delay between the character presentations increases, the occurrence of attentional shifts would induce a progressive decrease in the probability of reporting characters flashed at the same location. Therefore, comparing the probability of reporting several characters presented at the same place versus at different locations as a function of the delay between the character presentations should allow us to determine whether attentional shifts occur in such a task. 

Methods

Participants

Ten healthy volunteers (21–30 years old), with normal or corrected-to-normal vision, gave written informed consent to participate in this study. One of them was left-handed. The present protocol was approved by the Ethics committee of the Université Catholique de Louvain. 

Behavioral procedure

The task consisted, first, in detecting the presence, or the absence, of a target among six visual items and, second, in reporting characters that were flashed inside these CVS items (see Figure 1). Subjects were instructed to focus mainly on the CVS task and to consider the character reporting task as secondary; they were not told how many characters were displayed but were asked to report as many as possible. 

Each trial started with the presentation of a fixation point on the center of the computer screen placed at a distance of 55 cm from the subjects. The subjects had to fixate this point throughout the experiment. Eye position was controlled by means of an electro-oculogram and trials were interrupted and repeated, whenever subjects broke fixation. After 1500 ms, the CVS items, namely six gray circles (1.8 degrees wide, 4 degrees eccentricity) were displayed on the screen. These circles had a small bar attached to them, which was either vertical (target) or oriented at 20, 40, 60, 80, or 100 degrees from the vertical (distractors). One ASCII sign was displayed inside each CVS item but, in only one item, this sign was a relevant alpha-numeric sign that the subjects had to report. These relevant alpha-numeric signs were either a digit, from 1 to 9, or a letter, excluding the last four letters (from “w” to “z”); digits were presented in the six first frames whereas letters were presented in the remaining frames. In the remaining of this paper, letters, and digits will be collectively referred to as characters. In the other CVS items, the signs were irrelevant (“&,” “%,” etc.…). At every frame change (i.e., every 45 ms), the relevant alpha-numeric sign and two irrelevant ones were changed, the new relevant alpha-numeric sign being presented either at the same or at a different location. Eighteen frame changes occurred in each trial; the duration of a trial was therefore 810 ms (18 × 45 ms). The relevant sign was displayed in each CVS item in a pseudo-random order so that, in a given trial, a different relevant sign appeared three times inside each CVS item ( n = 6), with a variable delay between each presentation. At the end of each trial, subjects had, firstly, to press as fast as possible the “one” or the “three” key of the numeral pad to indicate that the target was present or absent, respectively. Secondly, they had to type on the keyboard, without any time constraint, the different characters identified inside the CVS items. Subjects then triggered the next trial by pressing the space bar. 

The experiment was divided into two sessions, performed on different days. The first session started with a training session during which the subject performed the CVS task alone, then the character identification task alone, and finally the dual task; it was followed by a first block of 200 trials. The second session consisted in three blocks of 200 trials each. 

Control Experiment A

Control Experiment A was conducted at the end of Experiment 1. In this experiment, subjects ( n = 6, 25–29 years old) performed either the same dual task (DUAL CVS), a single CVS task (SINGLE CVS) in which subjects were not asked to report characters, a single character reporting task (SINGLE CHARACTER), or a dual task with the emphasis placed on the character reporting task (DUAL CHARACTER). In the SINGLE CVS and DUAL CVS conditions, the number of CVS items was set either to 4 or 6 in order to estimate the set-size slope of the task. 

Data analysis

Most statistical analyses performed were ANOVA for repeated measurements. Post hoc tests were Bonferroni corrected. Percentages of detection were corrected by means of an arcsine-root transformation in order to obtain a distribution close to normal (Zar, 1996). 

Analysis on character report was carried out for nine different delays between character presentations (from 1 to 9 frames, corresponding to 45 to 405 ms). All trials were included, except aborted trials in which participants failed to provide a response to the visual search task. We did not analyze character pairs in which delays between the two reported characters were larger than 405 ms because they were very infrequent (less than 20 trials per subject for all intervals longer than 405 ms, see Figure 3). 

For reaction times analyses, we first took the median of the reaction times for each subject and then computed means and standard deviations across subjects. Only trials in which the response to the visual search task was correct were included in these analyses. 

Results

In Experiment 1, the average RT in the dual task was 1013 ± 375 ms for “target present” trials and 1046 ± 322 ms for “target absent” trials. On average, subjects reported 1.60 ± 0.99 correct characters (mean ± SD, n = 10) per trial. The probability to detect a character was influenced by the similarity between the target and the CVS item containing that character (Target present: F(5,45) = 25.54, p < 0.0001; Target absent: F(4,36) = 2.20, p = 0.08; see Figure 2A). Post hoc analysis showed that characters located inside the target were more likely to be detected ( p < 0.05). The difference in detection probability between the distractor most similar to the target and other distractors only leant towards significance (p >= 0.06). This effect of item-target similarity, in agreement with previous results (Zenon et al., 2008), indicates that the performance in the character reporting task critically depends on visual search features and therefore that this dual task paradigm is appropriate to track attention displacements during CVS (see Experiment 3 for further evidence). In addition, character detection probability was influenced by the position of the characters in the display (RM ANOVA: F(5,45) = 3.49, p < 0.05; see Figure 2B). Characters located on the horizontal meridian were more likely to be reported (post hoc analyses: all p < 0.05). 

Figure 2

 

Feature CVS performance. (A) Percentage of detection of the characters (number of characters reported divided by the number of characters displayed, multiplied by 100) for target present (black circles, solid line) and target absent (white circles, dashed line) trials as a function of similarity between the CVS item in which the character was displayed and the target. (B) Radial plot of the proportion of characters detected on each position. Item positions on the graph (gray circles) correspond to the real positions seen by subjects.

We found that characters flashed later during a given trial were more likely to be reported than characters flashed earlier ( F(17,153) = 98.62, p < 0.001). Several factors could account for this result. First, we found a gap in performance between the first six frames and later frames. Since digits were displayed in the first six frames and letters in the remaining frames, this gap in performance was presumably caused by a lower detection performance for digits than for letters. This issue was addressed in Experiment 2, in which digits were no longer used and in which the earlier part of the visual search task was investigated more specifically. However, the increase in character detection performance was not limited to the six first frames but consisted in a monotonic increase in character detection performance with onset time. One possible explanation for this increase in character detection performance with time is that subjects adopted a deliberate task-ordering strategy that consisted, first, in finding the CVS target and, second, in detecting as many characters as possible. However, this interpretation is doubtful since we found the same result in the SINGLE CHARACTER task in which subjects only had to perform the character reporting task (Control Experiment A; F(17,85) = 9.18, p < 0.001). Moreover, the difference between the presentation times of the earlier reported characters when the character task was performed alone or in association with the CVS task was much too small (average onset time of first reported characters: 533.4 ± 37.5 ms in the dual task and 489.4 ± 16.0 ms in the single character reporting task) to allow subjects to perform the CVS task within this interval. In addition, since RTs in target absent trials were typically longer, a task-ordering strategy would predict later average presentation times of reported characters in target absent trials. In contrast, there was no significant difference between the presentation delays of reported characters in the target present (530.6 ± 39.3 ms) and target absent (536.0 ± 39.5 ms) conditions ( F = 0.36, p = 0.57). Therefore, rather than a task priority effect, the finding that characters flashed later were more likely reported could reveal a masking phenomenon and/or a memory limitation. Indeed, as a character is flashed early on, several other symbols, including other characters, were subsequently displayed at the same location, and these symbols may act as masks on the previously displayed character. Regarding the memory issue, the delay between detection and report was, by definition, longer for characters flashed early, increasing the probability of memory trace decay. 

Lack of attentional shift in feature CVS

To determine whether attentional shifts occur during CVS, we compared the probability of reporting multiple characters flashed either inside the same CVS item or inside two different CVS items. We found that, when a character was detected at a given location, the probability to report another character displayed subsequently was higher for characters flashed inside the same item (24 ± 8%) than inside a different item (8 ± 4%) (two-way RM ANOVA: main effect of same vs. different: F(1,9) = 14.01, p = 0.005). 

In order to determine how this “preference for same” was influenced by the time interval between character presentation, we computed the relative difference (RD) between probabilities of character report in same vs. different items for each interval (RD = [same − different]/[same + different]). Our prediction was that, when subjects report two characters flashed at very short intervals, these characters would be more likely located inside the same item (“preference for same”) because there was no time for attention to shift. However, if attention shifted after a certain delay, this “preference for same” should progressively decrease as the interval between character presentations increased. In contrast, if attention does not shift during CVS, the “preference for same” should remain constant, irrespective of the interval between character presentations. 

We focused our analysis on trials in which subjects reported at least two characters (48 ± 22% of the trials; mean ± SD; n = 10) and analyzed each pair of reported characters as a function of their order of appearance on the display. For example, in trials where three letters were reported, we analyzed the pair formed by the first and second reported characters and the pair formed by the second and third characters but not the pair composed of the first and third reported characters. In most cases, either two (60 ± 21% of cases) or three characters (30 ± 13%) were reported; trials in which more than three letters were reported accounted only for 10% of the cases. This analysis was only performed on pairs of characters separated by 1 to 9 frames (i.e., flashed at a time interval ranging from 45 to 405 ms) because, as already mentioned in Methods, the probability of reporting two characters displayed at longer intervals decreased dramatically (see Figure 3). The average number of trials per subject included in each time interval condition (from 1 to 9 frames) used in this analysis was 24.6 ± 10.7 and 29.2 ± 13.6 for target absent and target present trials, respectively. 

Figure 3

 

Distribution of the time interval between pairs of successively reported characters for target present (blue) and target absent (green) trials.

We failed to find evidence for a significant effect of interval on RD (one-way RM ANOVA: F(8,72) = 1.59, p = 0.14), and more importantly, there was no correlation between RD and the presentation delay ( R = 0.13, p = 0.22, slope = −0.01, confidence interval (CI) = [−0.03, 0.01], Figure 4, red line). This shows that within the 45- to 405-ms window, the “preference for same” remained constant whatever the interval between the character presentations. This finding suggests that attention did not shift during the performance of this CVS task. Interestingly, we found similar results for trials in which the target was absent (two-way RM ANOVA: main effect of same vs. different: F(1,9) = 42.17, p < 0.0001; one-way RM ANOVA: effect of time intervals on RD: F(8,72) = 1.56, p = 0.15; regression: R = 0.16, p = 0.14, slope = −0.02, CI = [−0.06, 0.02]), indicating that the absence of attentional shift when the target was present cannot be explained by the fact that attention remained stationary on the target once detected. Moreover, even when the target was present, the probability of detecting two characters flashed inside the same item was identical whatever the nature of the item, target, or distractor (one-way RM ANOVA: F(1,9) = 0.15, p = 0.71), indicating that the “preference for same” was not only applicable to the characters detected inside the target. Finally, we found that the probability of reporting successive characters at the same location did not differ between the six positions of CVS items (one-way RM ANOVA: F(5,45) = 1.50, p = 0.21), demonstrating that this “preference for same” did not result from the spatial heterogeneity of the character report probability. 

Figure 4

 

Relative difference (RD, [same − different]/[same + different]) in dual feature ( Experiment 1, shown in red) and conjunction CVS (Experiment 2, in green), as a function of the time interval between reported characters. The dashed lines correspond to regression lines between RD and time interval.

Control Experiment A

Control Experiment A (see Methods) was conducted in order to confirm that the CVS task was inefficient and to investigate the possible interaction between the character detection and CVS tasks when performed together. 

Inefficiency of the CVS task

We compared the reaction times between the four items and six items CVS tasks, when performed either alone or in association with the character reporting task in three-way RM ANOVA (Target (present vs. absent) × Set Size (four or six) × Task (single or dual)). We found a main effect of set size ( F(1,5) = 36.38, p = .002) on RT but no significant interaction (Set Size × Task: F(1,5) = 0.26, p = .630; or Target × Set Size × Task: F(1,5) = .04, p = 0.846). The set-size slope in the SINGLE CVS was 31 ms/item for target present trials (RT mean ± SD: four items: 955 ± 242 ms; six items: 1016 ± 188 ms) and 39 ms/item for target absent trials (RT: four items: 1119 ± 164 ms; six items: 1197 ± 313 ms); in the DUAL CVS, the set-size slope was 20 ms/item (RT mean ± SD: four items: 1156 ± 285 ms; six items: 1196 ± 197 ms) and 47 ms/item (RT mean ± SD: four items: 1322 ± 227 ms; six items: 1416 ± 357 ms) for present and absent trials, respectively. These values are comparable to those reported in the literature for inefficient CVS (Wolfe, 1998), confirming that the CVS task was inefficient. 

Interactions between CVS and character reporting

A critical concern about dual tasks is the possible interaction between the two individual tasks namely, in the present study, the CVS, and the character reporting tasks. Indeed, it could be hypothesized that CVS was performed differently in the single and dual task conditions, hence hampering our conclusions about attentional displacements during CVS. However, several pieces of evidence do not support this assumption. Firstly, as already mentioned, we found, that the set-size slopes were not significantly different when the CVS task was performed alone or in association with the character task, suggesting a lack of interactions between the tasks. Secondly, we computed the attention operating characteristic (AOC), to estimate the additional cost of performing a dual task (Sperling & Melchner, 1978). To do so, we compared the performance of both the character reporting task and the CVS task in the different conditions tested in the Control Experiment A, i.e., SINGLE CVS, DUAL CVS, SINGLE CHARACTER, and DUAL CHARACTER (see Methods). The performance in the character reporting tasks (SINGLE CHARACTER, DUAL CHARACTER, and DUAL CVS) was quantified by measuring the character report rate and the performance in the CVS tasks (SINGLE CVS, DUAL CVS, and DUAL CHARACTER) was estimated by computing d′. We found that the character reporting task did not affect d′ (one-way ANOVA with CVS performance as a function of task conditions, SINGLE CVS, DUAL CVS, and DUAL CHARACTER: F(2,10) = 0.23, p = 0.80; Figure 5). This finding validates the use of the character reporting task in order to monitor attention allocation during CVS. In contrast, we found that the character report performance was significantly affected by CVS (one-way ANOVA with character task performance in function of task conditions, SINGLE CHARACTER, DUAL CHARACTER, and DUAL CVS: F(2,10) = 6.35, p = 0.01; Figure 2). 

Figure 5

 

Attention Operating Characteristic. Large blue and large red dots correspond to performance in the single character detection (Y axis, same convention as in Figure 2) and single CVS tasks (X axis), respectively. Small blue and small red dots indicate the performance in the dual tasks with emphasis put on the character detection and on the CVS task, respectively. The horizontal and the vertical lines are best-fits to the single task data; the diagonal line is best-fit to the dual task data (joining the means of performance for both dual task conditions); the thicker portion is the estimated attention operating characteristic. The AOC line is near the “independence point,” located at the intersection of the vertical and horizontal lines, which is the point at which subjects would operate if they performed both tasks simultaneously without any interference, i.e., independently of each other. Gray dashed lines link the dots corresponding to single subjects in the different task conditions.

Discussion

The results of Experiment 1 demonstrate that an inefficient visual search task can be performed while attention remains stationary at a particular location. Indeed, we found that characters were more likely to be reported when flashed at the location of a previously reported character, suggesting that spatial attention was dwelling at that location. The observation that this preference for characters successively flashed at the same location did not vary with the delay between their presentations indicates that attention did not shift during the task performance. This preference for detecting successive characters at constant location is not explained by asymmetries in the performance field: while characters are more likely to be reported when displayed along the horizontal meridian, the preference for same was identical for all locations. Another potential confounding factor that must be excluded is the use of a dual task paradigm. These types of paradigms can be regarded as valid provided one can demonstrate that the addition of the concurrent task—the character reporting task in the present study—does not alter the processes underlying the main task, namely the CVS task. In particular, it could be argued that, in the dual task, subjects focus their attention strategically on a single location, in order to optimize their performance in the character reporting task, and that, consequently, measuring attentional allocation during the dual task cannot be used to infer attentional allocation during the visual search task. We demonstrated that this issue does not confound our results because the performance and set-size slope of the visual search task was not affected by the addition of the character reporting task. If visual search tasks required serial attentional shifts and that the addition of a character reporting task prevented these attentional shifts to occur, visual search performance should drop in the dual task condition, in contrast to the present results. 

Another important issue that has to be addressed when using a dual task is the possibility that the subjects perform the two tasks sequentially instead of concomitantly. Indeed, in the present study, it could be hypothesized that the subjects first performed the visual search task and then used the remaining processing time to carry out the character reporting task. However, our results do not support this view since the character report probability depended on the item-target similarity, demonstrating that the character reporting task is influenced by the visual search task. In addition, and more importantly, the difference between the mean presentation times of the two reported characters in the single character task and the dual task is much too small to allow the visual search to be performed within that interval. Additional evidence against the task-ordering hypothesis is provided in Experiment 2. 

Experiment 2: Covert conjunction search

In the pioneering work of Treisman and Gelade (1980), parallel and serial mechanisms of visual information processing were associated with feature and conjunction CVS, respectively. In feature CVS, the target is defined by a single feature difference (e.g., a red target among green distractors), whereas in conjunction conditions, two or more features are combined to define the target (e.g., a red circle among red squares and green circles). To explain why conjunction CVS, in contrast to feature CVS, yields steep set-size slopes, Treisman and collaborators proposed the existence of multiple feature maps (e.g., orientation, color, or luminance maps) in the visual system. In conjunction CVS, the multiple features of the target would have to be bound together to allow target discrimination and attention would be essential to perform this binding (Treisman, 1982; Wolfe & Cave, 1999). As a consequence, in conjunction CVS, attention would have to select each CVS item serially to allow the target to be discriminated from the distractors. However, since then, it has been shown that the set-size slopes of both conjunction and feature CVS, instead of being clearly distinct, are distributed along a continuum, ranging from 0 to about 50 ms/item (Wolfe, 1998). This has prompted Treisman (1999) and others (Deco, Pollatos, & Zihl, 2002; Duncan, 1995; Hamker, 2004; Itti & Koch, 2000; Wolfe, Cave, & Franzel, 1989) to propose alternative models of visual search. But importantly, the fact that similar set-size slopes can be obtained in conjunction and feature search tasks does not imply that the inefficiency originates from the same mechanism in both types of CVS. Therefore, it is important to determine whether, in contrast to inefficient feature CVS, attentional shifts occur during conjunction CVS. 

In order to address this question, in Experiment 2, we conducted the same analysis as in Experiment 1 on a conjunction visual search task. In addition, we tried to address a potential issue of Experiment 1 concerning the range of timings included in the analysis. Indeed, in Experiment 1, our analysis included more characters detected lately than early, due to the fact that character report probability increased with the presentation delay. Therefore, it could be argued that attentional shifts occurring during the early period of the search would have been missed. Part of this problem was caused by the use of digits, instead of letters, in the first six frames of the task. To overcome this issue in Experiment 2, we abandoned the use of digits and added a condition in which both the visual search items and the letters were displayed for only 405 ms. 

Methods

In this task, three blue and three yellow isoluminant circles with an attached bar were displayed on the screen, at a fixed eccentricity (4°). The orientation of the bar could be 0°, 20°, or 40° from the vertical. The target was defined as the blue circle with a vertically oriented bar; other items were used as distractors. Additionally, while in half of the trials 18 frames (45 ms) were displayed (see Figure 6), in the other half, only 9 frames were shown. These different trial durations were intermixed. In the 18 frames condition, the character presentation was the same as in Experiment 1 (see above). In the 9 frames trials, two different characters were always displayed simultaneously, at opposite locations and the position of the character pair was modified in every frame. Therefore, after only three frames, characters had been displayed in all six possible positions and after 9 frames, three different characters had been presented in each position. Only letters, and no digits, were used in this experiment in contrast to Experiment 1. The task of the subjects was the same as in Experiment 1, i.e., to detect the presence of the target, as quickly as possible, and then to report as many characters as possible. Again, emphasis was put on the visual search task and not on the character reporting task. Each subject performed 800 trials. 

Figure 6

 

Procedure of Experiment 2. Visual search stimuli were either yellow or blue circles with attached lines at one of three possible orientations. The target was the blue circle with a vertical line. Equal proportions of short (9 frames) and long (18 frames) trials were randomly interleaved. Response requirements were the same as in Experiment 1.

Control Experiments B and C

Two additional control experiments were conducted on the same pool of subjects. In Control Experiment B, either 4 or 6 CVS items were displayed, and subjects had only to report the presence or absence of the target (SINGLE CVS). In Control Experiment C, the display was exactly the same as in Experiment 2, but subjects had to report only the characters (SINGLE CHARACTER). In both control tasks, all trials were 18 frames long. 

Results

In the dual conjunction task of Experiment 2, subjects reported 1.72 ± 0.64 correct characters (mean ± SD, n = 8) per trial. The average RT was 1397 ± 457 ms for “target present” trials and 1429 ± 544 ms for “target absent” trials. The performance in the character reporting task was influenced by the similarity in orientation (two-way RM ANOVA, F(1,7) = 42.81, p < 0.0001) and color (two-way RM ANOVA, F(2,14) = 22.34, p = 0.002) between the target and the CVS item containing that character and also depended on their location in the display (one-way RM ANOVA, F(5,35) = 2.43, p = 0.05). These effects are in agreement with the results of Experiment 1 showing that attention allocation is influenced by the similarity between the CVS item and the target. We also found a higher detection rate of characters flashed at the same location (two-way RM ANOVA, F(1,7) = 37.62, p < 0.0001), a lack of effect of time intervals on RD ( F(8,64) = 1.72, p = 0.11), and an absence of correlation between RD and presentation intervals ( R = 0.07, p = 0.54, slope = 0.00, CI = [−0.02, 0.02], see Figure 4, green line), suggesting also an absence of attentional shift in the conjunction CVS task. 

In Experiment 1 the finding that attention does not shift during CVS could be confounded because our analysis included more characters detected lately than early, due to the fact that character report probability increased with the presentation delay. In Experiment 2, we were able to investigate the early period of the search, since in 50% of trials, the display presentation lasted only 405 ms (i.e., 9 frames). The d′ in this condition was much smaller than that measured in the long display duration condition (32% reduction, RM ANOVA: F(1,7) = 11.97, p = 0.01), showing that the processing of the CVS task was not yet completed within this 405-ms presentation. In this short presentation condition, the RD analysis differed slightly because the number of trials in which more than one character were detected was small; as a consequence, we grouped different interval conditions into only four intervals (45–90 ms, 135–180 ms, 225–270 ms, and 315–360 ms). Again, these results confirmed that RD was not affected by the intervals ( R = 0.09, p = 0.63), showing that even in the early stage of the search, when the target is still being looked for, no attentional shifts occurred. 

Control Experiment B: Inefficiency of the task

As for Experiment 1, we confirmed that the CVS task was inefficient by comparing RT between the 4- and 6-item conditions (set-size slope: 36 ms/item for target present trials and 50 ms/item for target absent trials; two-way RM ANOVA, F(1,7) = 6.40, p = 0.039). 

Control Experiment C: Absence of task-ordering strategy

The absence of task-ordering strategy was confirmed, as in Experiment 1, by comparing the presentation times of the first reported characters between the single character reporting task in Control Experiment C and the dual task of the main experiment. Here again, the difference between the average presentation times was too small (average onset time of the first reported character: 295.2 ± 34.7 ms in the dual task and 249.4 ± 42.5 ms in the single character reporting task) to allow subjects to perform the CVS task before the character reporting task; no difference was found between target absent (286.8 ± 29.7 ms) and target present trials (299.9 ± 45.6 ms; F = 1.18, p = 0.31). 

Discussion

In Experiment 2, we showed that in a conjunction CVS, the preference for detecting characters displayed successively at the same location was constant across time. This demonstrates that this conjunction CVS task was performed in absence of attentional shifts, similarly to the feature CVS task of Experiment 1. Moreover, we found the same results when exploring specifically the earlier part of the CVS, when the target is still actively searched for. 

It is noteworthy that the RD observed in the conjunction CVS in Experiment 2 was smaller (RD = 0.2 ± 0.11, mean ± SD) than in the feature CVS of Experiment 1 (RD = 0.42 ± 0.26). One possible explanation is that, in Experiment 2, instead of selecting a single location, spatial attention was distributed on several items because of grouping phenomena, increasing the probability to detect characters at different locations. Indeed, some authors have proposed that conjunction visual searches are performed exclusively on the set of items exhibiting one of the target feature, such as target color, other distractors being excluded from the search processes (Zohary & Hochstein, 1989). In addition, using a dual task similar to the one used in the present study, Kim and Cave (1999) have shown that during conjunction visual search, attention selected the set of items that shared the target color. In an earlier study, we found similar results, showing in addition that grouping phenomena were involved in the later part of the search process (Zenon et al., 2008). In agreement with this interpretation, we found in the present study that when two characters were detected in different locations, they were more frequently detected in items sharing the target color (interaction between color of the first and second characters in the pair, two-way ANOVA, F(1,7) = 8.88, p = 0.02, post hoc test: p < 0.05). 

Experiment 3: Covert cued attention task

To strengthen the conclusion that attention does not spontaneously shift during CVS, it was crucial to prove that our procedure was indeed able to detect attentional displacements in a condition where they are known to occur. To address this issue, we conducted a third experiment in which attention allocation was controlled by visual cues. 

Methods

Each trial started with the presentation of a central fixation point. After 1500 ms, six gray circles (1.8 degree wide, 4 degrees eccentricity, 50% contrast) were displayed on the screen, and ASCII signs were displayed inside each CVS item and replaced on successive frames as described above. The visual search items were simple circles with no bar attached to them. The target was a slightly red circle (10% increase in red luminance). To cue attention allocation, during each trial, the contrast of two gray circles successively increased to 100% for 45 ms, at random intervals; the first cue appeared either in the 3rd, 4th, or 5th frame (i.e., 135, 180, or 225 ms after the trial onset) and the second one in the 12th, 13th, or 14th frame (i.e., 540, 585, or 630 ms after the trial onset). Three frames, i.e., 135 ms, after either the first or the second cue (chosen randomly in each trial), the item previously cued became the target in 40% of the trials, i.e., its red luminance increased by 10%; in 10% of the trials, the target was presented in a “non-cued” location, and in the remaining 50%, no target was presented (see Figure 7). All trials (target present and target absent), except invalidly cued trials, were included in analyses on character detection rate. As before, the subjects had first to indicate whether a target was presented or not and second, they had to report as many characters as possible. Characters display was the same as in Experiment 2 except that 18 frames were displayed on all trials. Every 45 ms the character was changed, together with two irrelevant signs. The new character could appear either at the same or at a different location. Consequently, three different characters appeared during a given trial at each of the six item locations. Each subject performed 400 trials in this Experiment. 

Figure 7

 

Experimental procedure of Experiment 3. Display items were plain circles. The cues consisted in two transient contrast changes in different display items and the target was the circle that became transiently reddish. It followed either the first (3rd, 4th, or 5th frame) or the second cue (12th, 13th, or 14th frame) by three frames. Subjects had to detect whether a target was displayed and to report as many characters as possible.

Results

The results of this experiment confirm that subjects did actually make attentional shifts to the cued CVS items. Indeed, the character report rate increased significantly (one-way RM ANOVA: F(7,28) = 11.74, p < 0.0001) in the cued CVS items. Figure 8 shows the time course of the character report performance in the cued location, with respect to the cue onset (the maximal performance is reached around 4 frames, i.e., ≈180 ms, after cue onset). RTs measured from target onset were 1272 ± 461 ms (mean ± SD) and 1331 ± 409 ms for valid and invalid trials, respectively. Subjects reported an average of 1.57 ± 0.39 correct characters per trial. As in the feature ( Experiment 1) and conjunction ( Experiment 2) dual tasks, we found a “preference for same,” but in this case, the slope of the regression line computed between the RD and the time interval was negative ( F(8,56) = 8.83, p = 0.004, see Figure 9, blue line, and experimental procedure). This indicates that when an attentional shift occurred, the probability to detect two characters flashed in the same position decreased significantly with the time interval, in contrast to results from both feature and conjunction CVS dual tasks. 

Figure 8

 

Effect of cueing on attention allocation. Time course, with respect to the cue onset, of the character detection performance in the cued location ( n = 8). Error bars correspond to standard error values. The character detection performance in the cued location increased after cue onset and reached a maximum around 4 frames (∼180 ms) after cue onset.

Figure 9

 

Relative difference between same and different location in the cued attention experiment. Same conventions as in Figure 4. Data from the cued attention experiment are shown in blue along with data from the feature CVS experiment ( Experiment 1) for comparison purposes.

Discussion

To be able to interpret the main finding of Experiments 1 and 2 that the high probability of reporting two characters flashed at the same location remains identical whatever the interval between their presentations, it was critical to ascertain that this “negative” result was not caused by a lack of sensitivity or by the inappropriateness of the method we used to monitor attention displacements. The results of Experiment 3 clearly demonstrate that this is not a valid criticism since when attention was successively cued on different positions, the probability of reporting characters at constant location decreased as the time interval between the detected characters increased, in sharp contrast with what we observed in Experiments 1 and 2. 

It must be pointed out that in Experiments 1 and 2, both the character reporting task and the visual search tasks were based on the processing of shape whereas in Experiment 3, target detection only relied on color. Consequently, it might be assumed that the competition arising from the use of common resources for character and target discrimination in Experiments 1 and 2 was responsible for the lack of attentional shift observed in these experiments. However, we regard this hypothesis as very unlikely since we showed above that the performance in the character reporting task, and not in the visual search task, was diminished in the dual task condition; this suggests that the visual search task maintains a priority on the access to shape processing resources. Therefore, this putative competition between both tasks should not affect attentional allocation in visual search. 

General discussion

The present study provides support for the parallel models of CVS by demonstrating experimentally a lack of attentional shifts during inefficient CVS tasks. This finding is in agreement with the prediction made by the parallel visual search models that an increase in RT consequent to the addition of more distractors results from a more time consuming processing necessary to differentiate the target and distractor signals (Verghese, 2001), and not from additional attentional shifts. 

Questioning the validity of the serial models of visual search is not novel and has already been done by many authors. For example, Carrasco and Yeshurun (1998) used informative cues to control attentional allocation during a CVS task. They predicted that, according to serial models, a cue should cancel the set-size effect by suppressing the need to scan the entire display to search for the target. In contrast, they showed that, although decreased, the set-size effect was not completely suppressed by cueing the target location, a finding regarded as incompatible with the serial models. However, the cues used in the study of Carrasco and Yeshurun were valid in only 66% of the trials, and it cannot be excluded that the incomplete disappearance of the set-size effect was due to the partial validity of the cue rather than to the parallel nature of the search process. Along the same line, Eckstein et al. (2000), Palmer, Ames, and Lindsey (1993), and Palmer et al. (2000) used a model of parallel search, based on signal detection theory to predict accurately the performance (detection threshold) of human observers on feature (Palmer et al., 1993, 2000) or conjunction visual searches (Eckstein et al., 2000). These studies demonstrated convincingly that parallel models are able to explain the set-size effect observed in difficult visual search tasks. However, it remains unclear whether parallel models really outperform serial models in predicting performance of real observers. For example, when comparing the fit of the parallel model and that of a serial model (Eckstein et al., 2000), Eckstein assumed a “blind” serial model in which attentional selection was performed at random. Instead it is now demonstrated that attentional selection is biased towards items that share some features with the target (Duncan & Humphreys, 1989; Wolfe et al., 1989; Zenon et al., 2008), and consequently, the performance of parallel models should be compared to the accuracy of serial models assuming the existence of a guidance of visual attention. McElree and Carrasco (1999) took this point into account and compared the ability of parallel and serial search models to predict speed-accuracy trade-off in a conjunction visual search. Because, in conjunction searches, all distractors share a feature with the target, it was assumed that attention guidance was ineffective. However, in such conjunction tasks, it is known that one of the feature of the target is selected and that only distractors sharing this feature are actually processed (Zohary & Hochstein, 1989). Hence, the number of distractors that had to be scanned by attention under the serial model assumption in the McElree and Carrasco study was overestimated. In summary, all these studies were decisive in demonstrating that serial processes are not mandatory but they did not validate parallel models. The present study provides, for the first time, direct evidence for the lack of attentional shift during inefficient visual searches. 

Our results can be seen at variance with a classical phenomenon reported in the literature, namely the attentional blink effect (AB). Indeed, in paradigms in which two targets have to be detected in a continuous stimulus stream (RSVP tasks), a common observation is that the probability of detecting a second target is lower when presented up to 500 ms after the first one (Kristjánsson & Nakayama, 2002; Shapiro, Raymond, & Arnell, 1994). In contrast, we found a steady preference for characters appearing at the location of a previously detected character, called “preference for same” in the present study. It might be argued that AB was actually involved in our paradigm, inducing a decrease in the “preference for same” for short intervals but that this effect was completely counterbalanced by the effects of attentional shifts, inducing a decrease in the “preference for same” for long intervals. However, this explanation can be ruled out since, in Experiment 3, in which the character detection task was the same as in Experiments 1 and 2 and hence in which the AB effect should be comparable, we found a decrease of the “preference for same” with the time interval between detected characters, an effect exactly opposite to the one predicted by AB. The absence of AB effect in our task can be explained by several important differences with respect to typical AB tasks. In the present study, subjects did not have to detect the occurrence of a particular “target” character but only had to report characters they happened to detect during the performance of the CVS task. Moreover, six streams of characters were displayed instead of one in most RSVP experiments, and each character was masked by either irrelevant signs or by other characters. Finally, in the present tasks, 18 characters per trial were displayed (or 9 in some trials of Experiment 2), and participants were required to report as many of them as possible whereas in typical AB paradigms, only two targets are displayed and subjects have to report both of them. The mechanism underlying AB is still unknown, and the necessary and sufficient experimental conditions to observe AB in RSVP tasks are still uncertain (Raymond, 2003; Visser, Bischof, & Di Lollo, 2004), but obviously these conditions were not fulfilled in the present study. 

Our finding that attention does not shift serially during CVS seems inconsistent with the observation, in the present and earlier studies, that attention is allocated mostly on the target of the search (Kim & Cave, 1995; Zenon et al., 2008). One might question the function of target selection in visual search, given the absence of serial scanning. Two hypotheses can be proposed about the role of spatial attention in parallel visual search. First, it could be hypothesized that attention shifts to the target only after its detection and would hence not be involved in the search process per se. In that case, attention allocation on the target could be just a byproduct of detection without a specific function, or, alternatively, it might serve as a controller, used to verify the validity of the decision made by the parallel detection process. Second, attention, statically allocated, could be directly involved in the search process by improving the signal to noise ratio of the selected visual items. This function of attention is not incompatible with parallel models of search based on Signal Detection Theory (Verghese, 2001). Attention would facilitate search by selecting a potentially relevant location, guided by stimulus-driven and goal-directed factors (Nothdurft, 2006; Wolfe et al., 1989), hence improving the visual discrimination at that location (Theeuwes, Kramer, & Atchley, 1999). Further experiments, either manipulating attentional allocation by cues or using dual tasks similar to the ones used in the present study would be needed to determine which of these hypotheses is correct. 

Intuitively, one might assume that the perceptual and computational benefits of attentional shifts during visual search compensate for the time and resource costs, but our data do not support this view and rather show that observers rely only on parallel processes for target detection. It is noteworthy, however, that the absence of any attentional shift observed in the particular context of the present study does not necessarily mean that attentional shifts never occur in other conditions. In natural conditions, when CVS is performed without any time constraint, saccades are used to explore the visual scene, and attention shifts are known to precede these rapid eye displacements (Deubel & Schneider, 1996). It can be assumed that, during CVS too, in the absence of temporal constraint, attentional shifts could also occur. The crucial point of the present study is that inefficiency in CVS is not explained by serial attentional shifts and that the value of the set-size slope is independent of the number of shifts required to detect the target. This finding is of critical importance because serial models are still deeply anchored in the literature and widely used as a framework to interpret results from many studies on attention mechanisms which contrasted inefficient and efficient CVS in order to manipulate the attentional demand (Ashbridge, Walsh, & Cowey, 1997; Behrmann, Ebert, & Black, 2004; Donner et al., 2002; Müller et al., 2003; Nobre, Coull, Walsh, & Frith, 2003). The present study shows that this approach is most likely invalid and that the view that attention is, like a spotlight, serially displaced in search of the target is flawed. 

Acknowledgments

The authors are grateful to Marco Davare, Michael Andres and Rich Krauzlis for their comments on an earlier version of this manuscript. 

Commercial relationships: none. 

Corresponding author: Alexandre Zénon. 

Email: [e-mail]. 

Address: Systems Neurobiology Laboratory, The Salk Institute for Biological Studies, 10010 North Torrey Pines Road, La Jolla, CA 92037-1099, USA.