Capacity limits during perceptual encoding

Abstract

When a unique stimulus is embedded in an otherwise homogenous display, it is thought to “pop-out” due to its relative increase in salience. We investigated whether the visual system has the ability to equally prioritize multiple salient pop-out items competing for awareness or whether this early stage of visual processing is constrained by capacity limits. We used signal detection ( d′) methods to determine if sensitivity to a salient pop-out item decreases as function of the number of total salient items present in the visual display. Participants engaged in a signal detection task where they had to report the presence or absence of a simple display change that involved either a pop-out or a non-salient distractor stimulus. Results across four experiments consistently showed that sensitivity to changes involving pop-out stimuli was significantly reduced after more than one of these items was present in the visual array. Results are discussed in terms of neural models of visual encoding and other known capacity limits during visual processing.

Capacity limits during early perceptual encoding

How does the visual system effectively prioritize a subset of available information for further processing? Most cognitive models propose a dual stage process of visual encoding, where the first stage has an unlimited or very large capacity and is followed by a second stage that is capacity-limited (for review see Marois & Ivanoff, 2005). Stimulus prioritization during this initial stage of visual processing is based on low-level perceptual aspects of the presented information, which, in most conditions, are thought to be automatically encoded in a stimulus-driven manner at the expense of other available information. These low-level stimulus attributes can include abrupt transients in the display such as changes in luminance (e.g., Irwin, Colcombe, Kramer, & Hahn, 2000; Theeuwes, 1995), the onset of motion (e.g., Abrams & Christ, 2003), or the onset of new stimuli (e.g., Jonides, 1988; Yantis, 1993; Yantis & Jonides, 1996). This stimulus-driven salience can also be mediated through a stimulus' relative uniqueness on some simple feature (e.g., color, size, shape, orientation) relative to the other stimuli present in the visual array. For example, a single thick black line presented among many thin black lines, or a single red square embedded in an array of many green squares will typically be detected without effort due to their relative uniqueness or salience against the uniform background stimuli. In other words, the more salient a singleton stimulus is relative to adjacent distractor stimuli, the more prioritization the singleton receives during this initial stage of processing. This phenomenon is often referred to as pop-out since, to an observer, the salient singleton subjectively seems to “pop-out” of the visual display relative the background distractors (Egeth, 1966; Folk & Egeth, 1989; Hoffman, 1979; Julesz & Bergen, 1983; Treisman, 1985). It is because of this seemingly effortless processing that the initial stage of stimulus-driven encoding is thought to have a very large capacity provided that relative salience differences between distractor and a singleton are maintained. 

The apparent reflexive nature of early perceptual processing stands in stark contrast to later stages of visual encoding where clear capacity limits exist. A common example of this is the observation of capacity limits in visual short-term memory (VSTM), where it is often demonstrated that people can accurately hold only about four encoded items in memory at any given moment in time (Cowan, 2001; Luck & Vogel, 1997). Another example of this capacity limit comes from observations during serial visual encoding when the target stimulus shares a simple feature with distractor stimuli (a red X target among an array of red Os and black Xs). Here, target detection is no longer based on a simple feature alone, but a conjunction of features, which means that the system must now hold two feature templates in memory to accurately locate the target item. As a result, target detection performance for conjunction items drops significantly relative to detection performance for pop-out items, indicating that a conjunction search comes at a cost. In other words, it seems clear that attentionally demanding later stage processing is subject to severe capacity limitations. 

While much has been discovered about the capacity limits of later stages of visual processing, there is little information regarding even the existence of a capacity limit during stimulus-driven encoding. As such, it remains unknown whether a similar bottleneck also exists earlier in the visual processing stream, thus limiting the number of stimuli that can effortlessly pop-out of a visual display with no performance cost. One study that did examine this question, although this was not their primary interest, came from Folk and Egeth (1989) as they presented subjects with vertical and horizontal lines as pop-out targets embedded among diagonal lines as distractors and asked subjects to report whether all targets (i.e., vertical or horizontal lines) had the same orientation or whether one differed (Folk & Egeth, 1989; Experiment 1). They found that increasing the number of preattentively processed pop-out targets from 2 items to 4 items increased reaction times (RTs), suggesting that capacity limitations are already at work at this early processing stage. Interestingly, in a similar experiment, no difference in RT was associated with increasing the number of targets when diagonal line distractors were replaced by circles, leading the authors to suggest that this was due to an increase in relative salience between targets and distractors (Folk & Egeth, 1989; Experiment 4). Therefore, it appears that when the relative salience difference between targets and distractors is sufficiently high, multiple targets can indeed pop-out as indicated by the absences of RT changes for displays with more than one pop-out target. 

It is possible, however, that measures of reaction time are not sufficiently sensitive to detect changes for multiple pop-out targets and that a measure based on perceptual sensitivity for detecting the presence of pop-out items might reveal a decline as the number of salient items present in the display increases. Support for this claim comes from the biased competition model of selective processing (Desimone & Duncan, 1995), which suggests that bottom-up stimulus salience can resolve competition between stimuli competing for awareness. Therefore, when more than one salient stimulus competes for awareness, it is plausible that both will have mutually suppressive effects on each other's neural representation, thereby reducing the visual resolution of both stimuli. 

The neural representation of a single pop-out item during initial visual encoding in the context of competitive bias has been explored using neurophysiological measures. Recent fMRI work by Beck and Kastner (2005) demonstrated that neural suppression in extrastriate cortex for multiple items presented in close proximity was eliminated when one item is significantly different from the others. In other words, pop-out items are prioritized in early visual cortex in a stimulus-driven manner. Other evidence of this sensitivity for pop-out items in early visual cortex comes from single cell recordings in primates. It is well established that cells in V1 have small receptive fields with larger surrounding inhibitory regions. This functional set-up has been demonstrated to be sensitive to gross feature changes relative to a homogenous background. For example, Nothdurft, Gallant, and Van Essen (2000) showed that a pop-out stimulus (e.g., a horizontal line) presented in the center of a receptive field of a V1 neuron surrounded by lines of an orthogonal orientation evoked larger responses then when a heterogeneous display was presented. What these studies show is that the visual cortex is very sensitive to salient difference in an otherwise homogenous display. In other words, a single pop-out item can effectively be selected for prioritized processing as early as V1. 

The question remains, however, whether more than one salient stimulus can be prioritized by the visual system while maintaining equal degrees of perceptual representation. While multiple items of equal salience embedded within a homogenous display should all fall within different receptive fields of individual neurons, they may nonetheless modulate the relative neural representation of each item (Beck & Kastner, 2005). In other words, do multiple salient items receive equal degrees of prioritization with no decline in perceptual sensitivity to these individual items? The present study used signal detection methods (Green & Swets, 1966; Macmillan & Creelman, 1991) to address the issue of capacity limits during stimulus encoding. Through this technique, perceptual sensitivity to the presence of a salient stimulus in the visual array can be assessed. Typically, the signal detection model measures the relationship between hit and false alarm rates to compute an independent measure of discrimination ability between a set of display conditions. Sensitivity (d′) is computed as a measure of how well a target event is detected among some degree of background noise. A larger d′ indicates an increased ability for an observer to detect signal from background noise in that given condition. We employed a detection task (Pashler, 1988) to compute relative sensitivity to display changes involving either a pop-out or distractor item as a function of the total number of pop-out items, the Total Pop-Out Index (TPI), present in the visual display. It was hypothesized that if more than one salient item can be effortlessly detected, sensitivity toward display changes involving these items will remain unaltered as the TPI is increased. Conversely, if there were some perceptual cost associated with multiple pop-out items present in the display, a marked decline in sensitivity should be observed at the TPI point where some capacity is reached. 

Experiment 1

We used a signal detection task to measure relative sensitivity to the offsets of both pop-out and distractor stimuli. Participants were shown an initial array of sixty squares in spatial locations chosen at random for each trial. All squares were green with the exception of 0–4 pop-out items, which were red. The locations of the pop-out stimuli were randomly selected for each trial. After the presentation of the initial array, it was removed for several milliseconds and then displayed again with a slight change on half the trials; namely, the removal of either a pop-out or distractor stimulus. The participant's task was to report whether or not they observed a change in the display. In addition, participants simply had to detect a change in any stimulus present on the screen regardless of its features, and each presented item had an equal chance of being involved in a change. Although the difference between green distractor squares and red pop-out squares was very salient, measures were taken to ensure that both forms of stimuli were equiluminant (see Methods section). 

Sensitivity for the detection of pop-out and distractor offsets was calculated using d′. Importantly, during brief stimulus presentations, measures of d′ typically reflect earlier visual processes and are relatively insensitive to later response-related selection (Handy, Kingstone, & Mangun, 1996; McDonald, Teder-Sälejärvi, & Hillyard, 2000). Also of importance is the fact that estimates of d′ are independent from response bias when a heterogeneity of variance between both samples is assumed. As such, sensitivity scores are thought to be relatively unaffected by the response strategy an observer may adopt (Rotello, Masson, & Verde, 2008). Thus, we predicted that if multiple salient stimuli competing for awareness were equally processed by the visual system, sensitivity to pop-out offset should be invariant as TPI increases. If, however, a decrement was observed at some level of TPI, it would be indicative of a capacity limit for a finite amount of equally salient items the visual system can prioritize at one given moment. 

Methods

Subjects

Thirteen undergraduates ( M age = 19 years; SD = 2.71) from the University of Toronto participated for course credit. All participants were naive to the purpose of the study and had normal or corrected-to-normal vision. All participants gave informed consent and all ethical guidelines set by the University of Toronto were adhered to in full. 

Apparatus and procedure

Stimuli were presented on a 19-in ViewSonic Graphic Series G90fb monitor (1024 × 768 resolution; 85-Hz refresh rate). Viewing distance was held constant at 44 cm by using a chinrest. The entire display subtended 40° × 30° of visual angle. The main display consisted of a total of 60 squares, each subtending 0.4° × 0.4° of visual angle. The Total Pop-Out Index (TPI) was manipulated by presenting 0, 1, 2, 3, or 4 pop-out items, which were red squares among green distractor squares. The colors of both targets and distractors were adjusted using a photometer so that they were equiluminant (14.0 cd/m ²) and displayed on a black background (0.2 cd/m ²). A typical trial sequence is shown in Figure 1. A centrally presented fixation cross was displayed for 700 ms, followed by the onset of the array of squares for 100 ms. A blank screen was then presented for 50 ms, which was then followed by the same array of squares previously presented in the trial with a slight modulation depending on the condition. On half the trials, one of the squares was removed from the display. There were two different offset conditions: the removed stimulus could with equal probability be a pop-out item or a distractor stimulus. This second array of squares remained on screen for an additional 250 ms at which point it was masked for 500 ms. Participants then were asked to report whether or not they perceived an offset by pressing the “z” key for yes or by pressing the “/” key for no. When an offset was correctly detected, a hit was recorded. Conversely, if participants incorrectly indicated the presence of an offset, a false alarm (FA) was recorded. The level of d′ was calculated for each condition. Participants completed 20 practice trials before completing 600 experimental trials equally distributed across all factors of offset conditions (pop-out target versus distractor) and TPI. 

Figure 1

 

A typical trial sequence for Experiments 1 and 2. Pop-out and distractor stimuli were equally likely to be removed between both stimulus displays.

Results and discussion

Table 1 presents subjects' average performance in detecting the offsets of both pop-out and distractor stimuli as a function of TPI. To examine participants' relative sensitivity to these display changes, d′ was calculated at each level of TPI within each factor of offset condition (pop-out or distractor). When either the hit rate is one (100%) or the FA rate is zero, d′ has an unlimited numeric value and hence cannot be included in the analysis. To account for this, the estimates of d′ were converted as suggested by Macmillan and Creelman (1991), and proportions of one were converted to [1 − 1 / (2N)] and proportions of zero were converted to [1 / (2N)], with N representing the number of trials within that proportion. 

Table 1

 

The average percentage of correct detections (correct rejections for the TPI-0 condition) for Experiment 1 is presented as a function of TPI and offset condition. Standard deviations are presented in brackets.

	TPI-0	TPI-1	TPI-2	TPI-3	TPI-4
Pop-out stimuli	–	91.53 (7.86)	82.54 (14.05)	69.69 (5.67)	61.81 (16.65)
Distractor stimuli	92.74 (cr) (13.27)	14.45 (11.71)	11.10 (14.70)	13.14 (11.20)	10.52 (7.14)

Estimates of d′ were submitted to a 2 (offset condition) × 4 (TPI) repeated measures ANOVA. This revealed a main effect of offset condition, demonstrating that, overall, participants were more sensitive to the offset of pop-out items compared to the offset of distractor items, F(1, 12) = 244.75, p < 0.001, partial η ² = 0.95. A main effect of TPI, F(3, 36) = 7.83, p < 0.001, partial η ² = 0.40, was also observed, indicating that sensitivity to the detection of offsets varied as a function of TPI. The interaction between both these factors was also significant, F(3, 36) = 9.18, p < 0.001, η ² = 0.43. Planned comparisons were conducted between each adjacent level of TPI within each offset condition to test whether there were any decrements in sensitivity ( d′) as the TPI increased. Comparisons using two-tailed t-tests were conducted with a corrected threshold of p < 0.05, using the Student–Newman–Keuls procedure. As shown in Figure 2, there was a significant decrement in sensitivity to the detection of a pop-out item offset between the TPI of 1 and the TPI of 2, t(12) = 2.82, p < 0.05. No other observed decrements of d′ between adjacent TPI levels were observed. Thus, even though sensitivity was relatively high in the TPI-2 condition (2.61), a significant decrement in sensitivity to pop-out offset was still observed when compared to the TPI-1 condition (3.28), thus indicating a decrease in the visual system's ability to prioritize multiple salient items in the display. If there were no such capacity limit, we should not observe any change in sensitivity to salient offsets when multiple pop-out items are present in the visual array. In Experiment 2, we aimed to further test this phenomenon by employing two display set sizes, one small and one large, to test whether the absolute amount of visual information presented (i.e., crowding) mediates the visual system's ability to prioritize salient events. 

Figure 2

 

The average sensitivity scores ( d′) for both pop-out and distractor offset conditions as a function of TPI for Experiment 1. Error bars represent one SEM.

Experiment 2

In Experiment 2 we sought to investigate whether the results of the previous experiment were due to the large set size of sixty items used. Specifically, it is possible that participants had difficulty encoding multiple salient items when many stimuli were spread out in a large search display. Therefore, we employed a lower set size of 20 items as well as the previous larger set size to observe if the degree of stimulus crowding has any effect on the visual system's ability to prioritize multiple salient stimuli. 

Methods

Subjects

Fourteen undergraduates ( M age = 19 years; SD = 2.10) from the University of Toronto participated for course credit. All participants were naive to the purpose of the study and had normal or corrected-to-normal vision. All participants gave informed consent and all ethical guidelines set by the University of Toronto were adhered to in full. 

Apparatus and procedure

The apparatus and procedure were the same as in Experiment 1 except that there was now a set size of either 20 or 60 squares in total, and the TPI now ranged from 1 to 4 items. Because of this, the number of trials was increased to 1200 trials, which were equally distributed across all factors of offset condition, TPI, and set size. Both set size displays subtended the same degree of visual angle, thus resulting in different densities of the displays. 

Results and discussion

Table 2 presents subjects' average performance in detecting the offsets of both pop-out and distractor stimuli as a function of TPI and set size. Sensitivity to these events was again calculated using d′. Estimates of d′ were submitted to a 2 (offset condition) × 2 (set size) × 4 (TPI) repeated measures ANOVA. This again revealed a main effect of offset condition, F(1, 13) = 250.59, p < 0.001, partial η ² = 0.95 and a main effect of TPI, F(3, 39) = 21.24, p < 0.001, partial η ² = 0.62. A main effect of set size was also observed, F(1, 13) = 34.86, p < 0.001, partial η ² = 0.72, suggesting that this factor modulated overall sensitivity to offset events. These main effects were further qualified with several interactions. An offset condition × set size interaction, F(3, 39) = 10.35, p < 0.01, partial η ² = 0.44, an offset condition × TPI interaction, F(3, 39) = 32.18, p < 0.001, partial η ² = 0.71, a set size × TPI interaction, F(3, 39) = 4.32, p < 0.01, partial η ² = 0.25, and an offset condition × set size × TPI interaction, F(3, 39) = 4.78, p < 0.01, partial η ² = 0.27, were observed. These results were further examined by subsequently performed planned comparisons to investigate at what point a decrement in offset sensitivity occurs as a function of both TPI and set size. 

Table 2

 

The average percentage of correct detections for Experiment 2 is presented as a function of TPI, offset, and set size conditions. Standard deviations are presented in brackets.

	TPI-1	TPI-2	TPI-3	TPI-4
Pop-out stimuli (20 set size)	94.66 (4.05)	86.28 (7.31)	78.14 (12.66)	73.71 (15.10)
Distractor stimuli (20 set size)	20.57 (8.71)	16.28 (10.52)	17.19 (9.87)	17.85 (11.38)
Pop-out stimuli (60 set size)	92.76 (5.57)	82.00 (9.24)	70.57 (16.64)	56.57 (17.74)
Distractor stimuli (60 set size)	5.00 (3.54)	5.53 (4.25)	7.73 (7.58)	6.68 (5.99)

The first series of tests were conducted within each set size condition in order to confirm the effects observed in Experiment 1, again using two-tailed t-tests with a corrected threshold of p < 0.05, employing the Student–Newman–Keuls procedure. As shown in Figure 3, there was again a significant decrement in sensitivity to a pop-out offset between a TPI of 1 and a TPI of 2, in both the 20 set size, t(13) = 3.26, p < 0.05 (3.66 vs. 3.01), and 60 set size, t(13) = 3.44, p < 0.05 (3.40 vs. 2.90), conditions. A significant decrease in sensitivity to pop-out offsets was also observed between a TPI of 2 (3.01) and 3(2.58) in the 20 set size condition, t(13) = 3.17, p < 0.05, and between a TPI of 3 (2.46) and 4 (1.60) in the 60 set size condition, t(13) = 3.11, p < 0.05. In addition to this, a significant decrement in detecting distractor offsets was observed between a TPI of 1 (1.07) and 2 (0.73) in the 20 set size condition, t(13) = 3.44, p < 0.05. The second set of planned comparisons focused on contrasting the two set size conditions to observe if an increase in total visual information would differentially affect the pattern of relative sensitivity to pop-out items previously observed. Importantly, there was no significant difference between the 20 and 60 set size conditions at a TPI of 1, 2, and 3 in both offset conditions. A significant difference was, however, observed at a TPI of 4, t(13) = 3.70, p < 0.05, where participants were more sensitive to pop-out offsets in the 20 set size condition (2.65) relative to the 60 set size condition (1.60). The present set of results in Experiment 2 replicated the observed decrement in the visual system's sensitivity for multiple concurrently occurring salient events observed in Experiment 1, while also providing evidence that the previous effects at low TPI levels were not due solely to some stimulus crowding or absolute information account. Having said this, an overall significant difference in d′ between set sizes, specifically at a TPI of 4, was observed, suggesting that stimulus crowding did have some effect in how salient stimuli were prioritized by the perceptual system. This might relate to the mutually suppressive effects within visual cortex that are thought to take place when multiple salient items compete for awareness (e.g., Beck & Kastner, 2005; Torralbo & Beck, 2008). Nonetheless, when considering low TPI levels where more than two salient visual stimuli competed for awareness, a marked decrease in ability to detect a change accurately was observed regardless of the total amount of information present in the array. 

Figure 3

 

The average sensitivity scores ( d′) for offset and set size conditions as a function of TPI for Experiment 2. Error bars represent one SEM.

Experiment 3

In Experiments 1 and 2, participants were presented with pop-out items that shared a unique feature relative to the homogenous background items, namely their red color. Because of this, it is possible that our previous effects could be due to an attentional set for to color red and not due to their relative salience to background items (Folk & Remington, 1992). We therefore used a heterogeneous multiple pop-out display in Experiment 3, in order to test if the above reported effects were in fact due to a capacity limit in sensory encoding and not an attentional control set. 

Methods

Subjects

Fourteen undergraduates ( M age = 21 years; SD = 2.85) from the University of Toronto participated for course credit. All participants were naive to the purpose of the study and had normal or corrected-to-normal vision. All participants gave informed consent and all ethical guidelines set by the University of Toronto were adhered to in full. 

Apparatus and procedure

The apparatus and procedure were the same as in Experiment 1 with the exception of the pop-out items consisting of heterogeneous colors (red, cyan, purple, and orange). Colors were chosen randomly and no more than one of each potential color was presented in each trial. That is, a display with three pop-out targets could contain one red, one cyan, and one purple square embedded among green distractor squares. 

Results and discussion

Table 3 presents subjects' average performance in detecting the offsets of both pop-out and distractor stimuli as a function of TPI. Sensitivity to these events was again measured by computing d′. Estimates of d′ were submitted to a 2 (offset condition) × 4 (TPI) repeated measures ANOVA. This revealed a main effect of offset condition demonstrating that, overall, participants were more sensitive to the offset of pop-out items compared to the offset of distractor items, F(1, 13) = 443.26, p < 0.001, partial η ² = 0.97. There was also a significant interaction between the factors of offset condition and TPI, F(3, 39) = 7.51, p < 0.001, η ² = 0.36. Again, planned comparisons using two-tailed t-tests were conducted with a corrected threshold of p < 0.05, between each adjacent level of TPI within each onset condition to test whether there were any decrements in sensitivity ( d′) as the TPI increased. As shown in Figure 4 there was in fact a significant decrement in sensitivity to the detection of heterogeneous pop-out item offset between the TPI of 1 (3.13) and the TPI of 2 (2.67), t(13) = 2.90, p < 0.05, and the TPI of 2 and 3 (2.24), t(13) = 2.57, p < 0.05, indicating that the results of Experiments 1 and 2 were not driven by an attentional control set for one particular stimulus feature. No other observed decrements of d′ between adjacent TPI levels were observed. 

Table 3

 

The average percentage of correct detections (correct rejections for the TPI-0 condition) for Experiment 3 is presented as a function of TPI and offset condition. Standard deviations are presented in brackets.

	TPI-0	TPI-1	TPI-2	TPI-3	TPI-4
Pop-out stimuli	–	89.84 (5.25)	79.42 (9.14)	64.20 (12.15)	59.61 (16.06)
Distractor stimuli	88.97 (cr) (10.45)	6.25 (3.66)	5.99 (5.03)	6.24 (5.30)	4.61 (4.32)

Figure 4

 

The average sensitivity scores ( d′) for both heterogeneous pop-out and distractor offset conditions as a function of TPI for Experiment 3. Error bars represent one SEM.

Experiment 4

Even though previous experiments demonstrated a significant decrement in perceptual sensitivity when more than one pop-out item was present, it is possible that this was due to the fact that participants had to concurrently monitor changes in both pop-out and distractor items, thus presenting an added attentional demand. We thus conducted a fourth experiment where the display remained the same, but task instructions were changed so that participants reported only the offset of pop-out items. Such instructions should lower the attentional load and allows us to test if our previous effects were indeed perceptual in nature. 

Methods

Subjects

Twenty undergraduates ( M age = 22 years; SD = 5.37) from the University of Toronto participated for course credit. All participants were naive to the purpose of the study and had normal or corrected-to-normal vision. All participants gave informed consent and all ethical guidelines set by the University of Toronto were adhered to in full. 

Apparatus and procedure

The apparatus and procedure were the same as in Experiment 1, except that participants no longer were asked to report the offset of distractor items, thus reporting only the offset of a pop-out item. 

Results and discussion

Table 4 presents subjects' average performance in detecting the offsets pop-out stimuli as a function of TPI. Sensitivity was again measured by computing d′. Estimates of d′ were submitted to a 4 (TPI) repeated measures ANOVA. This revealed a main effect of TPI level, F(3, 57) = 36.41, p < 0.001, partial η ² = 0.67. Planned comparisons, using two-tailed t-tests, were conducted with a corrected threshold of p < 0.05 and were conducted between each adjacent level of TPI to test whether there were any decrements in sensitivity ( d′) as the TPI increased. As shown in Figure 5 there was a significant decrement in sensitivity to the detection of pop-out item offset between the TPI of 1 (3.64) and the TPI of 2 (3.28), t(19) = 3.47, p < 0.05, and the TPI of 2 and 3 (2.78), t(19) = 3.58, p < 0.05, indicating that the results of previous experiments were in fact due to a capacity for detecting changes in pop-out offsets and not due to an increase in attentional demand from having to monitor all display items on screen. No other observed decrements of d′ between adjacent TPI levels were observed. 

Table 4

 

The average percentage of correct detections for Experiment 4 is presented as a function of TPI. Standard deviations are presented in brackets.

	TPI-1	TPI-2	TPI-3	TPI-4
Pop-out stimuli	94.72 (5.21)	89.06 (6.86)	81.03 (7.33)	78.54 (8.97)

Figure 5

 

The average sensitivity scores ( d′) for the detection of pop-out offsets as a function of TPI for Experiment 4. Error bars represent one SEM.

Experiment 5

Finally, it remains possible that an attentional control set for abrupt offset events might have contributed to previously observed effects (e.g., Yantis, 1993; Yantis & Jonides, 1996). This fifth experiment therefore used target onsets instead of offsets to test this possibility. The general task remained the same except display changes involving pop-out items were now presented as onsets after all items in the initial display were individually masked, thus requiring participants to report whether there was a pop-out onset. 

Methods

Subjects

Thirteen undergraduates ( M age = 22 years; SD = 5.20) from the University of Toronto participated for course credit. All participants were naive to the purpose of the study and had normal or corrected-to-normal vision. All participants gave informed consent and all ethical guidelines set by the University of Toronto were adhered to in full. 

Apparatus and procedure

The apparatus and procedure were similar to that of Experiment 1 with the key difference being that a square was added instead of being removed from the display. Specifically, after the initial display of 59 squares, each square was then individually masked by a box with a random assortment of lines subtending 0.7° × 0.7° of visual angle. The initial display had 0–3 pop-out items present. On each trial, an additional box mask also appeared in a randomly chosen spatial location that was not yet occupied by any stimulus. The array of squares remained individually masked for 50 ms, at which point the array of masks was removed, revealing the presence of an additional stimulus on half the trials; half the time this new item was either a red pop-out square or a green distractor square. The TPI in this case represents the total number of pop-out items present once the new square was revealed and ranged from 0 to 4 items. Once the array of squares was displayed for an additional 250 ms, the whole display was masked for 500 ms and participants then indicated if they observed a square onset by pressing “z” for yes and “/” for not. Hits and FAs were again recorded, allowing for relative sensitivity ( d′) of these events to be calculated as a function of TPI. 

Results and discussion

Table 5 presents subjects' average performance in detecting the onsets of both pop-out and distractor stimuli as a function of TPI. Sensitivity to these events was again measured by computing d′. Estimates of d′ were submitted to a 2 (onset condition) × 4 (TPI) repeated measures ANOVA. This revealed a main effect of onset condition demonstrating that, overall, participants were more sensitive to the onset of pop-out items compared to the onset of distractor items, F(1, 12) = 353.80, p < 0.001, partial η ² = 0.97. There was also a significant interaction between the factors of onset condition and TPI, F(3, 36) = 4.19, p < 0.001, η ² = 0.28. Again planned comparisons between each adjacent level of TPI, using two-tailed t-tests were conducted with a corrected threshold of p < 0.05. As shown in Figure 6, there was again a significant decrement in sensitivity to the detection of pop-out item onset between the TPI of 1 (3.46) and the TPI of 2 (3.10), t(12) = 3.17, p < 0.05, indicating that the results of the previous four experiments were not due to an attentional control set for offset events. 

Table 5

 

The average percentage of correct detections (correct rejections for the TPI-0 condition) for Experiment 5 is presented as a function of TPI and onset condition. Standard deviations are presented in brackets.

	TPI-0	TPI-1	TPI-2	TPI-3	TPI-4
Pop-out stimuli	–	90.59 (4.45)	86.58 (10.64)	84.93 (9.13)	85.64 (9.37)
Distractor stimuli	91.06 (cr) (10.91)	4.08 (8.25)	2.36 (2.22)	3.71 (2.63)	2.97 (4.19)

Figure 6

 

The average sensitivity scores ( d′) for both pop-out and distractor onset conditions as a function of TPI for Experiment 5. Error bars represent one SEM.

General discussion

In Experiment 1, a significant decrement in the visual system's ability to prioritize multiple concurrent salient items was observed when the number of pop-out items present in the visual array increased from one to two. This indicated that some capacity limit of visual information occurs during stimulus encoding. With this said, participants were still overall more accurate at detecting display changes involving pop-out items relative to distractor items regardless of TPI condition. This clearly shows that these items still receive prioritized processing. Nevertheless, the reduced sensitivity for salient visual items as TPI increases indicates that once more than one salient item competes for perceptual resources, the relative salience of each item is reduced. This phenomenon was also demonstrated in Experiment 2 under both small and large set sizes, thereby indicating that this observation was not a product of stimulus crowding effects or the absolute amount of visual information present in the visual array. Further, an attentional control set account for these observed results is unlikely as Experiment 3 demonstrated that when pop-out stimuli existed in a heterogeneous context, a capacity limit was still observed. In addition, the results of Experiment 4 further clarify that this capacity was not produced by an added attentional demand of having to monitor both distractors and pop-out items in the display. Finally, when participants reported the presence of pop-out onsets instead of offsets, a consistent pattern of results once again emerged. Together, these results indicate that when two or more salient items compete for perceptual representation, they attenuate each other's ability to be processed as efficiently as when a single salient item is presented in isolation. 

Individual differences between this observed capacity limit were small but still present. As depicted in Figure 7, a few participants did not show a decline in sensitivity to pop-out displays between a TPI of 1 and 2, but rather showed this decline between a TPI of 2 and 3. Thus, for these individuals, up to two pop-out items were apparently encoded without any performance cost. Therefore, it appears that a capacity limit for the visual system's ability to process concurrent salient items with equal sensitivity is about 1 or 2. This fits well with recent work by Franconeri, Alvarez, and Enns (2007) who demonstrated that only 2–3 precise spatial locations could be selected by the visual system at one time. Even though this estimate of spatial region selection is slightly higher than our own capacity observations, it should be noted that our task was not impingent on spatial encoding but simply the detection of a change in the array at a randomly chosen location. These observed effects are also complemented by findings indicating that when a certain region of space is selected for further processing, it reduces the spatial resolution of other objects present in the visual array (Yeshurun & Carrasco, 1998). Thus it is plausible that when one or two salient items are encoded, the resolution of the remaining items is significantly reduced, thereby accounting for the decreased degree of sensitivity in detecting changes involving these items. 

Figure 7

 

Depiction of sensitivity differences between high and average capacity participants. Data from all experiments were analyzed collectively with participants who did not show a decrement in sensitivity ( d′) between a TPI of 1 and 2 analyzed separately. High-capacity individuals do not show a decrement in sensitivity to pop-out stimuli until more than 2 are presented, while the average participant shows this decrement after more than 1 pop-out stimuli are presented. Error bars represent one SEM.

Our findings have implications beyond the pop-out literature. For example, when considering the phenomenon of subitizing, the rapid enumerating of a small number of visual items without effort, similar processes might be operating as those observed in the current study. There is discussion in the literature regarding whether subitizing occurs at early preattentive stages of visual encoding or at a later attentional selection stage. Typically, RTs in response to reporting the number of briefly presented items on a screen increase around three to four total items, indicating a switch from a preattentive process to a more attentionally demanding counting process (e.g., Jensen, Reese, & Reese, 1950; Piazza, Giacomini, Le Bihan, & Dehaene, 2003; Trick & Pylyshyn, 1994). When considering our results based on perceptual sensitivity, however, it appears that the capacity to encode multiple salient stimuli without a cost seems to be limited to one or two items. Indeed, recent work by Railo, Koivisto, Revonsuo, and Hannula (2008) directly comparing subitizing during both divided and full-attention conditions found similar results. The authors observed a significant decline in enumeration accuracy between one and two presented items in the preattentive condition. The capacity was, however, larger in the full-attention condition, where a decline was not observed until three items were presented. Taken together, these results suggest that capacity limits, potentially in preattentive processing, are perhaps not restricted to embedded salient items within a uniform background but are of a more general nature that limit the total amount of items that can be encoded at one time. 

Interestingly, similar capacity limits have been proposed to exist in visual working memory. In a recent review, Jonides et al. (2008) discussed several studies showing that only one item could be held in VTSM. One such study was conducted by Garavan (1998), who found that participants could only attend to one object held in VSTM at one given time by having them keep two running counts of intermixed displays of two shape categories. Responses to update the total count for each category were significantly longer when the preceding item was of the opposite category, indicating that only one item could be held in visual working memory for further processing at one time. 

The results of the present study imply that capacity limits in VSTM under certain conditions might originate at the stimulus encoding phase of visual processing. This raises the possibility, however, that our observed set of results may actually be due to VSTM constraints as opposed to some purely perceptual account. After all, our paradigm did include a delay period. This is, however, unlikely because if the visual system's ability to prioritize multiple concurrent salient visual events was primarily gated by VSTM capacities, we would expect to find no difference between low TPI levels when considering participants' detection performance, as most observers do very well in tests of VSTM capacity measured by change-detection paradigms with set sizes below 4 (e.g., Alvarez & Cavanagh, 2004). Further to this, as average VSTM capacity is around three or four items (e.g., Vogel, Woodman, & Luck, 2001), it is unlikely that a VSTM limit is responsible for our reported effects. We observed a consistent detection decrement from 1 to 2 TP1, which would suggest that our participants had an unlikely low VSTM capacity of only 1 item.¹ Our data, however, cannot completely rule out the possibility that VSTM or attentional limitations are involved in this decrement in sensitivity to multiple salient items competing for awareness. Further follow-up work directly examining how visual cortex allocates perceptual resources in the context of multiple pop-out displays using electroencephalography (EEG) is currently being conducted in our laboratory to elucidate this process. 

The current study also speaks to the larger question of how the brain allocates neural resources when two or more salient items compete for awareness, and how this “decision” is reconciled. When more than one stimulus is present in the visual field, mutually suppressive interactions occur in the neural response to each stimulus, thereby degrading the representation of each item (Miller, Gochin, & Gross, 1993). When one of the presented stimuli is processed preattentively in the context of a pop-out display, these competitive interactions are eliminated, and the pop-out stimulus is given priority in visual cortex (Beck & Kastner, 2005). In the context of the current results, the question that arises is how are two or more salient stimuli items neurally represented. One possibility is that one to two salient items are given preferential neural representation, thereby increasing their resolution, while additional salient items will be down-regulated. Another possibility is that when two or more salient items are encoded, they compete for awareness in a mutually suppressive manner, much like the suppressive interactions that multiple items in a homogeneous display would produce. Evidence for a biased competition account of capacity limitations during early visual processing has recently come from Torralbo and Beck (2008), who found that increased stimulus proximity increased stimulus competition, but only when items were presented in the same hemifield. As receptive fields in later areas of visual cortex are large enough to encompass both hemifields, this result would not be expected unless stimulus competition is actually taking place in early areas of visual cortex where receptive fields span only one hemifield. A similar mechanism where competition in early visual cortex is responsible for our observed results could also be possible (see also Motter & Simoni, 2006; Reddy & VanRullen, 2007). 

In conclusion, our finding of stark differences in sensitivity to change events involving pop-out and distractor items at all non-zero levels of TPI indicates that the prioritization of pop-out stimuli is not completely eliminated in a multiple pop-out context, but the allocation of perceptual resources in a multiple pop-out context is nonetheless inherently different compared to a single pop-out context. 

Acknowledgments

This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada to Susanne Ferber. Greg West was supported by a Canada Graduate Scholarship from the Natural Sciences and Engineering Research Council of Canada. We thank Hrag Pailian, Wardah Malik, and Shawnna Balasingham for assistance with data collection. 

Commercial relationships: none. 

Corresponding author: Greg L. West. 

Email: [e-mail]. 

Address: Department of Psychology, University of Toronto, 100 St. George Street, Toronto, ON M5S 3G3, Canada. 

Footnote

Footnotes

1   ¹We thank an anonymous reviewer for pointing this idea out to the authors.