Competition increases binding errors in visual working memory

Abstract

When faced with maintaining multiple objects in visual working memory, item information must be bound to the correct object in order to be correctly recalled. Sometimes, however, binding errors occur, and participants report the feature (e.g., color) of an unprobed, non-target item. In the present study, we examine whether the configuration of sample stimuli affects the proportion of these binding errors. The results demonstrate that participants mistakenly report the identity of the unprobed item (i.e., they make a non-target response) when sample items are presented close together in space, suggesting that binding errors can increase independent of increases in memory load. Moreover, the proportion of these non-target responses is linearly related to the distance between sample items, suggesting that these errors are spatially specific. Finally, presenting sample items sequentially decreases non-target responses, suggesting that reducing competition between sample stimuli reduces the number of binding errors. Importantly, these effects all occurred without increases in the amount of error in the memory representation. These results suggest that competition during encoding can account for some of the binding errors made during VWM recall.

Introduction

In order to successfully perform many tasks, it is necessary to encode, maintain, and retrieve information using visual working memory (VWM). Often, multiple visual objects must be maintained simultaneously, such as when remembering rejected distractors while performing a visual search (Emrich, Al-Aidroos, Pratt, & Ferber, 2010). When storing multiple items in VWM, however, there are a number of computational problems that must be overcome. First, because cognitive resources are limited, VWM cannot represent an unlimited amount of information (Marois & Ivanoff, 2005). This is evidenced by the finding that VWM performance decreases substantially as the memory load increases (Vogel, Woodman, & Luck, 2001). Although numerous studies have investigated precisely how information in VWM is limited, it remains debated whether this problem is solved by storing a limited number of high-precision, high-resolution objects (Anderson, Vogel, & Awh, 2011; Cowan & Rouder, 2009; Zhang & Luck, 2008) or whether the proportion of resources dedicated to each representation decreases as the memory load increases, resulting in a larger number of low-precision representations (Bays & Husain, 2008, 2009; Huang, 2010). 

The second problem that must be solved when faced with remembering multiple items is accurately maintaining and recalling bound representations. That is, if multiple items are stored in memory, then the identities and locations of each item must be correctly maintained and recalled in order to use that information in a meaningful way. While a number of studies have examined whether VWM stores bound or independent features (e.g., Fougnie, Asplund, & Marois, 2010; Vogel et al., 2001), relatively fewer studies have examined binding errors in VWM (i.e., misbinding of features to incorrect locations; Johnson, Hollingworth, & Luck, 2008; Wheeler & Treisman, 2002). Recently, Bays, Catalao, and Husain (2009) demonstrated that individuals make a significant number of spatial errors, particularly as the number of to-be-remembered items increases. In that study, the authors utilized a memory recall procedure developed by Wilken and Ma (2004) to examine what proportion of responses could be accounted for by incorrectly reporting the color of an unprobed item (i.e., a non-target). A previous study by Zhang and Luck (2008) demonstrated that responses could be composed of either target responses (normally distributed around the target color, with some amount of error) or guesses (uniformly distributed responses). The analysis by Bays et al. (2009), however, demonstrated that a significant portion of the “guesses” were not random; instead, they could be accounted for by non-target responses. In other words, participants were remembering significantly more items than were correctly reported, but they were not always able to accurately maintain or recall the location of multiple items. 

The source of these non-target responses in VWM recall remains unclear, however. Bays et al. (2009) attributed these errors to noise in the memory process itself rather than to errors in encoding. Specifically, as the amount of memory resources dedicated to each item decreases, there should be a proportional increase in the amount of spatial error. This increase in noise in spatial memory would result in a proportional increase in binding errors for nearby objects. The conclusion that these responses occurred independent of encoding limitations was supported by the finding that the proportion of non-target responses was unaffected by changes to the sample duration. That is, increasing the amount of time participants have to encode items has no effect on non-target responses, suggesting that they are unrelated to limitations in encoding processes. 

Distributing attentional and perceptual resources over multiple objects, however, has significant behavioral consequences, independent of noise in VWM representations. For example, reporting one feature of two objects is much more difficult than reporting two features of the same object (Duncan, 1984). Therefore, it is possible that errors in spatial memory could be accounted for by limitations in early attentional or perceptual processes. In other words, the precision of VWM representations (including spatial precision) may be limited by processes that occur prior to VWM maintenance. 

One source of the resulting binding errors could lie in competition for representation that occurs between multiple attended items. That is, cells in the visual system are tuned to respond to a preferred stimulus in a particular region of space. Competition arises if two objects fall within the same receptive field of a cell, as both objects compete for that neuron's response (for a review, see Desimone & Duncan, 1995). Consequently, competing objects interact in a mutually suppressive way, leading to suppressed responses for a stimulus relative to when no competition exists between objects (Kastner, De Weerd, Desimone, & Ungerleider, 1998; Kastner et al., 2001). Thus, presenting multiple objects in sequential rather than simultaneous displays can mitigate both the competition for representation that occurs at the neural level (Kastner et al., 1998, 2001), as well as the behavioral consequences of this competition (Duncan, 1980). Given that these effects of competition emerge independent of memory maintenance requirements, it is possible that competition in early stages of visual processing could lead to information being incorrectly bound in, and subsequently recalled from, VWM. In other words, errors in early processes would be encoded into VWM and propagated downstream. 

Importantly, competition for representation increases as items are presented closer together in space, as these items will compete for representation within a greater number of receptive fields. Thus, the ability to select and attend to multiple objects is directly affected by the distance between them (Bahcall & Kowler, 1999; Mounts, 2000), particularly when objects need to be individuated (McCarley & Mounts, 2007). Similarly, the number of locations that can be accurately selected may depend on how precisely locations must be selected; when items are presented in dense displays, the number of locations that can be accurately selected is much lower than when targets are presented further apart (Franconeri, Alvarez, & Enns, 2007). Interestingly, illusory conjunctions also increase when objects are presented in close proximity (Cohen & Ivry, 1989), indicating that competition may affect the very binding processes that could lead to non-target responses. 

The goal of the present study is to examine the source of non-target responses in a VWM recall task (Wilken & Ma, 2004). More specifically, we aimed to test whether they can result from competition for representation that occurs before VWM maintenance. Accordingly, in a series of experiments, we manipulated the amount of competition between memory items and examined its effect on the proportion of non-target responses, as well as on the precision (error) of correctly recalled items. If competition affects the ability to accurately encode information into VWM, then the proportion of non-target responses should increase when items are presented close together and compete for a greater number of receptive fields (Experiment 1). Moreover, the effect of competition on non-target responses should be directly proportional to the distance between competing representations (Experiment 2). However, a resource account of non-target responses could similarly account for an increase in binding errors when items are presented close together in space. Consequently, if competition for representation, rather than spatial memory error, increases the number of non-target responses, then these errors should decrease when items are presented sequentially rather than simultaneously (Experiment 3). The results demonstrate that competition significantly increases the proportion of non-target responses and that this effect cannot be accounted for solely by an increase in noise in the memory representation itself. 

Experiment 1

If multiple objects are presented in the visual display, they interact in a mutually suppressive way and compete for neural representation (Desimone & Duncan, 1995; Kastner et al., 1998, 2001). Furthermore, the effects of these competitive interactions are spatially graded. That is, because the size of receptive fields in the visual stream increases from less than 2° in primary visual cortex to 4–6° in V4 (Kastner et al., 2001), two objects that fall within 2° should compete for representation from V1 to V4 but, when separated by several degrees, should only compete for representation within areas V4 and beyond. The behavioral consequences of this change in receptive field size has been demonstrated in the form of localized attentional interference (LAI), whereby the processing of multiple targets is most impaired when both items are contained within a very small region of space, with performance increasing as a function of the distance between items (Bahcall & Kowler, 1999; McCarley & Mounts, 2007, 2008; McCarley et al., 2004; Mounts, 2000; Mounts & Gavett, 2004; Mounts, McCarley, & Terech, 2007). In other words, as the distance between targets decreases, the competition for neural representation increases, resulting in greater impairments in the identification and representation of those stimuli. 

If this competition for representation also affects the ability to accurately encode information in VWM, then similar decreases in VWM performance should be observed when the distance between memory stimuli is reduced. Consequently, Experiment 1 used the three-component model described by Bays et al. (2009) to test whether competition for neural representation (defined as the distance between objects) has a similar effect on VWM performance as is observed during LAI. Specifically, we sought to test between three alternative predictions. If competition for representation affects the overall fidelity of VWM representations, then decreasing the distance between sample items should result in an increase in both the proportion of non-target responses and an increase in error (as measured by the standard deviation, SD) of target responses. In other words, increasing the amount of noise in the representation should result in a reduction in memory precision for both the target color and its location. In contrast, if non-target responses result from competition for representation during encoding rather than from an increase in noise in the VWM representation itself, then decreasing the distance between sample items should only affect the proportion of non-target responses. It is important to point out, however, that this result by itself cannot distinguish between a competition account and a resource account of non-target responses (see below). 

Methods

Participants

Eighteen undergraduates (4 males, 17 right-handed, ages: 19–36 years; M = 22) from the University of Toronto participated in this experiment for partial course credit. All participants had normal or corrected-to-normal vision and had self-reported normal color vision. All procedures were approved by the University of Toronto Research Ethics Board. 

Apparatus

Participants were seated in front of a 19-inch CRT monitor at a distance of approximately 57 cm. The screen resolution was 1280 × 960 and the refresh rate was 60 Hz. Stimuli were generated and presented using Matlab and the Psychophysics Toolbox 3 extensions (Brainard, 1997; Kleiner et al., 2007). 

Stimuli and procedure

Each trial began with the presentation of 1–3 colored squares for 100 ms, presented around a central black fixation cross 1.2° × 1.2°. Following a 1,000-ms blank delay containing only the fixation, the probe display was presented. During the probe, the locations of the sample items were indicated by black placeholder boxes. One of the sample locations was identified as the target location by displaying a placeholder with a line width of 8 pixels. The remaining non-target locations were indicated with a line width of 4 pixels. In addition to the memory placeholders, a color wheel containing all possible sample colors was presented surrounding the test area (Figure 1). 

Figure 1

 

(Left) Schematic of the recall task. Participants are instructed to remember as many of the items presented in the sample as possible and to click on the location of the color wheel presented in the probe that matches the color of the probed (highlighted) item. (Right) Schematic of the three different types of responses extracted by the mixture model analysis. If the color is correctly remembered, participants should select the target color on a large proportion of trials. These target responses (P _T) are normally distributed around the target color. If the target color is forgotten, participants may guess. These guesses (P _G) are assumed to be random and so are uniformly distributed across all values. On some proportion of trials, however, participants will incorrectly report the color of an unprobed, non-target item (P _NT). Adopted from Bays et al. (2009).

Participants were instructed to remember as many of the sample items as possible over the delay period and to indicate the color of the target square by clicking on the location on the color wheel that most closely matched the color of the cued item. During instructions, emphasis was placed on accuracy, and participants were instructed to make multiple responses if the selected color did not match closely enough to the remembered color of the target item. Each time a color was selected, the placeholder at the target location was replaced with a colored square that matched the shade of the participant's response. The trial ended when the subject pressed the keyboard space bar, and the final selected color was taken as the participant's response for that trial. Participants were also instructed to guess a random color if they could not accurately recall the color of the target item and to actively avoid selecting the color of a non-target item if they knew that it was not the color of the probed location. Each trial was followed by a 500-ms ITI, with only the fixation cross displayed. 

Memory stimuli were selected from an array of 252 colors, each of which corresponded to an equally spaced point on the hue–saturation–value (HSV) color wheel. Saturation and value (brightness) were held constant at the maximum value. Thus, the intensity and brightness of the stimuli remained largely constant regardless of particular stimulus hue. For the probe display, the color wheel contained all 252 colors arranged on an annulus with an outer radius of 8.3° and an inner radius of 6°. The colors of all target and sample stimuli were selected randomly, with the constraint that any two sample colors had to be separated by a minimum of 20 values on the color wheel (28.5°). 

Sample squares were 1.2° × 1.2° in size and were presented at one of 18 locations on an invisible circle with a radius of 5.4°. Critically, the distance between sample stimuli was manipulated (see Figure 2A). In the high-competition condition, two stimuli were located at adjacent locations, while in the low-competition condition, two empty locations occupied the locations between two stimuli. Thus, for a set size of two items (Load 2 condition), high-competition stimuli were separated by a center-to-center distance of 1.9° and were contained within 3.1°, while low-competition stimuli were separated by a distance of 5.4° and were contained within 6.6°. Although the Load 2 condition could assess whether the distance between items could affect memory performance, it required that the low- and high-competition conditions be presented with different stimuli configurations. Therefore, we used the same configuration in the Load 3 condition, such that each display contained two stimuli in a high-competition configuration and the third item located in a low-competition configuration relative to one of the other stimuli (Figure 2A). Thus, to control for stimuli configuration the maximum distance between three items in the Load 3 condition was 6.6°, as in the Load 2 condition. Critically, using the same configuration of stimuli, the probed stimulus in the Load 3 condition could be either a low-competition or high-competition item. When two items were presented, the location of the target square was assigned randomly. For set size three, the target item in the high-competition condition was randomly sampled from either of the two high-competition items. Although this design led to a greater number of probe trials to the low-competition items (50% of trials) than to the high-competition items (25% each), potentially encouraging participants to adopt a strategy of attending to the single low-competition item, this design prevented the reverse strategy of preferentially attending to the high-competition items (which would be probed on 66% of trials if all items were probed equally), while allowing us to acquire a greater number of trials per condition. In addition, responses were analyzed for statistical outliers that would have indicated a strategy of attending to either group of items (see below). 

Figure 2

 

(A) Schematic of the stimuli configurations in the Load 2 and Load 3 conditions. In the Load 3 condition, the trial type was determined by identifying which of the sample items was the cued item during the probe. (B) Results obtained from the mixture model. All of P _T, P _NT, and the SD were significantly affected by increasing set size. Only P _NT (middle) was affected by competition, with a greater proportion of non-target errors under high competition. Error bars denote within-subject standard error of the mean (SEM).

Participants performed a total of 80 trials of each condition (Load 1; Load 2, low competition; Load 2, high competition; Load 3, low competition; Load 3, high competition), randomly interleaved in blocks of 50 trials. Prior to beginning the experimental task, participants performed 50 trials each of a 2-, 4-, and 6-item change-detection task to establish baseline VWM capacity and to become familiarized with the general procedure. Colored squares that subtended 0.65° × 0.65° were sampled without replacement from seven colors (black, white, red, yellow, blue, green, and purple) and were presented briefly around the fixation cross on an invisible 4° × 4° grid. After a delay of 1,000 ms, a probe display was presented and participants had to judge whether or not a change had occurred. Capacity (K) was estimated for each task using the Pashler–Cowan formula (Cowan, 2001; Pashler, 1988). K estimates ranged between 1.7 and 5.0 (M = 3.2). This practice task was followed by instructions for the experimental task and a practice of 5–15 trials. The length of the entire experiment was approximately 1 h. 

Analysis

The analysis was performed using the method of Bays et al. (2009; http://www.bayslab.com). For each trial, the angular distance between the selected color value and the target color value was obtained, and a probabilistic model of all responses was calculated (Bays et al., 2009). The model assumes a mixture of three components that are each fit to the distribution of responses for each participant and condition: target responses (P _T), which are normally distributed around the target value using a circular analogue of the Gaussian distribution (the Von Mises distribution); non-target responses (P _NT), which are drawn from a normal distribution of responses around the non-target value with the same SD as the target responses; and random guesses (P _G), which have a uniform distribution across all potential values. This method differs from the method of Zhang and Luck (2008), in that the proportion of responses that are assumed to be guesses will be smaller, as some of these responses will be extracted by P _NT. Values for each parameter were obtained separately for each participant and condition using maximum likelihood estimation (Bays et al., 2009). 

Hypothesis tests for experimental manipulations were performed using repeated measures analysis of variance (ANOVA) and t tests for the maximum likelihood estimates of each of the measures obtained from each subject and condition. Effects of set size were examined across Loads 1–3, while the effects of spatial competition (low vs. high) were considered only for Loads 2 and 3. 

Parameter estimates from the three-component model for each observer were examined for statistical outliers (>3 SD) prior to analysis. To exclude the possibility that participants adopted a strategy of attending to only the low- or high-competition items in the Load 3 configuration, differences in accuracy between the two types of probes were examined for outliers. Two individuals demonstrated a greater than 20% change in P _T between the two conditions and were therefore eliminated from analysis. The SD of the difference in accuracy for the remaining 16 participants was 6%. 

Results

Proportion of target responses

To determine if spatial separation affects whether sample items are correctly recalled, the maximum likelihood estimates of the proportion of target responses (P _T) from the mixture model were examined. Collapsing across competition conditions, a significant effect of set size on P _T was observed, F(2,30) = 27.5, MSE = 0.004, p < 0.001. This effect was found to be significant between one (M = 0.97) and two (M = 0.92) items, t(15) = 3.11, SEM = 0.015, p = 0.007, indicating that increasing the number of items in the display has a negative effect on VWM performance, even when the number of sample items is well within the four-item “capacity” of VWM (Figure 2A). 

Examining the effects of spatial separation for Loads 2 and 3 revealed a significant main effect of set size, F(1,15) = 39.5, MSE = 0.005, p < 0.001, but no effect of spatial competition, F(1,15) = 0.41, MSE = 0.002, p = 0.53, and no interaction, F(1,15) = 0.96, MSE = 0.001, p = 0.34. Thus, while increasing the number of items in the display reduces the probability of any item being correctly recalled, the distance between objects had no effect on target accuracy. 

Proportion of non-target responses

While spatial separation has no effect on the probability of recalling an item, a critical question is whether competition for representation affects the ability to correctly encode, and subsequently recall, the location of each item. Consistent with the results of Bays et al. (2009), P _NT increased as the number of to-be-remembered items increased, F(1,15) = 12.2, MSE = 0.001, p = 0.003 (Figure 2B). 

In contrast to P _T, the distance between items (i.e., spatial competition) had a significant effect on P _NT, F(1,15) = 8.4, MSE = 0.002, p = 0.011. Thus, more non-target errors were made under high spatial competition (M = 0.05) relative to when the sample objects were located further apart (M = 0.019). In addition, a significant interaction between set size and competition was observed, F(1,15) = 5.8, MSE = 0.001, p = 0.029. That is, the effect of competition was stronger at the larger set size. 

Target error (SD)

The finding that P _NT is affected by the distance between sample items suggests that competition for representation at early stages of processing increases the likelihood that information will be incorrectly bound and encoded into VWM, resulting in an increase in spatial errors. It remains possible, however, that competition simply reduces the fidelity of the representations stored in VWM. If competition increases the amount of error in the memory representation itself, then the amount of error in target responses should also increase under high competition. That is, previous studies have demonstrated that while features are stored largely independently in VWM, errors tend to increase across multiple feature dimensions when increasing the load on memory resources (Bays, Wu, & Husain, 2011; Fougnie et al., 2010; Wheeler & Treisman, 2002). Thus, if spatial errors simply result from an increase in error in the memory representation, then competition should have a similar effect on the amount of error in non-target responses. Accordingly, to examine the precision of target responses, the SD of the normal distribution obtained from the three-component mixture model was examined (Bays et al., 2009). This measure reflects the precision of only those items that are successfully stored and reported from memory (i.e., P _T and P _NT). Confirming the effects of previous studies, a significant effect of set size was found when collapsing across competition condition, F(2,30) = 22.6, MSE = 0.002, p < 0.001. Thus, each sample item added to the display increased the error of target responses, including between one and two items, t(15) = −7.38, SEM = 0.009, p < 0.001. Importantly, neither the effect of competition, F(1,15) = 1.45, MSE = 0.003, p = 0.25, nor the interaction with set size, F(1,15) = 0.85, MSE = 0.003, p = 0.37, was significant. Thus, the precision of target representations was affected by the number of sample items but not by the distance between items. 

Proportion of guesses

The proportion of guesses (P _G) also increased with set size, F(2,30) = 13.1, MSE = 0.004, p < 0.001, collapsed across competition condition. In contrast, neither the effect of competition, F(1,15) = 3.1, MSE = 0.003, p = 0.098, nor the interaction was significant, F(1,15) = 0.816, MSE = 0.003, p = 0.38. Thus, although competition had a significant effect on the number of non-target responses, competition between items had no effect on the proportion of guesses. 

Discussion

Overall, the results of this experiment support the hypothesis that presenting sample items close together in space results in an increase in non-target responses. That is, while the number of sample items affected all measures, the distance between sample items affected the proportion of non-target responses, without any effect on the other measures. Moreover, items that were stored in memory were stored with a similar resolution irrespective of the amount of spatial competition between them. We propose, therefore, that non-target responses can result from competition for representation during processing stages prior to VWM maintenance (i.e., during selection and encoding): When multiple objects have to be attended and encoded into VWM, these items compete for neural representation in early stages of visual processing. This competition will result in a greater occurrence of binding errors, which are then propagated downstream into VWM representations. 

Experiment 2

The results of Experiment 1 suggest that decreasing the spatial separation between sample items results in an increase in binding errors in VWM. One possibility is that this increase reflects increased error in the encoding processes that results from competition. Consistent with this interpretation, evidence from the attention literature indicates that illusory conjunctions (or misbinding of incorrect features to objects during perception) increase when objects are presented close together in space and are within the focus of attention (Cohen & Ivry, 1989). That is, even in the absence of any explicit memory requirements, binding errors are related to spatial proximity. The effects of competition are also directly related to the distance between competing representations (Bahcall & Kowler, 1999; Kastner et al., 2001; McCarley & Mounts, 2007, 2008; McCarley et al., 2004; Mounts, 2000; Mounts & Gavett, 2004; Mounts et al., 2007). Thus, if binding errors in VWM are reflective of the extent of competition for representation during encoding, the proportion of non-target responses should vary as a function of the distance between sample items. Accordingly, in Experiment 2, we tested a single set size over a range of target separations to examine the spatial specificity of non-target responses. 

Methods

Participants

Fourteen University of Toronto undergraduates (1 male, 13 right-handed, ages: 18–23 years; M = 20.0) participated in this experiment for partial course credit. 

Stimuli and procedure

The procedure was identical to that of Experiment 1, with the following exceptions. On each trial, participants were presented with four colored sample stimuli for 100 ms, randomly presented in one of four conditions. Thus, the set size in this task was held constant. Critically, the distance between objects was manipulated, as the stimuli were presented with increasing separation between items (Separations 1–4). That is, all four sample items were presented at adjacent locations in Separation 1 and with 1–3 empty locations between items in Separations 2–4, respectively (Figure 3). The center-to-center and total linear distances between items are presented in Table 1. As in Experiment 1, the goal for participants was to remember as many of the items as possible over a 1,000-ms delay period and to select from a color wheel the color of the probed item. Responses were analyzed using the same method as in the previous experiment, calculating the precision of all responses (1/SD), as well as obtaining maximum likelihood estimates for each of the three factors in the model described by Bays et al. (2009). The colors and locations of each of the sample patterns were assigned randomly, with the constraint that any two sample colors had to be separated by a minimum of 20 values on the color wheel (28.5°). Although the locations of individual items were not recorded in the initial experiment, a Monte Carlo simulation using 10,000 trials per condition was performed to examine quadrant and hemifield effects (see Discussion section). In each trial, the location of each sample item was determined, and the number of quadrants and hemifields in which all four items appeared was computed. Participants performed 50 trials each of a 2-, 4-, and 6-item change-detection task prior to beginning the experiment. K estimates in this task ranged between 1.6 and 4.8 items (M = 3.4). All participants performed above chance and were included in all analyses. 

Figure 3

 

(A) Schematic of the stimulus arrays used in each condition in Experiment 2. (B) Results of the mixture model as a function of the distance between sample items. The distance between sample items had a significant effect on P _NT (center), while P _T and SD were unaffected. The dashed line corresponds to the significant linear contrast. Error bars denote within-subject SEM.

Table 1

 

Distances between stimuli for Experiment 2.

	Center-to-center distance		Total distance
	Minimum	Maximum	Minimum	Maximum
Separation 1	1.9°	7.5°	3.1°	8.7°
Separation 2	3.7°	10.6°	4.9°	11.8°
Separation 3	5.4°	10.8°	6.6°	12°
Separation 4	7.5°	10.8°	8.7°	12°

Results

Proportion of target responses

Examining the mean values obtained from the three-factor model revealed that increasing the separation between items had no effect on P _T, F(3,33) = 0.46, MSE = 0.01, p = 0.71. That is, consistent with the effects of Experiment 1, spatial competition had no effect on the number of correctly reported targets, indicating that memory maintenance and recall processes are largely independent of the distance between sample items. 

Proportion of non-target responses

The distance between memory items did have a significant effect on the number of non-target items reported, F(3,33) = 3.42, MSE = 0.006, p = 0.028. That is, consistent with the effects observed in Experiment 1, more non-target errors were made under high competition (i.e., when items were presented close together). Furthermore, the effect of separation on P _NT demonstrated a significant linear contrast, F(1,11) = 8.09, p = 0.016. Thus, these results extend the findings of Experiment 1 by demonstrating that the effect of competition on the number of incorrectly reported non-targets is linearly proportional to the distance between items. 

Target error (SD)

The results of Experiment 1 revealed that target error was unaffected by the distance between sample items, suggesting that the increase in non-target errors under high competition were not the result of an increase in error in the memory representation. Similarly, examining the error of target representations in Experiment 2 revealed that there was no effect of competition on the SD of target responses, F(3,33) = 0.94, SEM = 0.02, p = 0.43. Thus, although the proportion of non-target errors was directly related to the distance between sample items, targets were reported with similar precision independent of the amount of spatial competition between them. 

Proportion of guesses

Finally, examining the effect of competition on P _G indicated that the proportion of guesses remained unaffected by the separation between memory items, F(3,33) = 0.4, MSE = 0.02, p = 0.76, just as in Experiment 1. 

Discussion

The results of the current experiment are consistent with the prediction that since binding errors in attention and perception increase when items are presented close together in space (Cohen & Ivry, 1989), binding errors in VWM should also be related to the distance between sample items during encoding. Indeed, we found that non-target responses were linearly related to the spatial separation between items. Thus, the results provide further evidence that the effects of competition on P _NT are likely to occur before VWM maintenance, impacting performance at the initial perception and representation stage of the to-be-remembered objects. Although spatial competition did have a significant effect on P _NT, the distance between objects had no effect on the SD of the normal distribution of reported targets obtained in the mixture model. That is, when targets were reported, they were reported with a similar amount of precision, regardless of the distance (and, therefore, the competition) between items. Consistent with the results of Experiment 1, we observed a dissociation between the amount of error of maintained memory representations (SD) and the proportion of spatial errors (P _NT). Together with the findings of Experiment 1, the results suggest that competition affects how information is encoded into VWM but not the fidelity or precision of those items that are maintained and recalled from VWM. In contrast, increasing the number of to-be-remembered items affects both the fidelity of VWM representations and the proportion of non-target responses. Thus, the increase in non-target responses with increasing set size is potentially an incidental cost that occurs from the greater competition for representation during VWM encoding. This possibility will be discussed in more detail in the General discussion section. 

There is, however, a potential alternative explanation to the present results. Specifically, it is possible that the configurations used in the present experiment led to a linear increase in the number of visual hemifields or quadrants over which sample items were presented. Previous studies have demonstrated that information in VWM may be represented by distinct resources in the left and right hemispheres (Buschman, Siegel, Roy, & Miller, 2011; Delvenne, 2005; Umemoto, Drew, Ester, & Awh, 2010). Moreover, information presented in the same hemifield or quadrant is subject to more competition than information presented in distinct hemispheres (Alvarez & Cavanagh, 2005; Scalf & Beck, 2010) or quadrants (Carlson, Alvarez, & Cavanagh, 2007). Monte Carlo simulation (see Methods section) indicated that although the average number of hemifields over which the items were presented increased from Separation 1 to Separation 3 (1.3, 1.7, and 2.0, respectively), items could only be presented in two hemifields in both the Separation 3 and Separation 4 conditions. Thus, although items were always distributed across resources in both hemispheres in these conditions, the proportion of binding errors continued to decrease, suggesting that the proportion of non-target responses is independent of the allocation of hemispheric resources. In contrast, the average number of quadrants in which sample items were presented increased roughly linearly with separation condition, with items presented in 1.8, 2.3, 2.9, and 3.7 quadrants, respectively. Thus, while in the Separation 1 condition a quadrant with one sample item was very likely to contain at least one other item, in the Separation 4 condition items were often presented within independent quadrants. Consequently, it is possible that the proportion of binding errors is affected not only by the absolute distance between each of the items but by the competition for representation by independent resources within each visual quadrant. This interpretation, however, is still consistent with the conclusion that competition for representation during encoding increases the number of binding errors. Ultimately, whether items compete for representation within a visual quadrant (Carlson et al., 2007) or as a function of the distance between items (Bahcall & Kowler, 1999; Franconeri et al., 2007; Kastner et al., 2001; McCarley & Mounts, 2007, 2008; McCarley et al., 2004; Mounts & Gavett, 2004; Mounts, 2000; Mounts et al., 2007), the functional mechanism underlying these effects remains the same: Items compete for neural representation within spatially defined receptive fields. When items are presented close together in space, the amount of competition will increase, resulting in an increase in binding errors. Moreover, these effects are also likely to occur at early stages of sensory processing rather than during maintenance stages, as previous studies have demonstrated that competition for representation within a hemifield occurs during early sensory encoding (Buschman et al., 2011; see General discussion section). 

Experiment 3

The results of the first two experiments demonstrate that competition between sample items has a significant and spatially specific effect on non-target responses, revealing that these errors result from competition that occurs during early perceptual processing (i.e., while the items are still present in the display). It is possible, however, that these effects could still be related to an increase in error in the memory representation itself. That is, if spatial memory is subject to normally distributed error, then non-target responses could increase when items are presented close together, since more items will fall within the range of spatial uncertainty. Accordingly, in Experiment 3, we examined the effects of competition during encoding without changing the distance between sample items but instead by manipulating when items were presented. That is, early studies examining the effect of competition on selective attention demonstrated that processing multiple objects in sequential displays reduced the behavioral effects of competition relative to when those objects were presented simultaneously (Duncan, 1980). Moreover, neuroimaging studies have revealed that the neural responses to multiple stimuli are reduced during simultaneous presentation and increases when items are instead presented simultaneously (Kastner et al., 1998, 2001). Consistent with the effects on attention, a recent study has demonstrated that presenting memory stimuli in sequential displays results in an increase in VWM performance (Ihssen, Linden, & Shapiro, 2010). That is, change-detection performance is worse for two categorically different arrays when they are presented simultaneously and therefore compete for representation. The results of Experiments 1 and 2, however, suggest that the effects of sequencing the memory arrays should be limited to non-target responses. That is, reducing competition for early representation by presenting arrays of sample items sequentially should result in a reduction in spatial errors, without affecting the precision of target representations successfully maintained and recalled from VWM. To test this prediction, Experiment 3 examines the proportion of target and non-target responses in sequential (SEQ) and simultaneous (SIMU) displays. 

Methods

Participants

Twenty right-handed undergraduates (5 males; ages: 18–22 years; M = 19.9) from the University of Toronto participated in the experiment for partial course credit. 

Stimuli and procedure

The procedure was similar to those of Experiments 1 and 2, while using a design based on those used by Ihssen et al. (2010). Each trial began with the presentation of a central fixation cross for 500 ms. Stimuli could then be presented in one of three conditions (Figure 4): In the SEQ condition, two sets of four colored squares were presented for 200 ms each. Critically, each of the sample displays was followed by an 800-ms delay. Thus, onset of the probe display occurred 2,000 ms after the onset of the first sample array; in the SIMU condition, all eight of the sample items were presented simultaneously in one 400-ms display. Extending the sample to 400 ms enabled the encoding duration to remain consistent between the SEQ and SIMU conditions, as encoding time is known to have a significant effect on VWM performance (Bays et al., 2009), although 400 ms is sufficient time to encode all eight items (50 ms/item; Vogel, Woodman, & Luck, 2006). Critically, the display could either be presented immediately after the initial fixation cross, with the delay lasting 1,600 ms, or the sample was presented after an additional 800 ms, during which only the fixation was present, and immediately followed by a 800-ms delay period. Thus, half of the SEQ trials had a delay period consistent with the first display of the SIMU condition, with the delay period of the other trials consistent with the second SIMU display; finally, in the REP condition, all eight sample items were presented twice, in two 200-ms sample displays presented with the same timing as the SEQ trials. Participants performed 65 trials of each of the condition (SEQ vs. SIMU) × temporal position (1st vs. 2nd displays) combinations and REP condition. 

Figure 4

 

Schematic of the conditions used in Experiment 3.

The stimuli were presented with the same parameters as those in Experiments 1 and 2, with a few exceptions. In all conditions, sample items were presented in the four adjacent locations immediately above and below the central fixation cross. In the SEQ condition, sample items were presented in groups of four items (above and below the fixation). The order of presentation was selected randomly, such that either the top or bottom set of items could appear first. The target item was selected randomly from one of the four locations in either the first or second group (SEQ condition) or from any of the eight items (SIMU and REP conditions). 

Sample colors were selected from the same sample space as in previous experiments, with the requirement that items within each set of four items (above and below the fixation) had to be separated by at least 20 values (28.5° in the color space). No restrictions were made between the groups of items. During the probe display, participants were instructed to select the color that most closely matched that of the probed item and to guess a random color if they could not remember the color of the probed item. Participants also performed 50 trials each of a 2-, 4-, and 6-item change-detection task prior to beginning the experiment. K estimates in this task ranged between 2.2 and 4.8 items (M = 3.4). 

As in the previous experiments, responses were analyzed for the three factors of P _T, P _NT, and P _G using the method of Bays et al. (2009). Maximum likelihood estimates for each of the response types were analyzed by collapsing across temporal position and evaluated using repeated measures ANOVA. The effect of temporal order was also examined by performing a 2 (SIMU vs. SEQ) × 2 (first vs. second display) repeated measures ANOVA. That is, for the critical comparison of SIMU vs. SEQ, it is possible to determine whether changes in performance are limited to stimuli in the first or second display. 

To ensure that none of the subjects adopted a strategy of attending to only the first or only the second group of items, P _T and P _NT, as well as the difference scores between the first and second displays for both conditions, were subjected to analysis for outliers. This analysis resulted in one subject's data being removed for preferentially reporting targets in the first display of the sequential condition (P _T = 0.6, M = 0.22, SD = 0.14) and another for demonstrating a difference in target report exceeding 53% (M = −0.20, SD = 0.16). Another subject was removed from further analyses for making more than 74% non-target responses in the second display of the SEQ trials (M = 0.27, SD = 0.17), reducing the total number of subjects to 17. 

Results

Proportion of target responses

To examine the effect of the different presentation conditions, responses were collapsed across the temporal positions (first and second displays). This analysis revealed no significant effect of presentation condition on P _T, F(2,32) = 0.506, MSE = 0.009, p = 0.61. Thus, overall, whether items were presented sequentially or simultaneously or even repeatedly had little effect on the probability of correctly recalling a target item (Figure 5). 

Figure 5

 

Results of the mixture model in Experiment 3. (A) Values averaged over both displays. (B) Values separated for the first and second displays in the SIMU and SEQ conditions. The number of non-target responses (P _NT) was reduced overall in the SEQ condition relative to the SIMU condition. Although this effect was larger overall in the second display, the interaction was not significant. Error bars denote within-subject SEM.

Examining the effects of temporal positions, however, did reveal differences in the probability of correctly reporting a target. While the main effect of condition was not significant, F(1,16) = 2.93, MSE = 0.006, p = 0.106, P _T was significantly greater in the second display than in the first display, F(1,16) = 10.1, MSE = 0.013, p = 0.006. There was also a significant interaction between temporal position and display condition, F(1,16) = 16.78, MSE = 0.008, p = 0.001, suggesting that this effect was driven by the increase in P _T in the second display of the SEQ condition. This indicates that reducing competition between sample items alone does not result in an increase in VWM performance; otherwise, improvements would have been observed for both temporal positions. Moreover, the finding that this improvement is restricted to the second display indicates that it is likely the result of limited memory encoding and/or maintenance resources and recency effects (Jiang & Kumar, 2004; Jiang, Kumar, & Vickery, 2005; Kumar & Jiang, 2005). 

Proportion of non-target responses

If spatial errors are related to competition between representations during encoding rather than to memory error, then the proportion of non-target responses should be lower when there are fewer items competing for representation (i.e., when items are presented in sequential rather than simultaneous displays). Although the main effect of presentation condition was not significant when collapsing across temporal condition, F(2,32) = 1.725, MSE = 0.024, p = 0.194, examining the simple effects revealed that the main effect of presentation condition was significant, F(1,16) = 5.6, MSE = 0.029, p = 0.031. That is, comparing the SEQ and SIMU conditions directly revealed that the proportion of non-target errors was significantly reduced in the SEQ condition. Importantly, there was no significant effect of temporal position on P _NT, F(1,16) = 0.119, MSE = 0.047, p = 0.74. The interaction was also not significant, F(1,16) = 1.75, MSE = 0.049, p = 0.205, although the difference between the SEQ and SIMU conditions was greater in the second display. 

Target error (SD)

Examining the amount of error in target responses (SD) revealed no significant effect of presentation condition, F(2,32) = 0.85, MSE = 71.6, p = 0.438. Examining the simple effects also revealed no significant effect of presentation condition, F(1,16) = 2.9, MSE = 83.4, p = 0.108, as well as no effect of temporal position, F(1,16) = 0.396, MSE = 69.1, p = 0.538. The interaction between temporal position and presentation condition was also not significant, F(1,16) = 2.5, MSE = 73.6, p = 0.137. Thus, while presenting items in two separate displays had significant effects on the proportion of both target and non-target responses, the fidelity of VWM representations remained unaffected by this manipulation. 

Proportion of guesses (P_G)

Presentation condition had no significant effect on the proportion of guesses, F(2,32) = 0.517, MSE = 0.037, p = 0.60. Examining the SEQ and SIMU conditions alone revealed no significant effect of condition, F(1,16) = 1.62, MSE = 0.046, p = 0.22, and no effect of temporal position, F(1,16) = 1.28, MSE = 0.068, p = 0.275. No interaction was observed, F(1,16) = 0.131, MSE = 0.054, p = 0.722. 

Discussion

Consistent with the findings of Experiments 1 and 2, the results of Experiment 3 demonstrate that presenting sample items in sequential displays eliminated competition between memory items, resulting in a reduction in the number of spatial errors made during VWM recall. Moreover, reducing competition during encoding had no effect on the amount of error in target responses, consistent with the first two experiments. Importantly, the distance between sample items remained constant between the SEQ and SIMU conditions, suggesting that these effects are unrelated to error in spatial memory. Instead, these results provide further evidence that competition during encoding increases binding errors in VWM. 

Given the results of Experiments 1 and 2, some may argue that it is surprising to observe effects of competition between the items in the top and bottom displays, given that the distance between those items is rather great, and therefore, the amount of competition between them should be small. We argue here, however, that the results provide further support that competition for representation is a significant source of binding errors that occur during VWM encoding. That is, even though the amount of spatial competition is low, the mere presence of additional items during encoding is enough to increase the number of binding errors; simply presenting items in different displays removes that source of competition, resulting in more accurate VWM encoding. 

Interestingly, the proportion of target responses was also found to increase in the SEQ condition but only for the second display. Although this improvement in memory performance is consistent with the effects observed by Ihssen et al. (2010), this finding is in contrast to those of Experiments 1 and 2, in which no improvement in P _T was observed when competition was reduced. It is unlikely, however, that reducing competition during encoding resulted in an increase in the ability to store and maintain representations in VWM. Specifically, as noted above, no significant increase in target SD was observed in the SEQ condition, indicating that memory items could not be stored with significantly greater fidelity when presented sequentially. 

Although the results of the present experiment are in line with those of a previous study that demonstrated an increase in item confusion errors with simultaneous as opposed to sequential displays (Frick, 1985), they are inconsistent with findings from another recent study (Gorgoraptis, Catalao, Bays, & Husain, 2011). In that study, the authors presented an array of up to six memory items in either one simultaneous display or in a sequence of displays, each with a single item. Using that design, the authors found that memory performance was greater when sample items were presented in simultaneous displays, demonstrated both by an increase in the proportion of correct target responses and a decrease in the proportion of non-target responses. How can the discrepancy between these two studies be explained? One possibility is that the effects of the study by Gorgoraptis et al. (2011) appear to have been driven primarily by the difficulty in updating longer sequences, as non-target responses were highest for the longest arrays and for the most distal item. In other words, resources were predominantly allocated to the encoding of each new item, resulting in a decrease in resources allocated to each previous item. Thus, the observed decrease in non-target responses in the sequential arrays (as well as the increase in target accuracy) could represent an increase bias to report more recent items and, therefore, represent an error in recall rather than a true binding error. Importantly, that study also observed no overall effect of sequencing the array on the amount of error of target responses; more recent items were reported with much less error, however, again suggesting a preferential allocation of resources to more recent items in the array. Thus, these findings suggest an important role for the allocation of attentional resources in establishing the contents of VWM. In other words, when attentional selection is equal, competing representations vie for VWM resources, with features occasionally being misbound during encoding; when selection and encoding resources are predominantly allocated to a single item, however, it will be encoded more readily, as well as potentially incorrectly recalled after other items have been displaced from memory. Further research is required, however, to fully reconcile these conflicting findings. 

General discussion

In three experiments, we examined whether the proportion of non-target responses (P _NT) is affected by the configuration of sample stimuli during encoding. In Experiment 1, non-target responses were greater for items that were presented close together in space than those that were presented further apart. In Experiment 2, this effect was demonstrated to be spatially specific, as non-target items increased as a function of the distance between sample items. Experiment 3 revealed that when items do not compete for representation during encoding (i.e., by presenting them in sequential displays) the proportion of non-target responses decreases. Accordingly, we conclude from these findings that competition for neural representation during encoding stages can produce binding errors in VWM. 

Competition and binding errors

Consistent with our conclusions, recent neurophysiological evidence suggests that competition in VWM is strongest during early perceptual representation suggesting that competition for encoding resources limits VWM performance for multiple items (Buschman et al., 2011). Critically, the loss of neural information occurred almost immediately after processing the memory array. Thus, this study may indicate why increasing the encoding duration beyond 500 ms has no effect on the proportion of non-target responses (Bays, Gorgoraptis, Wee, Marshall, & Husain, 2011): Specifically, even with longer sample durations (800 ms), the amount of information about object identity coded by neurons is always limited by competing representations during sensory representation (Buschman et al., 2011). Moreover, even with cues as long as 1,200 ms, the number of locations that can be selected is affected by the density of the display (Franconeri et al., 2007). In other words, competition between representations may affect the ability to select and encode items even beyond the durations tested here. The presence of binding errors during very long sample durations (Bays, Gorgoraptis et al., 2011), however, may indicate that at least some of these errors may result from a loss of fidelity in the memory storage/maintenance processes (see below). 

The account that binding errors in VWM occur during encoding is in line with findings from the attention literature showing that illusory conjunctions increase when objects are presented close together in space (Cohen & Ivry, 1989). Thus, binding errors may reflect a general inability of the visual system to overcome the competition for representation that occurs when multiple objects fall within a single receptive field (Desimone & Duncan, 1995). Typically, selective attention will determine which representation wins out and will be propagated downstream into VWM (Desimone & Duncan, 1995; Kastner & Ungerleider, 2001). Consistent with this account, recent findings have suggested that VWM performance may ultimately be determined by the ability to selectively attend relevant information (Fukuda & Vogel, 2009). That is, attention may serve as the bias signal that mediates the efficient selection and encoding of information into VWM. In the absence of efficient selection mechanisms, however, the representation that wins out may not be the desired (target) information or may potentially be influenced by information from competing representations, resulting in binding errors. 

Binding errors in VWM

The finding that competition for representation during selection and encoding leads to binding errors in VWM is largely inconsistent with the argument that binding errors occur primarily during the maintenance or recall stages of VWM (Bays et al., 2009; Bays, Gorgoraptis et al., 2011; Gorgoraptis et al., 2011). That is, according to this proposal, as the number of items stored in VWM increases, the proportion of resources dedicated to each item will decrease. In the experiments presented here, however, we consistently found differences in binding errors without concurrent changes to the overall memory load. Thus, these findings indicate that at least some of the non-target responses observed during VWM recall occur from processes that are independent of VWM maintenance resources. Critically, our finding suggests that the increase in binding errors with increasing set size may reflect an incidental cost that results from the increase in competitive interactions that occurs at large set sizes. That is, increasing the number of items in a sample display results in an increase in the number of items competing for representation within a finite number of receptive fields, in addition to increasing load on VWM maintenance resources (Buschman et al., 2011). Consequently, non-target responses and the SD of target responses may reflect independent sources of error relating to the distinct effects of set size on encoding and maintenance processes, respectively. 

It is likely the case, however, that at least some non-target responses reflect error in the memory processes itself. For example, Fougnie et al. (2010) examined the binding of color and orientation features using the three-component model and demonstrated that the number of features reported from a non-target item increased when six features needed to be reported (color and orientation of three items) compared to when only one of the features was relevant. Thus, increasing the memory load by increasing only the number of relevant features can increase the number of incorrectly reported features, suggesting that a reduction in available resources (and a concomitant increase in memory error) could account for some of the non-target responses observed in VWM. 

Moreover, there are additional factors other than load and competition that can increase the proportion of non-target responses. Specifically, Gorgoraptis et al. (2011) found that the number of non-target responses increase when arrays are presented in a sequence of single items. While at first blush this finding contradicts the results of Experiment 3, the results are also inconsistent with a purely resource-based account. That is, although neither the memory load nor the amount of competition increased by presenting the array in a sequence, this manipulation significantly increased the number of non-target responses. Thus, this finding suggests that errors can increase from attentional manipulations; by strongly biasing the encoding and maintenance of some items over others, these items were much more likely to be recalled from memory, even when the response was incorrect. For example, in the same study, fewer non-target responses were exhibited when an item was cued, supporting the proposal that attentional biases can push around the relative weight of items in VWM. In other words, these non-target responses may not reflect binding errors per se but may instead reflect the relative weighting of information in a resource-limited system. 

Competition in VWM

The finding that competition for representation increases the number of non-target responses is consistent with a number of recent studies providing evidence for the idea that competition may limit VWM performance (Buschman et al., 2011; Ihssen et al., 2010). Moreover, previous studies have demonstrated that VWM performance decreases in the presence of irrelevant distractors (Fukuda & Vogel, 2009; McNab & Klingberg, 2008; Vogel, McCollough, & Machizawa, 2005). Although this finding has been taken as evidence of a poor selection mechanism (Fukuda & Vogel, 2009), it could similarly provide evidence that competition for representation in early visual processing affects the ability to accurately encode information in VWM. In other words, the presentation of irrelevant distractors increases the number of objects competing for representation by limited perceptual resources, as well as memory resources. Consequently, the observed decreases in performance in the presence of distractors could result from competition for early perceptual representation, even if those items are never encoded into VWM. 

Conclusion

Overall, our three experiments provide support for the notion that spatial competition among items can lead to binding errors at the encoding stage. Accordingly, these findings suggest a mechanism in which competition plays a critical role in limiting VWM performance. Thus, models of VWM that incorporate competition as a critical limiting factor of VMW performance (e.g., Shapiro & Miller, 2011) may be important for fully understanding the limited nature of information processing in the visual system. Moreover, such models may find parallels with other limitations in visual cognitive performance (e.g., Franconeri, Jonathan, & Scimeca, 2010), thereby helping to integrate our understanding of attention, perception, and memory and revealing how the resource-limited visual system manages to solve the problem of processing multiple objects. 

Acknowledgments

The authors would like to thank Steven Bostan for assistance running Experiment 2 and Jay Pratt, Morgan Barense, Martin Paré, Jeffery S. Johnson, Steve Franconeri, and two anonymous reviewers for useful feedback. This research was supported by a National Science and Engineering Research Council (NSERC) grant to S.F. 

Commercial relationships: none. 

Corresponding author: Stephen M. Emrich. 

Email: [e-mail]. 

Address: Department of Psychiatry, 1056 Wisconsin Psychiatric Institute, 6001 Research Park Blvd., Madison, WI 53719, USA.