Characterizing the nature of visual conscious access: The distinction between features and locations

Abstract

The difference between the roles of features and locations has been a central topic in the theoretical debates on visual attention. A recent theory proposed that momentary visual awareness is limited to one Boolean map, that is the linkage of one feature per dimension with a set of locations (L. Huang & H. Pashler, 2007). This theory predicts that: (a) access to the features of a set of objects is inefficient whereas access to their locations is efficient; (b) shuffling the locations of objects disrupts access to their features whereas shuffling the features of objects has little impact on access to their locations. Both of these predictions were confirmed in Experiments 1 and 2. Experiments 3 and 4 showed that this feature/location distinction remains when the task involves the detection of changes to old objects rather than the coding of new objects. Experiments 5 and 6 showed that, in a pre-specified set, one missing location can be readily detected, but detecting one missing color is difficult. Taken together, multiple locations seem to be accessed and represented together as a holistic pattern, but features have to be handled as separate labels, one at a time, and do not constitute a pattern in featural space.

Introduction

What can an observer perceive at one time? This is one of the most fundamental questions that needs to be addressed in studies on perception and attention, and yet it has not been very thoroughly addressed in the literature. The reason for this, as we illustrated in Huang and Pashler (2007, see also Huang, 2010), seems to be that the literature has often conflated the two concepts of selection and access. Therefore, the question above, which is indeed a question on the nature of conscious access, is often assumed to have already been answered and is therefore neglected, even though what has really been answered is the question of the mechanism of selection. 

Selection vs. access

The conceptual distinction between selection and access can be found in a very straightforward definition of visual attention. Early visual processes often provide a large amount of visual information, but there is a severe limit on the part of this information that can gain access to visual awareness. Therefore, some mechanism must make a selection regarding the parts of the information that will or will not reach visual awareness. In Boolean map theory, this severe limit is denoted as access, and so it is conceptually different from the mechanism of selection. Access is basically the same concept as the capacity limits of the processing/representational/attentional resource used in the literature. 

Duncan (1980a, 1980b) was the first to make the distinction between selection and access very clear. He reported that when observers attempt to detect one or more targets, adding extra distractors has little deteriorating effect on performance; however, performance is significantly better in tasks involving only one target compared to those involving more than one target (Duncan, 1980b). He suggested that perceiving targets requires access to a limited-capacity conscious system, whereas the visual system can effectively exclude distractors on the basis of an unconscious selection mechanism (Duncan, 1980b). The terms access and selection, as used in Boolean map theory, respectively, refer to access to the limited-capacity conscious system and the unconscious selection mechanism. 

It is also important to note that, as used in Boolean map theory, the terms conscious access and visual awareness mainly imply access consciousness and not phenomenal consciousness as defined by Block (1990, 1995, 2007). That is to say, when it can be reasonably ensured that information will not be lost at some other stage of cognitive processing, having access to some information is a sufficient and necessary condition for it to be reported with a behavioral or verbal response. This is certainly not to say that phenomenal consciousness is not important; we merely wanted to start with a relatively simple and tractable definition. 

An analogy may help to further clarify the distinction between selection and access: Suppose a worker needs to find a number of machine parts from storage and move all of them to a small work room. In this process, the efficiency of the worker is analogous to the efficiency of selection, whereas the size of the work room door is analogous to the limit of access. One can certainly imagine the situation in which the parts found by the worker cannot be moved into the room quickly enough, and so they pile up outside the room; analogously, relevant information can be easily selected, but not all of it can gain access to consciousness (e.g., the color task in the present study). On the other hand, there could also be a situation in which the worker cannot find the parts quickly enough, and so the worker does not have enough parts to take through the door; analogously, relevant information is very difficult to find, but, once found, gaining conscious access to this information is easy (e.g., a very difficult visual search). 

The majority of the empirical studies on visual attention (e.g., those using the visual search paradigm; see Treisman & Gelade, 1980; Wolfe, 1994) have actually tested the question of selection, not access. This is because, in these studies, the researchers were only usually interested in studying the difficulty of finding a target (i.e., excluding the distractors) and in measuring how this difficulty varies under different conditions of experimental manipulation; however, gaining access to a target was usually made rather easy in all different conditions and was therefore of no particular importance to the purpose of the studies. As a result, although remarkable progress on the principle of selection has been made in recent decades, relatively little is known about access. As mentioned above, Duncan (1980a, 1980b) showed that there is a severe capacity limit to access. However, the further question regarding the nature of this limit, namely how this limit is defined, had been largely neglected in the literature until our recent studies on Boolean map theory. 

Boolean map

In our Boolean map theory, we tried to address the nature of access through the notion of a data structure (i.e., the Boolean map). We proposed that momentary visual awareness is limited to one Boolean map, that is the linkage of one feature per dimension with a set of locations (Huang, 2010; Huang & Pashler, 2007; Huang, Treisman, & Pashler, 2007). As illustrated in Figure 1, Boolean map theory has clear and direct implications for access to features and locations. For stimuli such as the set of colored balls in Figure 1a, the multiple colors have to be accessed one at a time because a Boolean map—the format of momentary visual awareness—can only contain one featural label at a time (Figure 1b). Multiple locations, on the other hand, can be perceived simultaneously because a Boolean map can contain multiple locations (Figure 1c). 

Figure 1

 

Pattern analysis of colors and locations from Boolean map theory. (a) Stimuli: A pattern of colored balls. (b) Analysis of colors: The multiple colors have to be accessed one at a time because one Boolean map—the format of momentary visual awareness—can only contain one featural label at a time. (c) Analysis of locations: The multiple locations, on the other hand, can be perceived simultaneously because a Boolean map can contain multiple locations.

A pattern analysis task

In the present study, I wanted to study the research question of access rather than the mechanism of selection. This gave me direction with regard to which paradigm I should use. As mentioned above, a visual search is well suited for studying the mechanism of selection because the main difficulty in the task is to select the target (i.e., to exclude the distractors). Here, I needed a task that is, in some sense, the opposite of a visual search (i.e., one in which the main manipulation is on access and the selection is always easy). In the present study, I presented a set of items and made all of them relevant to the task; therefore, the selection of relevant information was rather easy and the main difficulty in the task was gaining access to the information regarding these items, which can be manipulated systematically. A perceptual task using this type of stimuli is commonly called a pattern analysis task. 

In addition, I wanted to test the predictions from the specific data format of the Boolean map, which, as mentioned above, mainly concerns access to locations and features. Therefore, the pattern in the present study consisted of a set of items that varied in terms of feature values (e.g., colors) as well as locations, and the predictions of the Boolean map theory were tested by asking the following question: Can the information in a pattern, or, to be more precise, the locations and features of its elements, be accessed efficiently or not? 

So far, our treatment of the concept of pattern is straightforward and structuralistic: the set of information contained in the elements of a pattern. Specifically, the term locations of a pattern refers to the sum of the locations of the pattern elements, and features of a pattern (e.g., the colors of a pattern) refers to the sum of the features of the pattern elements. Clearly, as has been argued for a century by Gestalt psychologists, when the elements constitute a whole, the information in the whole could be different from the sum of the information in the elements, so one may reasonably ask whether our conceptualization of pattern has neglected important aspects of the visual pattern analysis task. Nevertheless, the tasks employed in this study required the observers to make judgments at the level of the individual elements rather than to match a global template, and so it was reasonable to start by adopting a somewhat structuralist view and to regard the pattern as merely the sum of all the elements. Here, I will focus on whether or not the information about these elements can be accessed efficiently. A Gestalt perspective will be considered in the General discussion section. 

Previous studies on pattern analysis tasks and visual attention

Perceiving a pattern is a basic visual task that human observers perform again and again each day. The performance of such a task (i.e., how fast and how accurately we can do it) is naturally limited by visual attention. However, the role of attention in pattern analysis has not been explored very thoroughly, especially when compared to the role of attention in some other basic tasks, such as visual searches (for reviews, see Quinlan, 2003; Wolfe, 1998). 

The relationship between pattern analysis and visual attention has been explored from a few perspectives. One line of research has focused on the whole–part relationship mainly from a Gestalt perspective (e.g., Hoffman, 1980; Navon, 1977) and another on the role of attention in some classic tasks, such as symmetry perception (Huang & Pashler, 2002; Olivers & van der Helm, 1998). These studies have shed important light on the relationship between pattern analysis and visual attention. However, to the best of my knowledge, they have approached the question of characterizing the nature of conscious access only in relation to very specialized situations. In the present study, I attempt to address this question in a more general way, as illustrated above. 

Predictions from the Boolean map theory

The very different style of information access illustrated above would predict very different costs of adding extra items to a pattern analysis for feature tasks and location tasks. For feature tasks, adding an extra feature would add one new step of comparison and this would translate into a substantial increase in response time. In location tasks, multiple locations can be compared simultaneously, so adding an extra location should have little impact on response time. Therefore, Boolean map theory would predict a substantial set size effect on response time (i.e., slope) in a color task, but not in a location task. In Experiments 1 and 2, this prediction was tested in a pattern comparison task (judging whether or not two patterns were identical). As shown in Figure 2a, two panels, each containing several colored balls, were presented on the left and right sides of a display. In a location (color) task, the observers attempted to judge whether or not the locations (colors) in the two panels matched. As illustrated above, the slope should be much steeper in a color task than in a location task. 

Figure 2

 

Stimuli and results of Experiments 1 and 2. (a) Stimuli: In a color task (Experiment 1), the observers had to judge whether the two set of colors in the left and right panels matched; the locations could be either matched (two top-left displays) or shuffled (two top-right displays). In a location task (Experiment 2), the observers had to judge whether the two set of locations in the left and right panels matched; the colors could be either matched (two bottom-left displays) or shuffled (two bottom-right displays). (b) Results: First, the slope of color comparison (red empty line) was substantially greater than the slope of location comparison (blue empty line). Second, the effect of shuffling the other dimension was substantially greater on color comparison (difference between the two red lines) than on location comparison (difference between two blue lines). See text for details.

Moreover, Boolean map theory makes a further prediction about how features and locations depend on each other, namely that access to locations can proceed without noting features, through the creation of a Boolean map covering all of the items (e.g., Figure 1c). However, because of the map nature of access, it is obligatory that access to features is accompanied by information on the locations where these features reside (e.g., Figure 1b). In other words, an observer can see the locations of all of the items while having a poor knowledge of their colors but cannot see the colors without knowing where the items are. This predicted asymmetry was tested by adding two shuffled conditions to the pattern comparison tasks of Experiments 1 and 2 (i.e., the four displays on the right side of Figure 2a): locations could be shuffled in color tasks (two top-right displays); whereas colors could be shuffled in location tasks (two bottom-right displays). In these shuffled conditions, even for matched displays, there were massive mismatches in the task-irrelevant (i.e., shuffled) dimension; the observers were instructed to ignore these differences and only compare the task-relevant dimension. For example, in the two displays of the color task with the locations shuffled (two top-right displays), the locations of the balls are shuffled so that they are entirely mismatched; however, in the matched display, the left and right sides have the same five colors (green, black, yellow, red, and blue) and therefore the judgment should be “matched”, whereas in the mismatched display, only the left side has the color blue and only the right side has the color white and so the judgment should be “mismatched”. 

As predicted above, in the bottom four displays in Figure 2a, we can perceive and compare the locations of the patterns with little influence from the patterns' colors; therefore, the performance of the observers should be basically the same regardless of whether the colors correspond perfectly in the two panels (i.e., the two bottom-left displays) or are shuffled (i.e., the two bottom-right displays). In the top four displays in Figure 2a, accessing the colors should become significantly more difficult when the locations are shuffled (i.e., the two top-right displays) compared to when the locations correspond perfectly (i.e., the two top-left displays); this is presumably because it is obligatory that access to the colors is accompanied by the locations, and therefore, it would naturally be more difficult to access colors from mismatched locations than from matched locations. In brief, shuffling the locations should make a color task significantly more difficult, but shuffling the colors should have little impact on a location task. 

Experiments 1 and 2: Encoding colors and locations

Methods

Observers

The observers in this study were university undergraduate students, all of whom had normal or corrected-to-normal vision. There were, respectively, 9 and 10 observers in Experiments 1 and 2. 

Apparatus

Stimuli were presented on a 1,024 × 768 pixels CRT color monitor. The observers viewed the display from a distance of about 60 cm and entered responses using a keyboard. The program was written in Microsoft Visual Basic 6.0 and was run in Microsoft Windows XP using timing routines tested with the Blackbox Toolkit. 

Procedure

Each trial began with a small white fixation cross presented in the center of the screen for 400 ms. Following a short blank interval (400 ms), the stimuli display was presented, and it remained on the screen until a response was made. The observers made their judgment and responded by pressing one of two adjacent keys (pressing “j” key for “match” and “k” key for “mismatch”; tasks and standards of “match/mismatch” judgment are described in the stimuli section below with the fingers of their right hand. They were asked to respond as accurately and as quickly as possible (i.e., speeded response). A pleasant or unpleasant tone sounded to indicate whether or not the response was correct, and the next trial began 400 ms later. The observers completed 10 blocks (50 trials each), with the first block being regarded as practice and excluded from the analysis. 

Stimuli

Figure 2a shows eight displays of the stimuli displays. There were two square-shaped panels in each stimuli display, and a set of balls was presented within each panel. The panels were gray against a black background. Each panel measured 5.7 cm × 5.7 cm and their centers were 5.2 cm away from the center of the display (one on the left and one on the right). 

In each trial, the set size (i.e., the number of balls in each panel) was randomly determined to be N = 1, 3, 5, or 7. Then, N colors and locations were generated (see below for details) and randomly paired into N objects. In 50% of the trials in Experiment 1, the color of one ball in one panel was changed to a new color, and it was the observers' task to determine whether there was such a mismatched color. In 50% of the trials, a whole new set of locations was generated and applied to all of the balls in one panel (i.e., the locations were shuffled). The color change and the shuffling of locations were both randomly determined and were independent of each other (i.e., each of the 2 × 2 = 4 possibilities accounted for 25% of the trials). In Experiment 2, the location of one of the balls in one panel was changed in 50% of the trials, and it was the observers' task to determine whether there was such a mismatched location. In 50% of the trials, a whole new set of colors was generated and applied onto the balls in one panel (i.e., the colors were shuffled). The location change and the shuffling of colors were both randomly determined and were independent of each other (i.e., each of the 2 × 2 = 4 possibilities accounted for 25% of the trials). 

To generate a new set of locations, the locations of the balls were randomly chosen from a 10 × 10 invisible grid with the constraint that no balls would be next to each other horizontally or vertically, although diagonal adjacency was allowed. Each unit of the grid measured 0.52 cm × 0.52 cm, and so the whole grid measured 5.2 cm × 5.2 cm, just fitting into the panel with some margin. To generate a new set of colors, the colors of the balls were randomly chosen from eight possible colors (red, green, blue, yellow, purple, cyan, black, and white) with the constraint that balls in the same panel would never have the same colors. 

Results and discussion

The results of Experiments 1 and 2 are presented in Figure 2b. The results for the trials with matching panels and those with mismatched panels were very similar, and so they have been averaged together to simplify the presentation of the data. First, the slope of color comparison was substantially greater than the slope of location comparison (275 ms/item, SD = 21 ms/item vs. 79 ms/item, SD = 8 ms/item; F(1, 17) = 84.19, p < 0.001), thus confirming the first prediction. In Figure 2b, this is shown by the fact that the slope of the red empty line is substantially greater than the slope of the blue empty line. Second, the effect of shuffling the other dimension (i.e., the difference between the slopes when the other dimension was and was not shuffled) was substantially greater for color comparison than for location comparison (181 ms/item, SD = 46 ms/item vs. 13 ms/item, SD = 5 ms/item; F(1, 17) = 14.39, p < 0.002), which confirmed the second prediction. In Figure 2b, this is shown by the fact that the difference between the two red lines is substantially greater than the difference between the two blue lines. 

One may challenge the present results by saying that perhaps the discriminability of the specific set of colors and locations were not comparable; in other words, the colors were similar to each other and so they had to be processed one at a time, whereas the locations were fairly different from each other and so they could be processed simultaneously. However, this is unlikely given that the response times were roughly equal when comparing one pair of balls (i.e., set size = 1), which indicates that the colors were approximately as discriminable as the locations. The difficulty is not in the comparison of color per se, but in the comparison of multiple colors simultaneously. 

Experiments 3 and 4: Change detection of colors and locations

The results of Experiments 1 and 2 were consistent with the predictions from the statement that access to colors is serial but access to locations is parallel. These results, however, only demonstrated this in the situation of coding features from new objects; therefore, a further important question is whether this same principle also applies to the monitoring of feature changes on old objects. To be more precise, Experiments 1 and 2 showed that when new objects appear, observers can only become aware of their features by visiting them one at a time. However, it remains unclear what will happen if the observers have perceived all the objects and then a feature of one object changes (e.g., from green to blue) in front of their eyes. If feature access is serial even for an old object, then the observer will not perceive that change until this object is visited again. Alternatively, it is possible that new featural changes on old objects will be automatically updated and so such a change could be perceived immediately. 

There was one practical concern if I wanted to test feature access on old objects. A change in feature would cause a motion (or flicker) signal in the visual system that, when it appears suddenly, would automatically attract attention (e.g., Abrams & Christ, 2003); therefore, the results would not reflect the question of feature access that I intended to test. To avoid this motion signal while still retaining the object structure, I used the classic approach of occlusion. When objects are briefly covered by a sliding occluder (∼200 ms), changes before and after occlusion do not cause a motion signal, but there is clear evidence that object structure largely survives through occlusion at this temporal scale (e.g., Scholl & Pylyshyn, 1999). I specifically employed stimuli that were similar to those in Experiments 1 and 2, but now only one single panel was presented in the center of the display. This panel was periodically covered by an occluder that moved up and down repeatedly in front of it, and the panel switched to the other frame (out of a total of 2 frames) each time it reappeared from behind the occluder. The observers attempted to judge whether or not the colors (or locations) in the two frames matched. In other words, the two panels in Experiments 1 and 2 were separated and presented alternately in different frames. 

This method is very similar to some previous studies on feature encoding (e.g., Saiki, 2003) and change blindness (e.g., Rensink, 2002). These studies have generally found that observers are rather poor in detecting feature changes. This finding is consistent with the notion that feature access is serial even for old objects, and thus an observer cannot perceive a change on an old object until that object is visited again. In Experiments 3 and 4, I tested this question in relation to the present purpose by comparing color and location tasks. If feature access is serial even for old objects, then the results should be very similar to those of Experiments 1 and 2 (i.e., a greater slope for a feature task than for a location task); otherwise, the gap between the slopes of feature and location tasks should be considerably reduced. 

Methods

The methods of Experiments 3 and 4 were identical to those of Experiments 1 and 2, respectively, with the following exceptions. In both Experiments 3 and 4, there were 14 observers and the shuffled conditions were removed. The two panels were not presented simultaneously on the left and right sides of the display as in Experiments 1 and 2; instead, they were now presented as two alternating frames of one single panel in the center of the display. In each trial, a 6.2 cm × 6.2 cm white occluder was presented, and this moved up and down periodically in front of the stimuli panel. This occluder repeated a loop consisting of four steps: staying stationary below the center (6.5 cm away) for 420 ms, moving upward at a speed of 39 cm/s for 330 ms, staying stationary above the center (6.5 cm away) for 420 ms, and moving downward at a speed of 39 cm/s for 330 ms. The whole loop took about 1,500 ms. As the occluder passed over them, the balls in the panel were temporarily covered (for about 160 ms), and the panel switched to the other frame (out of a total of 2 frames) when it reappeared from behind the occluder. In other words, the panel presented one of the two frames when the occluder was above the center and the other frame when the occluder was below the center. Both the generation of stimuli and the task in Experiments 3 and 4 were the same as in Experiments 1 and 2, respectively, but now the comparison was performed across the frames. In other words, in 50% of the trials in Experiment 3 (Experiment 4), the color (location) of one ball in one frame was changed to a new color (a new location), and it was the observers' task to determine whether there was such a mismatched color (mismatched location). The procedure is illustrated in Figure 3a and is also shown by Movies 1 (Experiment 3, color task) and 2 (Experiment 4, location task). 

Figure 3

 

Stimuli and results of Experiments 3 and 4. (a) Stimuli: An occluder moved up and down periodically in front of the stimuli panel. As the occluder moved over them, the balls in the panel were temporarily covered, and the panel switched to the other frame (out of a total of 2 frames) when it reappeared from behind the occluder. In a color task (Experiment 3), the observers had to judge whether the two set of colors in the two frames matched. In a location task (Experiment 4), the observers had to judge whether the two set of locations in the two frames matched. (b) Results: The slope of color comparison (red line) was substantially greater than the slope of location comparison (blue line). See text for details.

 

Movie 1

 

The stimuli of Experiment 3. An occluder moved up and down periodically in front of the stimuli panel. As the occluder moved over them, the balls in the panel were temporarily covered. When the panel reappeared from behind the occluder, it had switched to the other frame (out of a total of 2 frames). In Experiment 3, one ball could change color between the two frames.

 

Movie 2

 

The stimuli of Experiment 4. An occluder moved up and down periodically in front of the stimuli panel. As the occluder moved over them, the balls in the panel were temporarily covered. When the panel reappeared from behind the occluder, it had switched to the other frame (out of a total of 2 frames). In Experiment 4, one ball could change its location between the two frames.

Results and discussion

The results of Experiments 3 and 4 are presented in Figure 3b. The results for trials with matched panels and trials with mismatched panels were very similar, and so they have been averaged together to simplify the presentation of the data. The results of Experiments 3 and 4 showed that the differences between the color and the location task were basically the same as those revealed in Experiments 1 and 2. As in Experiments 1 and 2, the slope of color comparison was substantially greater than the slope of location comparison (72 ms/item, SD = 14 ms/item vs. 18 ms/item, SD = 4 ms/item; F(1, 26) = 12.88, p < 0.002). These slopes were generally much reduced compared to those of Experiments 1 and 2, probably because the stimuli were now presented in the fovea rather than the periphery and did not have to be compared across different parts of the display. Nevertheless, the gap between the color task and the location task, as measured by the ratio between their slopes, was not reduced from Experiments 1–2 (3.48 times) to Experiments 3–4 (3.92 times). Clearly, observers not only have to visit features one at a time for initial coding but also have to visit them one at a time to keep updated. 

Experiments 5 and 6: Detecting a missing color or location

Other distinctive predictions can be made from the Boolean map theory for tasks involving the detection of a missing feature or location: Locations can be simultaneously accessed, and therefore, it is easy to spot a missing value from a pre-specified pattern; features, on the other hand, have to be accessed one at a time, and so, to determine a missing value from a pre-specified pattern, the only strategy is to check and exclude all of the presented featural values until there is only one left. As an analogy to understand this strategy of detecting a missing feature, imagine that an observer has to determine a missing location, but the locations are only presented one at a time: the observer cannot determine the missing location until all of the locations have been presented and excluded. Intuitively, for the stimuli in Figure 4b, we can immediately see that one specific location is missing (i.e., on the left), but the fact that the color blue is missing is not immediately obvious, and we can only come to realize this after laboriously checking and excluding all of the colors present in the display. Experiments 5 and 6 tested these predictions. 

Figure 4

 

Stimuli and results of Experiments 5 and 6. (a) Stimuli of Experiment 5: One colored ball was briefly presented, and the observers had to determine, from a choice of six possible colors or locations, which one had been presented. (b) Stimuli of Experiment 6: Five colored balls were briefly presented, and the observers had to determine which one was missing from a choice of all six possible colors or locations. (c) Results of Experiment 5: The difference in the discriminability of the six colors and the six locations seems to reflect approximately a 2:1 ratio on exposure durations (i.e., the dotted blue curve approximately overlaps with the red curve). In other words, the discriminability in a location task was comparable to the discriminability in a color task in which the stimuli duration was twice as long. (d) Results of Experiment 6: The accuracy was dramatically higher in the location task than in the color task, even after compensating for discriminability (i.e., the dotted blue line is higher than the red line at 200 ms and above). See text for details.

Naturally, to compare the difficulty of detecting either a missing color or a missing location, I had to make sure that the colors and locations were roughly equally discriminable. Experiment 5 measured the discriminabilities of the color/location values by asking observers to detect the presence of one particular color/location value in brief displays. In Experiment 6, one item was missing from a hexagon-shaped pattern consisting of items of six primary colors (i.e., red, green, blue, yellow, cyan, and purple). The observers' task was to determine either which color (color task) or which location (location task) was missing. 

Methods

In Experiments 5 and 6, the basic aspects (e.g., observer selection, apparatus, fixation, and auditory feedback) of the methods were identical to those of Experiments 1–4 and are therefore not elaborated here. There were, respectively, 15 and 13 observers in Experiments 5 and 6. The stimuli displays were covered by masks after a certain period (i.e., the stimuli duration). The stimuli duration was randomly determined to be 25, 50, 100, or 200 ms in Experiment 5 and 25, 50, 100, 200, 400, or 800 ms in Experiment 6. The observers performed either a color or a location task (tasks described in the stimuli section below) and responded by clicking one of six buttons using a computer mouse. They were asked to respond as accurately as possible with no consideration for speed (i.e., an unspeeded response). The observers completed 12 blocks (75 trials each), with the first two blocks regarded as practice and excluded from the analysis. The blocks alternated between color and location tasks and the starting block was counterbalanced between observers. 

Stimuli

Examples of the stimuli displays, one each for Experiments 5 and 6, are given in Figures 4a and 4b. In each trial of Experiment 5, one color and one location were chosen from six choices (colors: red, green, blue, yellow, cyan, and purple; locations: the six corners of a virtual hexagon, each 2.08 cm away from the center) and were paired into one object, which was presented as the stimulus. The observers' task was to determine which color (or location) was presented and then to push one of six buttons, each of which corresponded to one color (or location). In each trial of Experiment 6, five colors and five locations were chosen from the same six choices as in Experiment 5 and these were paired into five objects, which were presented as the stimuli. The observers' task was to determine which color (or location) was missing from the six choices and then to push one of six buttons, each of which corresponded to one color (or location). 

Results and discussion

The results of Experiment 5 are shown in Figure 4c. The discriminability of locations was indeed higher than that of colors; for the same exposure durations, the accuracies were significantly higher in the location task than in the color task. This difference in discriminability seems to reflect approximately a 2:1 ratio on exposure durations, as is shown by the dotted curve in Figure 4c. In other words, the discriminability in a location task was comparable to the discriminability in a color task in which the stimuli duration was twice as long. This measurement allows us to compensate for the effect of discriminability when interpreting the results of Experiment 6. 

The results of Experiment 6 are shown in Figure 4d. The accuracies were dramatically higher in the location task than in the color task, even after compensating for discriminability (i.e., the dotted line). For the duration times of 200, 400, and 800 ms, the accuracy in the location task was substantially higher than that in the color task (p < 0.001 for all 3 duration times). This confirmed the above prediction that detecting a missing color is significantly more difficult than detecting a missing location. 

General discussion

In six experiments, the difficulty of gaining access to multiple colors was considerably greater than that of gaining access to multiple locations. These results are generally consistent with the prediction from the data format of the Boolean map, which claims that multiple locations can be accessed within a single map, but only one feature label per dimension can be accessed at one time. 

Gestalt perspective

As mentioned earlier, when studying pattern analysis, perhaps a more important perspective than the one I have considered so far (i.e., the efficiency of information processing on the level of the elements) is the Gestalt view (i.e., the whole is different from the sum of the individual elements). Nevertheless, an important distinction needs to be clarified. The literature has shown that there is global information that only emerges from the whole and does not exist in the individual elements. Examples of this include a large “H” made out of small “B”s (e.g., Navon, 1977) or the faces made out of fruits in the paintings of Renaissance artist Giuseppe Arcimboldo: the information about “H” is only obtained from the spatial arrangement of the small “B”s and is not decomposed into the individual “B”s; similarly, the information about the faces is only obtained from the spatial arrangement of the fruits and is not decomposed into the separate fruits. All of the tasks in the present study, however, test information on the level of the elements, so, from the Gestalt perspective, the present study is not about the holistic pattern per se, but rather about how the holistic pattern affects the perception of elements. 

Having clarified this, it seems that the formation of a holistic pattern dramatically facilitates the comparison of two sets of locations, specifically by allowing them to be compared simultaneously. This is in clear contrast to the fate of features, which do not constitute any holistic pattern and so have to be compared one at a time; this equates to the Boolean map-based interpretation. As stated in Huang and Pashler (2007, p. 609), multiple locations are represented together and become a holistic pattern via the mechanism of a Boolean map. The fact that only multiple locations, and not multiple features, constitute one holistic pattern is attributed to the single-feature-multiple-location format of the Boolean map. 

What is the nature of a holistic pattern that allows parallel access to multiple locations? It seems to be a space, more specifically an (x, y) plane, that the perceptual system uses to mark locations: marking (−1, −1) indicates that this location is occupied; marking (−1, −1) and (−1, 1) indicates that both of these locations are occupied. The critical point of this representation is that the presence of two locations can now be labeled as a single concept. To compare this with the case of feature, a similar mechanism would be a featural space. For example, in a color space used by the perceptual system to mark the presence of colors, marking “red” would indicate that the color red is present somewhere in the display, and marking both “red” and “green” would indicate the presence of both of these colors somewhere in the display. Obviously, this type of color space mechanism does not exist. Three locations, when presented simultaneously, can be perceived as three corners of a triangle, but three colors (e.g., yellow, green, and blue) can never be perceived as a triangle in color space—conceptually, there is simply no such thing. Colors, and features in general, can only be accessed one at a time because they themselves are accessed as concepts (i.e., labels) rather than as values that can be checked (or unchecked) in a space. The presence of an (x, y) plane is probably related to the very common retinotopical organization in the neural system, although it should also be mentioned that there is evidence to support the idea that the cortical areas primarily involved in color vision are also organized as color space (e.g., Zeki, 1980; see also Conway & Tsao, 2009; Kotake, Morimoto, Okazaki, Fujita, & Tamura, 2009). Therefore, the distinction between the presence of a location map and the absence of a color space is not self-evident from the neural mechanism, and there is plausibly a functional reason for this feature/location distinction (e.g., a strategy to deal with the computational load associated with the binding problem). 

Here, I am suggesting a fundamental dichotomy between the presence of an (x, y) plane and the absence of a color space. Is it possible that the colors I chose just happened not to form a “good configuration” in color space? For example, in Experiment 6, if the hue, saturation, and brightness of the colors are carefully adjusted so that they form a perfect hexagon in color space, would that then make the absence of one of them obvious? ¹ Intuitively, this seems to be unlikely, although further studies will be needed to definitely answer this question. 

The nature of the holistic pattern having been clarified, the presence of an (x, y) plane, and the absence of a color space are intuitively quite obvious, although it is not clear how previous theories on visual attention in any way entail this feature/location distinction. Indeed, despite its obviousness, this feature/location distinction seems to be the opposite of what would be predicted from the dominant theory in this field, namely the feature integration theory (Treisman & Gelade, 1980), as will be discussed in detail below. Next, I will consider previous theories that are potentially relevant to the issue of visual conscious access and also possible alternative theories. 

Previous studies on features vs. locations

The present study focuses on the distinction between visual access to features and locations. One relevant research question that has been addressed very thoroughly in the literature is the relation between detecting the presence of a feature and localizing it. However, with regard to the relation between the processing of features and locations, the knowledge accumulated in these studies has limited direct implications for the interpretation of the present experiments for two reasons. First, in these previous studies, the target location was defined by, and had to be reached through, the specific target feature (i.e., localizing a feature), and so that decided the inherent relation between them. However, in the present experiments, the locations that needed to be perceived were merely the locations of all presented objects and the observers did not have to select locations based on certain features. Therefore, conceptually the previous studies and the present experiments addressed two totally different research questions. Indeed, below I will argue that this conceptual difference is likely to explain why the present conclusion seems to conflict with the previous one. Second, in these previous studies, observers usually detected only one feature and its location, which sheds little light on the difficulty involved in accessing multiple features or locations. 

Nevertheless, given that these are the best developed studies on the relation between features/locations, the different types of theories that have been developed in these studies should be briefly reviewed so that the different types of relations between features and locations can be applied to the question of access to multiple features/locations, thus allowing us to see what the alternative theories are. 

Recently, Busey and Palmer (2008) provided a very clear overview on the relation between detecting the presence of a feature and localizing it. They divided the theories into three categories: (1) Privileged detection over localization. A typical example of this type of theory is feature integration theory (Treisman & Gelade, 1980). Based mainly on the finding that observers can report the presence of a feature but cannot report where it is, this type of theory suggests that detecting a feature is easier than localizing it and also occurs at an earlier stage. Feature integration theory is a grand theoretical framework that goes far beyond this specific question, and so it is elaborated in detail below; (2) Privileged localization over detection. A typical example of this type of theory was proposed by Sagi and Julesz (1985). This type of theory suggests that localizing a feature is easier than detecting it and also occurs at an earlier stage. Sagi and Julesz showed that the locations of horizontal and vertical items (against a background of diagonal items) can all be perceived in parallel, although the exact feature values (i.e., horizontal vs. vertical) have to be determined serially. They concluded that the locations of objects are processed in parallel, but their features are processed serially; (3) Similar processing between detection and localization. This type of theory suggests that detecting the presence of a feature and localizing it are similar. A typical example of this type of theory was proposed by Johnston and Pashler (1990), who found that when observers were asked to report on the feature and location of a target, they could always report its location if they could report its feature and that they could almost always report its feature if they could report its location. Johnston and Pashler's conclusion was that detecting the presence of a feature and localizing it are implemented in basically the same way. 

How can the conflict between these different claims be resolved? Johnston and Pashler (1990) discussed a few important issues that help clarify matters. First, Treisman and Gelade's (1980) conclusion on the superiority of detection over localization may not be reliable because of the negative information problem: if observers see no target and they know that one target is harder to detect than the other, they could make better-than-chance guesses by always reporting the hard-to-detect target rather than the easy-to-detect target. Second, Sagi and Julesz's (1985) finding on the superior performance of localization over detection may not mean that localizing a feature is easier than detecting it because of the different properties problem: observers may simply be detecting “odd items”; they therefore localize these items and know that they are “odd” without necessarily knowing whether they are vertical or horizontal. Taking these problems into consideration, the similar processing of detection and localization most precisely reflects the true situation. Busey and Palmer (2008) carefully avoided all of these problems and modeled the accuracy slope of both detection and localization. They found that the results are very consistent with an independent channel model in which information is optimally used for either detection or localization. In other words, they also supported the similar processing of detection and localization. 

What are the implications of these studies for the present study? First, the similar processing of detection and localization is explicitly embraced by Boolean map theory in its claim that the nature of conscious access is a map-like data format that inherently represents location(s), and it is only through this map will the feature labels have conscious access. In Experiments 1 and 2 of this study, the predictions about how features and locations depend on each other were also based on this notion. 

Second, the present experiments supported, in some sense, a privileged access to locations over features. Therefore, on the surface, there appears to be a conflict between the present finding and the previous conclusion (i.e., a similar processing between localization and detection). Why is this so? The conceptual difference mentioned above offers one plausible explanation. In previous studies, the target location was defined by the specific target feature, and so that perhaps forced the encoding of the feature because the location had to be reached through it. In the present experiments, the locations did not have to be reached through features. So it is perhaps only natural that access to locations is now superior to features. To draw an analogy, it might be true that a programmer working in a big company generally recognizes the faces of many colleagues (analogous to locations) but knows the names (analogous to features) of far fewer colleagues. This superior knowledge of faces compared to names is analogous to the present results. However, when the programmer looks up a particular technician on a list by name and then meets the technician, it would be the case that the knowledge of the technician's name would always accompany the knowledge of the technician's face, which is analogous to the previous finding that localization and detection are similar. However, this is only because the face is reached through the name in this specific case and does not really conflict with the general superior knowledge about random colleagues' faces compared to knowledge about their names. 

Third, and mostly importantly, the three types of relations between features and locations could be applied to the present question (i.e., access to multiple features/locations) so that we can see what the alternative or potential theories are. 

Alternative theories

The first important alternative theory to consider is feature integration theory (Treisman & Gelade, 1980). This theory suggests that there are parallel and unlimited-capacity early visual processes that generate simple featural information. This set of information is represented by a number of separate feature maps, which are responsible for basic visual features such as color, size, and orientation. A decision that only requires information from one single feature map (e.g., is there a red target?) can be completed by a pooled response generated from one feature map. This pooled response loses the location information, and so the observer can report the presence of the red target but does not know where it is. There is also a “master location map” that marks the locations of all of the objects, but, without attention, the features float on these locations rather randomly and are generally not at their correct locations. The attention given to a particular location will “bind” the features of that location together correctly so that an observer can correctly perceive the different features of an object and also its location. 

Since its creation, some aspects of feature integration theory have been seriously criticized (see Quinlan, 2003, for a review). Nevertheless, Treisman's fundamental insight that perceptual awareness has trouble maintaining multiple associations between multiple spatial and featural information (i.e., the binding problem) and the way she approached the binding problem (i.e., by conceptualizing it as a location map and a number of feature maps) has stood the test of time, substantially shaped this field, established the foundations for basically all later theories, and inspired thousands of follow-up studies (Quinlan, 2003). 

Feature integration theory, as I understand it, can be used to make predictions on the access to multiple features/locations. Feature maps seem to correspond fairly well with the mechanism of color space mentioned above. In feature integration theory, the features in a feature map are processed in parallel and are simultaneously available, so one could reasonably assume simultaneous access to these multiple features. The present study clearly suggests that such a mechanism of color space does not exist. This theoretical conflict shows that, even if color space's absence is rather obvious once pointed out, this absence still has important implications for theoretical developments in this field. 

Why then, one would naturally ask, would such a generally very insightful theory conceptualize the conscious access of features in a way that contradicts a rather obvious fact? I suspect that it is because the original conception of feature integration theory was mainly motivated by selection tasks (e.g., a visual search). The statement that features are simultaneously available was only intended for the mechanism of selection, but, because the theory did not correctly make the distinction between selection and access, this statement has been inappropriately generalized to access, and this therefore led to the discrepancy. In other words, the ability to select any feature from all of the presented features (i.e., selection) is different from the ability to gain access to multiple features (i.e., access). Observers have the former ability, but not the latter. As an analogy, suppose a student goes to a computer store to purchase a computer: All brands of the computers in this store are on sale for 1,000 dollars and this student has exactly this amount of money. Therefore, analogous to an observer's ability to select any feature, the student could freely choose any brand. On the other hand, analogous to an observer's difficulty to access multiple features, the student obviously could not purchase more than one computer. Thus, it is correct to say that all of the computers in the store (i.e., all of the features) are simultaneously available in the sense that they are all there for the student to freely choose from. However, it is not correct to say that all of the computers in the store (i.e., all of the features) are simultaneously available in the sense that the student could take all of them home. 

How does feature integration theory fit into the three categories mentioned above? The color space and the (x, y) plane were described above as the mechanisms of conscious access to, respectively, features and locations. Here, they will be used to conceptualize the different possible theories about access to features and locations. Using these terms, the three corresponding categories should, respectively, be as follows: (1) Privileged access to features over locations. This concretely implies that conscious access is equipped with the color space, but no (x, y) plane mechanisms. (2) Privileged access to locations over features. This concretely implies that conscious access is equipped with the (x, y) plane, but no color space mechanisms. (3) Similar access to locations and features. This concretely implies that conscious access is equipped with both an (x, y) plane and color space mechanisms. 

Although feature integration theory was placed in the first of these categories on the issue of detection/localization, it should now actually belong to the third category on the issue of access of features/locations (i.e., similar access to locations and features), because, unlike the situation of localizing a feature, simply perceiving all locations could be enabled by the master location map serving as the (x, y) plane mechanism. As mentioned above, my claim, which follows from the data format of Boolean map and is supported by the present findings, is that there is only an (x, y) plane and no color space mechanisms; thus, the claim belongs to the second category (i.e., privileged access to locations over features). With regard to the first category, I am not aware of any theory that holds this theoretical position, and intuitively, it is rather implausible. 

Relation with other Boolean map theory findings

As discussed at the beginning of this article, the Boolean map theory (Huang & Pashler, 2007) started by emphasizing the distinction between selection and access and then proposed that access can be characterized by the data format of a Boolean map. Huang and Pashler (2007) also proposed that selection, and indeed all top-down control in general, is only implemented by creation of a Boolean map. We have subsequently further tested various aspects of the theory. 

In terms of access, what are most directly relevant to the present study are the other tests on the distinction between features and locations. These tests include Experiments 3 and 4 in Huang and Pashler (2007) and the experiments in Huang et al. (2007). All of these experiments employed the successive/simultaneous comparison paradigm (Duncan, 1980a, 1980b; Shiffrin & Gardner, 1972). The rationale of this paradigm is given as follows: The two halves of stimuli items are shown to the observers (in masked brief displays) either simultaneously or successively. The observers' ability to perceive the stimuli will be superior in the successive rather than the simultaneous condition (i.e., a significant successive/simultaneous difference) if they have difficulty in gaining access to information from both halves at the same time. However, their ability to perceive the stimuli will be the same in both conditions (i.e., no successive/simultaneous difference) if they can gain access to information from both halves with no capacity limits. In these experiments, the successive/simultaneous differences were significant in the task of perceiving the colors of two items, whereas in the task of perceiving the locations of two items, the successive/simultaneous differences were approximately zero. Therefore, these experiments also gave support to the claim that an observer can have access to multiple locations at the same time but will have difficulty accessing multiple features at the same time. 

Huang (2010) compared the Boolean map and the concept of object side by side to see how well each of them could account for the data in a number of experiments. In Huang (2010), objecthood is operationally manipulated as lines connecting different items. This study produced a few findings. First, access to the information of two items is the same regardless of whether these belong to one object or to two separate objects. Second, same-object advantage for two features does not exist when these features belong to different parts of an object; it is only significant when these two features are different dimensions of one Boolean map. Third, cueing the to-be-reported feature among two possible features has no effect when these two features are different dimensions of one Boolean map but does have a significant effect when the two features belong to different parts of an object. Taken together, these results support the claim that the Boolean map, rather than the object, is the unit of access. This study tested another prediction from the data format of the Boolean map, and so it is fairly closely related to the present study. 

As mentioned above, Huang and Pashler (2007) proposed that all top-down control is implemented by creation of a Boolean map. This implies that, in the case of ambiguous structure, imposing a perceptual structure is implemented merely by allocating spatial attention in one way or another and not through any separate mechanism. Huang and Pashler (2007) further speculated that perhaps spatial attention forces perceptual structure to switch to a target structure by selecting a part that is meaningful in the target structure but not in the present structure, although this is not a claim that directly follows from the theory itself. Huang and Pashler (2009) refined this claim, replacing “meaningfulness” with the more precise term of “simplicity of interpretation” and suggested that the visual system chooses the interpretation that maximizes the simplicity of the attended regions. This is supported by the finding that the usual attention effect on a figure/ground assignment (Attention → Figure) can be reversed when “being the background” is a simpler perceptual interpretation. 

Huang and Pashler (2007) discussed the conceptual distinction between the selection of visual information (i.e., Boolean map) and the processing optimization (i.e., attentional advantage) on that location. Huang (in press) compared the time course of the Boolean map and of the processing optimization and confirmed that they (i.e., processing optimization and the Boolean map) can be empirically separated from each other. 

Visual access vs. other factors

All of the experiments in this study were conducted for the purpose of characterizing the limit of conscious access. The experiments involved tasks comparing two simultaneously presented patterns (Experiments 1 and 2), two patterns presented in two alternating frames (Experiments 3 and 4), and a presented pattern and a memorized template (Experiments 5 and 6). In all of these experiments, I tried to ensure that the main factor that would affect performance is the limit of conscious access. Nevertheless, given the diversity of the tasks, other potential factors that may have affected performance should also be considered. 

To clarify the potential factors involved, I will extend the worker analogy given previously. In this analogy, the efficiency of the worker is analogous to the efficiency of selection, whereas the size of the door of the workroom is analogous to the limit of access. In addition, the size of the room itself could be analogous to the capacity of visual short-term memory. One can certainly imagine the situation in which the parts completely fill the room; analogously, more relevant information can be easily accessed, but not all of it can be memorized (e.g., presenting 10 colors for 1 min and then testing observers on them). On the other hand, there could also be a situation in which the room cannot be fully utilized in a given time because the door is small; analogously, observers can still memorize more information but cannot access it (e.g., presenting two colors for 50 ms, masking them, and then testing observers on them). In both of these tasks (10 colors for 1 min; 2 colors for 50 ms), all of the colors are task-relevant so the selection mechanism is not critical. 

From this analogy, it is clear that access (i.e., the size of the door) is different from the limit of visual short-term memory itself (i.e., the size of the room). A question that naturally follows from this is: How is access, conceptualized as an attentional limit on perception, different from the “consolidation of visual short-term memory” (Potter, 1976), which is also a door for the visual short-term memory? My response to this question is that they are conceptually similar and empirically undistinguishable, and so they could well be the same thing. Conceptually, both concepts have adopted a two-stage model in which some mechanism excludes visual information, and the excluded information, which is either never perceived or perceived but not memorized, is not available for explicit report. Certainly, even given this similarity, these two possibilities (i.e., not perceived vs. perceived but not memorized) are still clearly different, but more importantly, it seems difficult to distinguish between them on an empirical basis. A classic demonstration of the consolidation of visual short-term memory (Potter, 1976) involved showing observers a rapid sequence of pictures; afterward, if the observers were asked about the presence of one specific picture, their accuracy was very low, but if they were given a verbal cue in advance (e.g., “rabbit” for a picture with a rabbit), their accuracy was substantially higher. Potter (1976) suggested that the superior accuracy in the “word preview” condition is evidence that the observers have seen all of the pictures, and so their difficulty when there is no verbal cue is caused by their inability to consolidate them into memory. Although this finding clearly has important implications, it is also open to the possibility of the existence of a temporal attention mechanism that can effectively exclude information without perceiving it. Likewise, the demonstration of the attentional limit on perception could also be explained by a memory-based interpretation: a large amount of information can be perceived from all of the items in a display, but only a small subset can be consolidated into visual short-term memory. Given the conceptual similarity and empirical indistinguishability between the attentional limit on perception and the consolidation of visual short-term memory, access in Boolean map theory is only used as a term on the basis of its function, without specifying whether the underlying mechanism is perceptual or memory-based. In other words, access is assumed to be equivalent to both the attentional limit on perception and the consolidation of visual short-term memory until, maybe, further empirical data are able to favor one of them or distinguish between them. 

I have discussed why the consolidation of visual short-term memory is not a concept that is meaningfully different from access. However, what about the other factors, namely selection and the capacity limit of visual short-term memory? It seems reasonable to assume that the mechanism of selection was not critical in any of the present experiments because the task always required the observers to check all of the elements rather than to select a small subset and exclude the others. As for the capacity limit of visual short-term memory (e.g., Pashler, 1988; Phillips, 1974), I shall discuss these experiments case by case. In Experiments 1 and 2, all of the elements were presented and were available for perception until a response was made, and so it seems reasonable to suggest that there was no need for the observers to rely on their memory. In Experiments 3 and 4, the elements were periodically covered by a moving occluder, and so visual short-term memory was indeed very important for these tasks. Nevertheless, this should not matter because Experiments 3 and 4 were designed to test whether observers automatically become aware of a changing feature on a old object; if they do, then they could focus on the changing item and verify the change without needing to memorize any other items. The fact that they needed to rely on memory to perform the tasks already suggests that they were not automatically aware of the new feature on an old object. In Experiment 6, the observers probably did indeed need to memorize the presented colors and to use their memory to figure out which color was missing. Nevertheless, this also should not matter, because if they could have spotted the missing color without checking the presented colors, as they did in the case of locations, then they only needed to remember one missing color. The very fact that they needed to memorize all of the presented colors to do the task suggests their lack of a color space mechanism. 

Serial access vs. limited-capacity parallel access

The Boolean map theory states that only one feature label per dimension at one time can be associated with a Boolean map; it therefore follows that access to features is strictly serial, whereas access to locations is parallel. This serial/parallel distinction predicts that response time increases rapidly with the number of items (i.e., the slope or set-size effect) in a task that requires access to features but not in a task that requires access to locations, and this was confirmed by the experiments. However, it should be noted that even if the former (i.e., the serial/parallel distinction) predicts the latter (i.e., the difference in the slopes), the latter cannot conclusively demonstrate the former because there are other reasons why the latter could occur (e.g., Huang & Pashler, 2005; Palmer, 1994; Palmer, Ames, & Lindsey, 1993; Palmer, Verghese, & Pavel, 2000). For one, these results could also be consistent with a parallel but limited-capacity access model. In this model, both the locations and the colors are accessed in parallel; the locations are marked on the (x, y) plane discussed above, whereas the colors are represented by a limited pool of representational resource. When there are more items, access to the locations remains good because of the (x, y) plane, but access to the colors becomes substantially worse because each item is now given less resource. Evidence that more directly supports serial access to features was discussed in Huang and Pashler (2007), and this should be explored further in future studies. 

Acknowledgments

This research was supported by RGC Grant 444108 from the Hong Kong government. I thank Alec Scharff and an anonymous reviewer for their very helpful comments on an early draft of this article. 

Commercial relationships: none. 

Corresponding author: Liqiang Huang. 

Email: [e-mail]. 

Address: Department of Psychology, The Chinese University of Hong Kong, Hong Kong, The People's Republic of China. 

Footnote

Footnotes

¹  This thought experiment was suggested by an anonymous reviewer.