The role of attention in retaining the binding of integral features in working memory

Abstract

Previous studies have suggested that retaining bindings in working memory (WM) requires more object-based attention than retaining constituent features. However, we still need to address the object-based attention hypothesis to determine both the generality (Does the object-based attention hypothesis of binding apply to feature bindings other than those tested?) and the reality (Was the observed effect in previous studies an artifact of the testing process?). We addressed these two issues by focusing on the binding of integral features, which was ignored in previous studies. Integral features can be manipulated independently but cannot be attended to or processed independently of each other, and they are primarily perceived in a more unitary fashion. Consequently, integral-feature bindings should be processed as integrated units without the help of extra object-based attention. We examined whether or not the object-based attention hypothesis applied to integral-feature bindings (generality), and these results enabled us to check the reality of the hypothesis. In line with our prediction, we found that a secondary task consuming object-based attention did not selectively impair the binding performance (Experiments 1, 2, 3, 5, and 7). The absence of selective binding impairment was not attributable to the use of an invalid secondary task (Experiment 4), failure to memorize the binding between length and width (Experiment 6), tapping the incorrect type of attention (Experiment 6), the feasibility of feature categorization (Experiment 7), or poor task performance (Experiment 7). Overall, these results suggest that the object-based attention hypothesis does not fit for the integral-feature bindings, and that the pivotal role of object-based attention reported by previous studies was reliable.

Introduction

Objects are comprised of many features such as color, shape, and direction, all of which are processed in separate regions of the brain (Livingstone & Hubel, 1988). The binding problem addresses how these features are combined into integrated objects in cognitive system (Treisman, 1996; Treisman, 1998). It is one of the core issues in cognitive science and has been investigated extensively in the last three decades (Schneegans & Bays, 2019; Wolfe & Robertson, 2012). Ample studies on perception have found that the involvement of attention was essential to having an integrated perceptual representation (e.g., Treisman & Gelade, 1980; Treisman & Schmidt, 1982; Wolfe & Robertson, 2012). However, the role of attention in retaining feature bindings in working memory (WM), a post-perceptual buffer in charge of storing and manipulating a limited set of information (Baddeley & Hitch, 1974; Cowan, 2001), is still the subject of some controversy (e.g., Allen, Baddeley, & Hitch, 2006; Allen, Hitch, & Baddeley, 2009; Baddeley, Allen, & Hitch, 2011; Z. Gao, Wu, Qiu, He, Yang, & Shen, 2017; Johnson, Hollingworth, & Luck, 2008; Peterson & Naveh-Benjamin, 2017; Shen, Huang, & Gao, 2015; for a review, see Allen, 2015). For example, one extensively debated issue is whether retaining bindings in WM requires more attention than the constituent single features (for reviews, see Allen, 2015; Schneegans & Bays, 2019). This question is theoretically important, because it sheds light on several key mechanisms of WM, including WM architecture and the interaction between perception and WM, among others. 

Does retaining binding require more attention than retaining constituent features in WM?

So far two different views have emerged as to whether retaining binding in WM requires more attention than the constituent features: passive view versus active view (for a review, see Schneegans & Bays, 2019). Although earlier studies of WM shed light on this issue (e.g., Gajewski & Brockmole, 2006; Vogel & Luck, 1997; Vogel, Woodman, & Luck, 2001; Wheeler & Treisman, 2002), Allen et al. (2006) initiated the direct investigation via a dual-task paradigm, which has been commonly adopted by later studies. In this paradigm, participants are required to memorize single features or bindings in different blocks, and a secondary task is added to compete for a specific type of attention with the memorized information. If one specific type of attention plays a pivotal role in retaining bindings in WM, then the secondary task will lead to a larger impairment to binding than to the constituent features (selective binding impairment). Otherwise, the secondary task will equally disrupt the performance of binding and the constituent features. Using this paradigm, most of the existing empirical evidence suggests that binding in WM is a passive process that does not require extra attention (e.g., Allen et al., 2006; Allen et al., 2009; Allen, Hitch, Mate, & Baddeley, 2012; Baddeley et al., 2011; Delvenne, Cleeremans, & Laloyaux, 2010; Johnson et al., 2008; Langerock, Vergauwe, & Barrouillet, 2014; Morey & Bieler, 2013). For example, in a series of studies Allen et al. found that a secondary task (e.g., digit backward counting task) consuming domain-general attention impaired the performance of single-feature and binding equally (Allen et al., 2006; Allen et al., 2009; Allen et al., 2012; Baddeley et al., 2011), which was verified by other researchers (e.g., Langerock et al., 2014; Morey & Bieler, 2013; but see Peterson & Naveh-Benjamin, 2017; Peterson, Decker, & Naveh-Benjamin, 2019). Moreover, Johnson et al. (2008) inserted a secondary visual search task, which taxed spatial attention, into the WM maintenance phase and did not find a selective binding impairment, which was replicated by later studies (e.g., Shen et al., 2015). 

The passive view has been challenged by recent studies that have suggested that retaining bindings in WM is an active process requiring more object-based attention than retaining constituent features (Fougnie & Marois, 2009; Z. Gao et al., 2017; He, Li, Wu, Wan, Gao, & Shen, 2020; Lu, Ma, Zhao, Gao, & Shen, 2019; Shen et al., 2015). In these studies, when a secondary task that consumed object-based attention (e.g., Duncan's object-feature report task; Duncan, 1984) was added to the WM maintenance phase, researchers consistently found that the secondary task resulted in a selective binding impairment. This result has been verified using several secondary tasks that tap into object-based attention (e.g., Duncan's object-feature report task, mental rotation task, transparent motion task, multiple-object tracking task) and entail distinct types of feature bindings (e.g., unitized visual binding, cross-module binding, cross-modal binding, cross-time binding, cross-space binding). 

Unfortunately, however, serious questions can be raised about both the generality and the reality of the object-based attention hypothesis of binding. Its generality can be questioned because it is unclear whether the object-based attention hypothesis of binding applies to feature bindings other than those tested; although several types of feature bindings have been tested, to our knowledge no study has addressed the boundary of object-based attention hypothesis. A sophisticated theoretical view can predict when a key factor works and when it does not. The reality of the object-based attention hypothesis of binding can also be questioned. The problem is that the observed selective binding impairment in previous studies may have been an artifact of the testing process, considering that the dual-task setting is fairly complex. The current study was motivated by these two issues of generality and reality. We reported a series of experiments that address the binding of integral features (see next session for an elaboration). The experiments showed that the object-based attention hypothesis did not hold for integral-feature bindings and that the selective binding impairment in previous studies was not an artifact of the testing process. 

Integral features and separable features

A fundamental issue that was long debated in earlier era of cognitive psychology was whether we perceive and process multifeatured stimuli analytically (in terms of their constituent features) or holistically (in terms of their overall similarity). Researchers have since found that the way in which a multifeatured stimulus is perceived and processed seems to depend on the nature of its constituent features. For example, when performing tasks such as speeded sorting and dissimilarity scaling, participants chose to analyze the feature structure of stimuli that varied in brightness and size (e.g., Attneave, 1950; Handel & Imai, 1972); however, in the same task, participants chose to process stimuli in terms of their overall similarity for the stimuli that varied in brightness and saturation (e.g., Garner & Felfoldy, 1970; Handel & Imai, 1972; Torgerson, 1958). These findings led to research to distinguish between two types of feature interactions (Attneave, 1950; Garner, 1974a; Lockhead, 1966; Shepard, 1964). Currently, the most well-accepted version of this distinction was posited by Garner (1974a), who distinguished between separable and integral features. Separable features are those that can be attended to and processed independently of each other, such as color and shape, area and color saturation, and area and brightness. Stimuli composed of separable features have feature structures that are directly perceived; in other words, the feature structure itself determines the similarity between stimuli (a linear summation of the absolute differences each of the features in isolation; cf. Bimler, Izmailov, & Paramei, 2013). In contrast, integral dimensions are those that can be manipulated independently but not attended to and processed independently of each other, such as width and height, brightness and saturation, and hue and color saturation. Stimuli composed of integral features are primarily perceived in a more unitary fashion, in terms of their overall similarity. That is, integral features form a seamless Gestalt, enabling viewers to perceive the stimuli as varying along a single “integral” dimension. Therefore, the similarity among multi-integral-featured stimuli is directly perceived, and the notion of multiple features loses meaning. Neuroimaging studies have demonstrated that there are distinct neural mechanisms for the perception of separable and integral features. Particularly, the integral features draw on overlapping neural populations during perception, whereas separable features are encoded by largely independent neurons (Drucker, Kerr, & Aguirre, 2009; Ganel, Gonzalez, Valyear, Culham, Goodale, & Köhler, 2006). 

The distinction between integral and separable features has critical implications for the study of a number of perceptual and cognitive processes. Integral and separable features contribute in different ways to selective attention, limits of processing, levels of processing, modes of processing, concept learning, and WM storage (Ashby, 1988; Bae & Flombaum, 2013; Cant, Large, McCall, & Goodale, 2008; Dykes & Cooper, 1978; Ganel & Goodale, 2003; Garner, 1974a; Garner, 1974b; Garner, 1976; Garner, 1978; Garner, 1983; Garner, 2014; Garner & Felfoldy, 1970; Kemler, 1983; Koene & Zhaoping, 2007; Shepard, 1964; Shepard & Chang, 1963; Smith & Kemler, 1978; Treisman & Gormican, 1988; Wickens, Hollands, Banbury, & Parasuraman, 2015). For example, because feature structures of separable-feature stimuli can be perceived directly, it has been suggested that little effort is needed to analyze their feature structure of separable stimuli, which then promotes selective attention to the individual features (Garner, 1974a; Garner, 1974b). Garner (1974b) even claimed that selective attention was mandatory with separable features. In contrast, because integral-feature stimuli are perceived in a unitary fashion, participants had to take considerable effort to analyze their feature structure. Additionally, the feature integration theory (FIT) proposed by Treisman and Gelade (1980) applies only to stimuli composed of separable features, explicitly claiming that integral features are conjoined automatically but separable features require attention for their integration. Critically, the distinct processes of integral and separable features are also revealed in WM. Bae and Flombaum (2013) found that the binding between separable features is not stable, resulting in more conjunction errors when memorizing multiple objects and less precision when memorizing two objects than when memorizing one; however, they also found that the binding between integral features was stable and conjunction errors rarely occurred, with similar precision when memorizing two objects or memorizing one. Fougnie and Alvarez (2011) revealed that for an object consisting of color and orientation (separable features), the representations of color and orientation in WM decayed independently in WM; however, for an object consisting of width and height (integral features), the representations of the two features decayed dependently. 

Considering the distinct processing mechanisms of integral and separable features, we will argue here that the object-based attention hypothesis of binding is constrained to separable-feature bindings. For integral-feature bindings, the constituent features will form one unit, roughly equivalent to one feature; therefore, retaining integral-feature bindings in WM is a passive process that should not require extra object-based attention relative to constituent features. However, all existent studies on the attentional mechanisms of binding in WM have used separable features (e.g., color and shape, color and location, color and orientation). To the best of our knowledge, no previous WM study has investigated the attentional mechanisms for bindings composed of integral features. 

Current research

To verify that the object-based attention hypothesis does not fit for integral-feature bindings, we used the same testing procedure employed in previous binding studies. Specifically, we required the participants to memorize single features or bindings between them, and we then added a secondary task to the maintenance phase of WM to consume their object-based attention (Z. Gao et al., 2017; Lu et al., 2019; Shen et al., 2015). We predicted that a nonselective binding impairment would be observed; however, if a selective binding impairment emerged, that would imply that the previous selective binding impairment had just been an artifact of the testing procedure. Additionally, because the current study did not aim to discover integral features in WM, we adopted a well-established integral feature combination: width and height (e.g., Fougnie & Alvarez, 2011; Wickens et al., 2015). Although the distinction between integral and separable features has been revealed in such tasks as speed sorting, restricted classification, and dissimilarity scaling (cf. Grau & Nelson, 1988), ample studies have suggested that the corresponding distinct mechanisms should not change in other tasks (e.g., Bae & Flombaum, 2013; Cant et al., 2008; Fougnie & Alvarez, 2011; Garner, 2014; Garner & Felfoldy, 1970; Goodale & Ganel, 2003; Kemler, 1983; Koene & Zhaoping, 2007; Treisman & Gormican, 1988). 

Experiment 1: The role of object-based attention

In line with previous binding studies (e.g., Allen et al., 2006; Allen et al., 2009; Allen et al., 2012; Z. Gao et al., 2017; Johnson et al., 2008; Langerock et al., 2014; Morey & Bieler, 2013; Peterson & Naveh-Benjamin, 2017; Peterson et al., 2019; Shen et al., 2015), we presented the same set of stimuli for the memory array of binding and feature conditions. 

To consume the participants’ object-based attention, we used transparent motion.¹ To demonstrate the existence of object-based attention, Valdes-Sosa et al. presented participants with two superimposed transparent moving surfaces that were defined by two interspersed and differentially colored sets of dots (i.e., random dot kinematograms, RDKs) (Valdes-Sosa, Cobo, & Pinilla, 1998; Valdes-Sosa, Cobo, & Pinilla, 2000). Participants had to judge the direction and speed of the surfaces. The dots had a short lifespan, and because some of them (e.g., 50%) moved in a common direction while others moved randomly, participants needed to direct their attention on a specific transparent surface instead of focusing on individual dots. Valdes-Sosa et al. revealed an object-based processing advantage: Accuracy was higher when the reported direction and speed were from one surface than when they were from two surfaces. This effect is explained by all features of an attended object being processed concurrently; each surface observed in transparent motion could be conceived as an object, and this object-based processing occurred at an early stage of visual processing (Pinilla, Cobo, Torres, & Valdes-Sosa, 2001; Schoenfeld, Tempelmann, Martinez, Hopf, Sattler, Heinze, & Hillyard, 2003). The transparent motion task has been acknowledged and adopted for the exploration of the mechanisms of object-based attention (Ciaramitaro, Mitchell, Stoner, Reynolds, & Boynton, 2011; Schoenfeld et al., 2003; Schoenfeld, Hopf, Merkel, Heinze, & Hillyard, 2014; for a review, see Chen, 2012). 

Because we were previously only interested in consuming object-based attention, we presented participants with only one RDK containing just one transparent surface (Shen et al., 2015). Participants judged the movement of the surface (top-left or top-right direction). To fulfill the task, participants had to process the RDK as one object. Critically, not using the standard task of Valdes-Sosa et al. (1998, 2000) does not mean that the adapted transparent motion task was not completed with object-based attention. The work of Valdes-Sosa et al. (1998, 2000) shows that our vision system treats each surface in an object-based manner, and this manner should not alter when only one surface is perceived. Therefore, even though we required participants to process only one feature of a surface (e.g., movement direction), the color of the surface could also be extracted into the object file. Corroborating this view, as with Duncan's object-feature report (Z. Gao et al., 2017; He et al., 2020; Shen et al., 2015), a secondary one-surface transparent motion task led to a selective binding impairment (Shen et al., 2015). 

Methods

Participants

A priori power analysis was conducted with the program G*Power 3.1.9.2 (Faul, Erdfelder, Buchner, & Lang, 2009). For a 2 × 3 repeated-measures analyses of variance (ANOVA) with a moderate effect size of f = 0.25 (Cohen, 1988), α = 0.05, and 1 – β = 0.90, an N of 24 is sufficient to detect an effect with a statistical power of 0.91. Twenty-four valid participants (6 males, 18 females), with an average age of 22.33 years (SD = 2.92), took part in the experiment. If a participant's overall performance of the WM task or the secondary task was at the chance level or if the accuracy in the memory task under the with-motion condition was 20% higher than that under the no-motion condition (i.e., failure of the secondary task), that participant would be replaced. As a result, one participant was replaced due to chance-level performance in the memory task, and one participant was replaced because the accuracy in the memory task under the with-motion condition was even 20% higher than that under the no-motion condition. All participants were undergraduate students from Zhejiang University, who signed consent forms, had normal color vision, and normal or corrected-to-normal visual acuity. The study conformed to Standard 8 of the American Psychological Association's Ethical Principles of Psychologists and Code of Conduct and was approved by the Research Ethics Board of Zhejiang University. 

Apparatus and stimuli

Stimuli were presented against a gray background (RGB values of 128, 128, and 128) on a 19-inch cathode-ray tube monitor with a resolution of 1024 × 768 pixels at a 60-Hz refresh rate. Participants were seated in a dark room, approximately 60 cm from the screen. 

The same set of stimuli were used for the memory array of binding and feature conditions. The memory items were three distinct rectangles; each was 200 pixels away from the screen center. The three rectangles formed a configuration of either an upward-pointing triangle or a downward-pointing triangle. The values of width and height of the three rectangles were selected from a set of 10 values without replacement: visual angles of 0.5°, 1.5°, 2.5°, 3.5°, 4.5°, 5.5°, 6.5°, 7.5°, 8.5°, and 9.5°. Probes were presented in the center of the screen. The probed information appeared in the memory array on half of the trials and not on the other half. In the feature condition, black lines (1 pixel thick) were used to indicate the value of the probed feature dimension.² When probing a new feature, the value of the displayed line deviated more than 2° from the corresponding values in the memory array; when probing a new binding, a new rectangle was used that had the width of an old rectangle and the height of another old rectangle in the memory array. 

The transparent motion task was adopted from Shen et al. (2015). Stimuli in the transparent motion (see Figure 1) consisted of 50 black dots (0.1° in diameter) that moved within a 3° × 3° virtual square region in the screen center. Across the trials, 35 dots moved coherently in the top-left (50% of trials) or top-right (50% of trials) direction, and the other 15 dots moved randomly in the six unused directions (i.e., moving in the left, right, bottom-left, bottom-right, top, or bottom direction). If a dot passed the border of a square region, it wrapped around to an opposite but symmetrical position. Dots moved at a speed of 1.3°/s or 2.0°/s and were randomly selected for each trial. It can be difficult for participants to fulfill the task by focusing on individual dots; instead, they had to direct their attention on the specific transparent surface, which was also confirmed by our previous pilot study. 

Figure 1.

 

Schematic illustration of a trial used in Experiment 1. Here we show a trial without probe changes.

Design and procedure

The experiment adopted a 2 (task load: no-motion task vs. with-motion task) × 2 (memory condition: feature vs. binding) within-subject design. The whole experiment was divided into two sessions according to whether a secondary task was involved, the order of which was counterbalanced among participants. Each session had two blocks: feature and binding, the order of which was fully counterbalanced using an ABBA structure. A feature block had 72 trials (36 trials probing width and 36 trials probing height were mixed), and a binding block had 36 trials. This setting resulted in a total of 216 trials. The trials within each block were displayed randomly. Before entering each block, participants completed at least 10 practice trials and began the formal experiment when their accuracy score was no less than 60% of the memory task and no less than 80% of the secondary task in practice. 

The procedure of a trial is shown on Figure 1. Each trial began with a 500-ms fixation, and participants were required to fixate on the screen center. After a 250-ms blank interval, the memory array was presented for 1000 ms. Participants had to memorize both the width and height of the rectangle in the feature condition while retaining the binding between width and height in the binding condition. After a 500-ms interval, a transparent motion task appeared for 2000 ms at most and disappeared immediately when a response was made. Participants made a button press to indicate whether the surface moved top-left (F on the keyboard) or top-right (J on the keyboard) in the with-motion condition, and they pressed the spacebar on the keyboard in the no-motion condition. Then, after a 500-ms interval, a probe was presented for 3000 ms at most and disappeared immediately when a response was made. Participants pressed a button to indicate whether the probe appeared in the memory array (F for absence and J for presence). If a participant did not make a response to the transparent motion task within 2000 ms, it was treated as a wrong response to the transparent motion task. Participants were asked to prioritize the transparent motion task, and the accuracy of both WM and motion task was emphasized. The interval between trials was 500 ms. The experiment lasted approximately 45 minutes. Participants were allowed to rest between blocks for 5 to 10 minutes. 

Analysis

Only trials with correct responses on the secondary task were further analyzed. To allow direct comparison with previous studies on this topic (e.g., Allen et al., 2006; Z. Gao et al., 2017; Shen et al., 2015), the change detection performance of the memory task is reported as corrected recognition (hits minus false alarms) (Snodgrass & Corwin, 1988). A two-way repeated-measures ANOVA was conducted on the corrected recognition, with task load (no motion vs. with motion) and memory condition (width, height, and binding) as within-subject factors. Moreover, given that Bayesian analysis of variance is better for providing evidence for the null hypothesis (Rouder, Morey, Speckman, & Province, 2012), we used a Bayesian repeated-measures ANOVA in JASP to calculate the Bayes factor (BF). For the main effect, we compared models containing this main effect to equivalent models stripped of this main effect; for interaction, we compared a full model (numerator, including the two main effects and the interaction) with a reduced model (denominator), in which the effect of interest was not included, to compute the BFs for the main effects and the interaction. A BF of 3 indicates substantial evidence for the selection of one model over the other, whereas a BF of 10 is considered to provide strong evidence for the selection of one model over the other (Jeffreys, 1961). 

Results

The overall accuracy for the secondary task was 97.86% (96.72% with-motion condition, 99.00% no-motion condition). The overall accuracy for the memory task was 66.61% (65.94%, 66.47%, and 67.41% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

Table 1.

 

Memory task accuracy, hit rate, and false alarms for each condition for Experiments 1 to 7. CR, corrected recognition.

 

The corrected recognition under each condition is shown in Figure 2. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 8.56, p = 0.01, \(\eta _p^2\; = 0.27\), BF = 12.52, suggesting that performance was significantly better under the no-motion condition than that under the with-motion condition. The main effect of memory condition was not significant, F(2, 46) = 0.45, p = 0.64, \(\eta _p^2\; = 0.02\), BF = 0.10, and the interaction between task load and memory condition was not significant, F(2, 46) = 0.32, p = 0.73, \(\eta _p^2\; = 0.01\). Confirming this null effect, the BF for the task load × memory condition interaction was only 0.16, favoring a reduced model (i.e., without interaction). 

Figure 2.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 1. For accuracy, hit rate, and false alarm data, see Table 1. Error bars stand for 95% confidence intervals (CIs).

Discussion

In line with our prediction, Experiment 1 demonstrated that the added transparent motion task impaired the performance of binding and single features equally, suggesting that retaining bindings of integral features in WM did not require more object-based attention than retaining constituent single features. This finding differs from those of previous studies (Z. Gao et al., 2017; Shen et al., 2015) and implies that previous selective binding impairments were not an artifact of the testing procedure. 

Although the participants could attend to the height or width of a rectangle according to the instruction, this method is not the default for processing integral-feature stimuli and required extra effort. It is possible that the participants actually processed and stored the same information between feature and binding conditions. Indeed, the accuracy among the three memory conditions was comparable. To test whether retaining integral-feature bindings required more object-based attention than retaining the constituent features, it was necessary to set a condition in which participants stored widths and heights in the feature conditions. We tested this situation in Experiments 2 and 3. 

Experiment 2: Displaying feature conditions in distinct blocks

In Experiment 2, we divided the feature block from Experiment 1 into two blocks: a width block and a height block. In the width block, only horizontal lines were displayed; in the height block, only vertical lines were displayed. Therefore, participants had to retain features in the feature condition. If Experiment 2 replicated the findings of Experiment 1, that would provide converging evidence supporting the idea that retaining integral-feature bindings does not require extra object-based attention. 

Methods

Twenty-four valid participants (14 males, 10 females), with an average age of 21.71 years (SD = 2.07), took part in the experiment. Four participants were replaced due to their chance-level performance in the memory task. 

The other aspects were the same as Experiment 1, except for the following: (1) in the feature conditions, three horizontal lines (1-pixel thick; Figure 3a) and three vertical lines (1-pixel thick; Figure 3b) were presented for the width condition and height condition, respectively; (2) participants were required to retain widths under the width condition and retain heights under the height condition; and (3) the experiment was divided into two sessions according to whether a secondary task was involved, and each session had three blocks (width, height, and binding), the order of which was fully counterbalanced using a Latin square. Each block had 36 trials. 

Figure 3.

 

Schematic illustration of memory arrays used in Experiment 2. There are three horizontal lines for width condition (top), three vertical lines for height condition (middle), and three rectangles for the binding condition (bottom).

Results

The overall accuracy for the secondary task was 97.70% (96.33% with-motion condition, 99.07% no-motion condition). The overall accuracy for the memory task was 70.86% (72.96%, 73.77%, and 65.79% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognition under each condition is shown in Figure 4. The two-way ANOVA revealed a significant main effect of task load, F(1, 23) = 7.30, p = 0.01, \(\eta _p^2\; = 0.24\), BF = 24.19, suggesting that performance was significantly better under the no-motion condition than under the with-motion condition. The main effect of memory condition was significant, F(2, 46) = 15.15, p < 0.001, \(\eta _p^2\; = 0.40\), BF = 120674.74; post hoc contrasts (Bonferroni-corrected) revealed that performance was significantly worse under the binding condition (0.31) than under the width (0.46) and height (0.48) conditions. The interaction between task load and memory condition was not significant, F(2, 46) = 1.89, p = 0.16, \(\eta _p^2\; = 0.08\). Confirming this null effect, the BF for the task load × memory condition interaction was 0.31, favoring a reduced model (i.e., without interaction). 

Figure 4.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 2. For accuracy, hit rate, and false alarm data, see Table 1. Error bars stand for 95% CIs.

Discussion

Experiment 2 replicated the finding of Experiment 1; that is, the secondary task impaired the feature and binding performance equally. These results suggested that relative to single features, retaining the integral-feature bindings does not require extra object-based attention. 

Experiment 3: Displaying both features separately in one block

In Experiment 3, three horizontal lines and three vertical lines were presented to the participants in the memory array in the feature block. Participants were required to memorize both widths and heights in the feature condition; therefore, the number of features to be memorized was comparable between Experiment 1 and Experiment 3 in the feature block. 

Methods

Twenty-four valid participants (8 males, 16 females), with an average age of 19.58 years (SD = 2.27), took part in the experiment. Two participants were replaced due to chance-level performance in the memory task, and one participant was replaced because the accuracy in the memory task under the with-motion condition was 20% higher than that under the no-motion condition. 

The other aspects were the same as Experiment 1, except for the following aspects: Under the feature condition, the memory items were six lines that included three widths and three heights (1-pixel thick; see Figure 5, top). They were presented within a 25° × 23° virtual rectangle region (centered at the screen center) without overlap. The region in which lines were presented was almost the same as that for rectangles under the binding condition. 

Figure 5.

 

Schematic illustration of memory arrays used in Experiment 3. There are three horizontal lines and three vertical lines for feature condition (top), and three rectangles for the binding condition (bottom).

Results

The overall accuracy for the secondary task was 97.84% (97.30% with-motion condition, 98.38% no-motion condition). The overall accuracy for the memory task was 61.57% (60.05%, 60.40%, and 64.25% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognition under each condition is shown in Figure 6. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 17.61, p < 0.001, \(\eta _p^2\; = 0.43\), BF = 185.45, suggesting that performance was significantly better under the no-motion condition than under the with-motion condition. The main effect of memory condition was significant, F(2, 46) = 5.11, p = 0.01, \(\eta _p^2\; = 0.18\), BF = 3.11. Post hoc contrasts (Bonferroni-corrected) revealed that performance was significantly better under the binding condition (0.28) than under the width (0.20) and height (0.21) conditions. The interaction between task load and memory condition was not significant, F(2, 46) = 1.06, p = 0.36, \(\eta _p^2\; = 0.04\). Confirming this null effect, the BF for the task load × memory condition interaction was 0.32, favoring a reduced model (i.e., without interaction). 

Figure 6.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 3. For accuracy, hit rate, and false alarm data, see Table 1. Error bars stand for 95% CIs.

Discussion

The findings of Experiment 3 were in line with Experiments 1 and 2, as the transparent motion task equally impaired the performance of feature and binding. Therefore, Experiments 1, 2, and 3 consistently supported the view that retaining integral-feature bindings in WM does not require more object-based attention than retaining constituent single features does. 

Because the key findings were null effect in Experiments 1, 2, and 3, at least two alternatives had to be addressed before reaching a solid conclusion. First, was the transparent motion in the current study really effective, although it had been successfully used in previous studies (e.g., Z. Gao et al., 2017; Shen et al., 2015)? Second, supposing that the transparent motion used was effective, was the current finding restricted to the transparent motion task? We addressed these two issues in Experiment 4 and Experiment 5, respectively. 

Experiment 4: Validity of transparent motion task

In Experiment 4, we examined the validity of the transparent motion task used in Experiments 1, 2, and 3. Following previous binding studies, we used separable feature pairs: distinct colors and shapes (Allen et al., 2006; Allen et al., 2009; Gajewski & Brockmole, 2006; Z. Gao et al., 2017; Johnson et al., 2008; Shen et al., 2015). It had been found that adding a transparent motion task into the WM maintenance phase drove a selective binding impairment (Z. Gao et al., 2017; Shen et al., 2015). 

Methods

Twenty-four valid participants (8 males, 16 females), with an average age of 18.33 years (SD = 0.94), took part in the experiment. Two participants were replaced due to not pressing spacebar in the no-secondary-task condition, and one participant was replaced due to chance-level performance in the memory task. 

The other aspects of the experiment were the same as Experiment 1, except for the following aspects: The memory items were three distinct colored shapes (see Figure 7); each was 108 pixels away from the screen center. The colors and shapes were randomly selected from a pool of six distinct values of each type (see Figure 8), and the three colored shapes formed a configuration of either an upward-pointing triangle or a downward-pointing triangle. In the feature block, half of the trials probed color and the other half of the trials probed shape; color probes were bubbles, and shape probes were hollow shapes (see Figure 8). When probing a new feature, a new color or shape not used in the memory array was presented in the feature condition. A new colored shape in the binding condition had the color of an old item and shape of another old item in the memory array. 

Figure 7.

 

Schematic illustration of memory arrays used in Experiment 4.

Figure 8.

 

Colors and shapes used in Experiment 4. (a) The colors (2.4° × 2.4° of visual angle about bubbles) are red (255, 0, 0 RGB), green (0, 255, 0 RGB), blue (0, 0, 255 RGB), yellow (255, 255, 0 RGB), magenta (255, 0, 255 RGB), and white (255, 255, 255 RGB), from left to right. (b) The shapes (2.4° × 2.4° of visual angle on average) are a circle, cross, diamond, star, triangle, and chevron, from left to right.

Results

The overall accuracy for the secondary task was 97.15% (95.72% with-motion condition, 98.57% no-motion condition). The overall accuracy for the memory task was 79.61% (86.61%, 76.50%, and 75.78% for color, shape, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognition under each condition is shown in Figure 9. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 25.23, p < 0.001, \(\eta _p^2\; = 0.52\), BF = 5447.04, suggesting that performance was significantly better under the no-motion condition than under the with-motion condition. The main effect of memory condition was significant, F(2, 46) = 25.44, p < 0.001, \(\eta _p^2\; = 0.53\), BF = 5.74E+8. Post hoc contrasts (Bonferroni-corrected) revealed that performance was significantly better under the color condition (0.73) than under the shape (0.53) and binding (0.52) conditions. The interaction between task load and memory condition was significant, F(2, 46) = 4.45, p = 0.02, \(\eta _p^2\; = 0.16\), BF = 1.41. 

Figure 9.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 4. For accuracy, hit rate, and false alarm data, see Table 1. Error bars stand for 95% CIs.

To deconstruct this interaction, planned contrasts were conducted and showed that the disruption caused by the secondary task was significantly larger under the binding condition (0.21) than under the color condition (0.06), t(23) = 2.91, p = 0.01, Cohen's d = 0.78, and larger under the binding condition (0.10) than under the shape condition, t(23) = 2.02, p = 0.05, Cohen's d = 0.51. 

Comparing Experiments 3 and 4

To further check whether there are distinct mechanisms for retaining separable-feature bindings and integral-feature bindings, we compared the results of Experiment 3 and Experiment 4 because the memory content was essentially parallel (i.e., in one block both features were retained and in the other block bindings were retained³). Because there is no direct feature dimension correspondence between the two experiments, to avoid feature dimension correspondence problems during the analysis, we pooled the two feature conditions as one feature condition. Moreover, the impairment between the two features was comparable in both experiments, suggesting that our data pooling was valid. We conducted a mixed ANOVA by taking task load (no-motion task vs. with-motion task) and memory condition (feature vs. binding) as within-subject factors and stimuli (integral features vs. separable features) as a between-subject factor. 

The three-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 46) = 37.92, p < 0.001, \(\eta _p^2\; = 0.45\), BF = 1.56E+6, suggesting that performance was significantly better under the no-motion condition than under the with-motion condition. The main effect of memory condition was not significant, F(1, 46) = 1.00, p = 0.32, \(\eta _p^2\; = 0.02\), BF = 0.22. The main effect of stimuli was significant, F(1, 46) = 80.17, p < 0.001, \(\eta _p^2\; = 0.64\), BF = 6.66E+8, suggesting that performance was significantly better in Experiment 4 than in Experiment 3. The memory condition × stimuli interaction was significant, F(1, 46) = 28.50, p < 0.001, \(\eta _p^2\; = 0.38\), BF = 16403.53, but the task load × stimuli interaction was not significant, F(1, 46) = 2.60, p = 0.11, \(\eta _p^2\; = 0.05\), BF = 0.64. The task load × memory condition interaction also was not significant, F(1, 46) = 0.41, p = 0.52, \(\eta _p^2\; = 0.01\), BF = 0.23. Critically, the task load × memory condition × stimuli interaction was significant, F(1, 46) = 9.05, p = 0.004, \(\eta _p^2\; = 0.16\), BF = 8.46. Further analysis showed that there was no significant difference between the feature impairment and the binding impairment in Experiment 3, F(1, 46) = 2.80, p = 0.10, \(\eta _p^2\; = 0.06\), whereas the binding impairment was significantly larger than the feature impairment in Experiment 4, F(1, 46) = 6.67, p = 0.01, \(\eta _p^2\; = 0.13\). 

Discussion

Experiment 4 revealed that the transparent motion task impaired the binding performance to a larger degree than it impaired the feature performance. This result replicated previous finding (e.g. Z. Gao et al., 2017; Shen et al., 2015), suggesting that retaining separable-feature bindings in WM requires more object-based attention than retaining constituent features does. Therefore, the transparent motion task used in Experiments 1, 2, and 3 was effective in consuming object-based attention. Moreover, the mixed ANOVA revealed that the task load × memory condition interaction was different between Experiment 3 and Experiment 4, suggesting that object-based attention played distinct roles in retaining integral-feature bindings and separable-feature bindings. 

Experiment 5: Replicating the key finding via a mental rotation task

Experiment 5 addressed whether the non-selective binding impairment in Experiments 1, 2, and 3 was constrained to the transparent motion task. We replaced the transparent motion task to a mental rotation task (e.g., mentally rotating the letter R). Previous studies have revealed that we have to mentally perform an object-based transformation when mentally rotating an object (for reviews, see Dalecki, Hoffmann, & Bock, 2012; Zacks & Michelon, 2005). That is, attention is operating on the representation of object: Participants mentally rotate an object relative to axes defined with respect to the object (object-based frame) without any movement in two-dimensional space. Moreover, behavioral (Hyun & Luck, 2007) and event-related potential (Prime & Jolicoeur, 2010) studies have found that object WM instead of spatial WM is the substrate for mentally rotating a letter. Considering that object-based attention, instead of spatial-based attention, plays a critical role in retaining object in visual WM (Barnes, Nelson, & Reuter-Lorenz, 2001; Matsukura & Vecera, 2009; Matsukura & Vecera, 2011; Woodman & Vecera, 2011), and visual WM is conceived as visual attention sustained internally over time (Chun, 2011; Chun, Golomb, & Turk-Browne, 2011; Kiyonaga & Egner, 2013), we argue that mentally rotating a letter consumes object-based attention (for a similar claim, see Jansen, & Lehmann, 2013) in WM (see experiments 1 to 3 in He et al., 2020; Shen et al., 2015). Supporting this view, we previously used a mental rotation task to consume object-based attention in exploring the binding mechanism of separable features (see experiments 1 to 3 in Shen et al., 2015). Akin to the transparent motion task and Duncan's object-feature report task, we consistently found that the mental rotation task impaired the binding performance to a large degree relative to constituent features in three types of bindings. If the findings of Experiments 1, 2, and 3 were due to specific parameters used in the transparent motion task, then a different result pattern may be observed; otherwise, Experiment 5 would replicate the findings of Experiments 1, 2, and 3. 

Methods

We recruited 24 participants (11 males, 13 females) with an average age of 22.08 years (SD = 1.87). 

The other aspects were the same as Experiment 1, except for the following aspects: We used a mental rotation task (see Figure 10) as the secondary task, wherein a 1.48° × 1.65° rotated black R in its canonical (50% of trials) or mirror-reversed form was presented at screen center. The stimulus was rotated clockwise or counterclockwise from an upright position by a random angle between 72° to 144°. The canonical/mirror-reversed form and rotation angle were combined randomly. Participants pressed a button to judge whether the letter was canonical (J on the keyboard) or mirror-reversed (F on the keyboard). In the no-rotation condition, they pressed the spacebar to ignore the letter. The letter appeared for 2000 ms at most and disappeared immediately when a response was made. If participants did not make a response to the rotation motion task within 2000 ms, it was treated as a wrong response to the transparent motion task. 

Figure 10.

 

Illustration of the stimuli used in mental rotation in Experiment 5.

Results

The overall accuracy for the secondary task was 97.45% (96.80% with-rotation condition, 98.11% no-rotation condition). The overall accuracy for the memory task was 64.75% (65.20%, 65.58%, and 63.46% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognition under each condition is shown in Figure 11. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 11.53, p = 0.002, \(\eta _p^2\; = 0.33\), BF = 5.49, suggesting that performance was significantly better under the no-rotation condition than under the with-rotation condition. The main effect of memory condition was not significant, F(2, 46) = 0.74, p = 0.48, \(\eta _p^2\; = 0.03\), BF = 0.16. The interaction between task load and memory condition also was not significant, F(2, 46) = 0.29, p = 0.75, \(\eta _p^2\; = 0.01\). Confirming this null effect, the BF for the task load × memory condition interaction was only 0.15, favoring a reduced model (i.e., without interaction). 

Figure 11.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 5. For accuracy, hit rate, and false alarm data, see Table 1. Error bars stand for 95% CIs.

Discussion

Experiment 5 used a mental rotation task to consume object-based attention and found that the secondary task significantly impaired the WM performance. Critically, in agreement with Experiments 1, 2, and 3, Experiment 5 found that the mental rotation task equally impaired the performance of feature and binding, suggesting that the findings of Experiments 1, 2, and 3 were not due to specific procedure of the transparent motion task. 

Interim summary

So far in Experiments 1, 2, 3, and 5, we consistently found that a secondary task consuming object-based attention did not lead to a selective binding impairment. To further examine this null effect, we pooled all of the data of these experiments together and calculated the effect size and Bayes factor, considering that more data will increase the statistical power. We conducted a three-way repeated-measures ANOVA on the performance of the WM task by taking task load (no secondary task vs. with secondary task) and memory condition (width, height, and binding) as within-subject factors and experiment (Experiment 1, 2, 3, and 5) as a between-subject factor. 

The three-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 92) = 41.72, p < 0.001, \(\eta _p^2\; = 0.31\), BF = 4.38E+7, suggesting that performance was significantly better under the no-secondary-task condition than under the with-secondary-task condition. The main effect of memory condition was not significant, F(2, 184) = 1.39, p = 0.25, \(\eta _p^2\; = 0.02\), BF = 0.08. The main effect of experiment was significant, F(3, 92) = 16.99, p < 0.001, \(\eta _p^2\; = 0.36\), BF = 584245.03, suggesting that performance was significantly better in Experiment 2 than in Experiments 1, 3, and 5 (p < 0.01) and better in Experiment 1 than in Experiment 3 (p = 0.002). The memory condition × experiment interaction was significant, F(6, 184) = 6.18, p < 0.001, \(\eta _p^2\; = 0.17\), BF = 32761.19, whereas the task load × experiment interaction was not significant, F(3, 92) = 0.23, p = 0.88, \(\eta _p^2\; = 0.01\), BF = 0.02. Critically, the task load × memory condition interaction was not significant, F(2, 184) = 0.02, p = 0.98, \(\eta _p^2\; < 0.001\), BF = 0.04, and it was not further modulated by a third factor experiment, F(6, 184) = 1.086, p = 0.37, \(\eta _p^2\; = 0.03\), BF = 0.08. 

These results suggest that, although we manipulated the load of secondary task successfully (\(\eta _p^2 = 0.31\), BF = 4.38E+7), the task load did not modulate the impairment across memory conditions (\(\eta _p^2 < 0.001\), BF = 0.04). Because the effect size was very low and the BF value was below 1/3, we argued that retaining integral-feature bindings in WM does not require more attention than the constituent features. 

Did participants memorize area of the rectangles instead of binding?

We have shown evidence that retaining integral-feature bindings in WM is a passive view, which holds across different designs and secondary tasks. However, some may argue that participants did not memorize the binding between width and height in Experiments 1, 2, 3, and 5 but instead memorized the area of the rectangles. To rule out this alternative, we performed a control experiment in which we used only the binding condition of Experiment 1 (72 trials; half for with-motion, half for no-motion). Critically, in 50% of the trials (α group), in a memory array containing three rectangles (A, B, and C), the area of rectangle A was the same as the product of the width of B and the height of C. In the no-change trials, the probe was A, and in the change trials the probe was a new one whose width was from B and height was from C. The other trials were the same setting as in Experiment 1 (β group). If the participants memorized the area of a rectangle, then they would tend to judge that the changed probe was an old one in the α group, leading to higher false alarm rate than in the β group. Otherwise, there should be no difference between the α and β groups. We tested 10 participants and found a non-significance between the two groups: t(9) = 0.15, p = 0.89, Cohen's d = 0.05 for no-motion condition; t(9) = 0.52, p = 0.62, Cohen's d = 0.17 for with-motion condition. This finding implies that participants did not simply memorize the square of the rectangles in the binding condition. 

It could still be argued, however, that participants did not memorize the exact area of rectangle but the rough area. We noticed that ample previous studies have shown that adults perceive or memorize the area of a rectangle by obeying a multiplicative rule (e.g., Algom, Wolf, & Bergman, 1985; Anderson & Weiss, 1971; for a review, see Rulence-Paques & Mullet, 1998); that is, adults first acquire the width and height of the rectangle and then integrate the two dimensions by applying a multiplicative operation. Therefore, an extra multiplication process is required when assessing the area of rectangle relative to its constituent features. Because the operation of multiplication theoretically requires the involvement of a central executive, a new prediction emerged if participants memorized the area (even rough area) of the rectangle: Retaining width–height binding requires domain-general attention from the central executive. This prediction could also explain the absence of selective binding impairment in Experiments 1, 2, 3, and 5 from a new perspective: Experiments 1, 2, 3, and 5 did not tap the core resource used for retaining integral feature bindings in WM. We addressed the new prediction in Experiment 6. 

Experiment 6: The role of domain-general attention

A multiplicative operation in calculating the area of a rectangle also implies that participants need to extract the width and height of a rectangle independently. Therefore, to test the multiplicative operation alternative, Experiment 6 replaced the secondary task in Experiment 1 with a digit task: A digit backward counting (BC) or articulatory suppression (AS) task was performed from the beginning of a trial. It has been suggested that more domain-general attention is needed in a BC task relative to in an AS task (Allen et al., 2009; Baddeley et al., 2011; Postma & De Haan, 1996). If participants indeed conducted a multiplicative operation, a larger impairment would be observed in the binding condition than in the feature conditions. 

Methods

We recruited 24 participants (9 males, 15 females), with an average age of 21.46 years (SD = 1.76). 

The other aspects were the same as in Experiment 1, except for the following aspects: We used a digit task to consume domain-general attention. At the beginning of each trial, two-digit numbers between 20 and 99 were presented on the screen center, which lasted for 2000 ms. In the BC condition, participants were instructed to count aloud in decrements of three from this number until the probe appeared. In the AS condition, as a baseline, participants simply repeated the two digits until the probe appeared. The interval between the memory array and probe was 1000 ms (cf. Allen et al., 2006). 

Results

The overall accuracy for the memory task was 68.34% (67.19%, 68.63%, and 69.21% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognitions under each condition are shown in Figure 12. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 14.37, p = 0.001, \(\eta _p^2\; = 0.38\), BF = 8.92, suggesting that performance was significantly better under the AS condition than under the BC condition. The main effect of memory condition was not significant, F(2, 46) = 0.80, p = 0.45, \(\eta _p^2\; = 0.03\), BF = 0.14. The interaction between task load and memory condition also was not significant, F(2, 46) = 0.18, p = 0.84, \(\eta _p^2\; = 0.01\). Confirming this null effect, the BF for the task load × memory condition interaction was only 0.15, favoring a reduced model (i.e., without interaction). 

Figure 12.

 

The corrected recognition (hit rate – false alarm) under each condition for Experiment 6. For accuracy, hit rate, and false alarm data, please see Table 1. Error bars stand for 95% CIs.

Discussion

Experiment 6 revealed that the consumption of domain-general attention did not lead to a selective binding impairment, implying that participants did not conduct a multiplicative operation to obtain the area of the rectangle. This finding is in line with Experiments 1, 2, and 3, supporting a passive view of retaining integral-feature bindings in WM. Moreover, Experiment 6 closed a gap in exploring the role of domain-general attention in retaining bindings in WM. Although ample studies have addressed the role of domain-general attention (e.g., Allen et al., 2006; Allen et al., 2009; Baddeley et al., 2011; for a review, see Allen, 2015), no study addressed its role in integral-feature bindings. Experiment 6 revealed that extra domain-general attention was not required either for retaining integral-feature binding in WM. 

Experiment 7: The influence of feature categorization and task difficulty

There were two key differences between our experiments tapping integral-feature bindings (Experiment 1, 2, 3, and 5) and our experiment tapping separable-feature bindings (Experiment 4). First, participants’ task performance in the integral-feature experiments was worse than in the separable-feature experiment (see Table 1). It is possible that the absence of any selective binding impairment was due to the floor performance of the integral-feature experiments. Second, participants had difficulty categorizing (or verbally coding) the integral features but could categorize the separable features. It is possible that the feature categorization itself modulated the role of object-based attention. To rule out the two alternatives, we ran an Experiment 7, building on Experiment 1: We lowered the memory load from three stimuli to two and made the integral feature easy to categorize. If one of the alternatives was correct, we would observe a selective binding impairment; however, if participants treated the information as integral features, then a finding similar to that of Experiment 1 would be observed. 

Methods

We recruited 24 valid participants (9 males, 15 females), with an average age of 19.42 years (SD = 1.29). One participant was replaced due to chance-level performance in the secondary task. 

The other aspects were the same as in Experiment 1, except for the following aspects: The memory array consisted of two horizontally displayed rectangles; each rectangle was 200 pixels away from the screen center. The values of width and height of the two rectangles were selected from a set of four values each (visual angles of 0.5°, 3.5°, 6.5°, or 9.5°) (see Figure 13), with a difference of at least 3° between the two feature values used. The settings of memory load and feature values together allowed the participants to categorize the features easily—for example, by using “short” or “long.” To avoid participants only memorizing one rectangle in the binding condition, we had three types of binding probe in each binding block (cf. Peterson & Naveh-Benjamin, 2017; Peterson et al., 2019): 18 trials probed an old item in the memory array, 18 trials probed a wrongly bounded item (i.e., width was from an old rectangle and height was from the other old rectangle), and eight trials probed a new item that contained a new value not used in the memory array and an old value from the memory array. The three types of probes were presented randomly. 

Figure 13.

 

Schematic illustration of the four values of feature used in Experiment 7 (drawn to scale). (a) The widths are 0.5°, 3.5°, 6.5°, and 9.5° of visual angle, from top to bottom. (b) The heights are 0.5°, 3.5°, 6.5°, and 9.5° of visual angle, from left to right.

Results

The overall accuracy for the secondary task was 98.04% (97.34% with-motion condition, 98.74% no-motion condition). The overall accuracy for the memory task was 82.07% (79.67%, 80.36%, and 86.18% for width, height, and binding condition, respectively). The descriptive data of the WM task are presented in Table 1. 

The corrected recognition for each condition is shown in Figure 14. The two-way ANOVA on the corrected recognition revealed a significant main effect of task load, F(1, 23) = 23.73, p < 0.001, \(\eta _p^2\; = 0.51\), BF = 10715.27, suggesting that performance was significantly better under the no-motion condition than under the with-motion condition. The main effect of memory condition was significant, F(2, 46) = 11.40, p < 0.001, \(\eta _p^2\; = 0.33\), BF = 1988.01. Post hoc contrasts (Bonferroni-corrected) revealed that performance was significantly better under the binding condition (0.72) than under the width (0.59) and height (0.61) conditions. The interaction between task load and memory condition was not significant, F(2,46) = 0.52, p = 0.60, \(\eta _p^2\; = 0.02\). Confirming this null effect, the BF for the task load × memory condition interaction was only 0.17, favoring a reduced model (i.e., without interaction). 

Figure 14.

 

The corrected recognition under each condition for Experiment 7. Error bars stand for 95% CIs.

Discussion

Experiment 7 lowered the memory load and presented fewer and more distinctive feature values; the overall WM performance was comparable or even better than the performance of Experiment 4 (separable feature), and participants could categorize the features. However, in line with Experiments 1, 2, 3, and 5, we still found that the consumption of object-based attention did not lead to a selective binding impairment. Therefore, neither feature categorization nor task difficulty could explain the findings on integral-feature bindings. 

General discussion

The current study investigated the generality and the reality of the object-based attention hypothesis of binding in WM. We predicted that integral features would be processed as an integrated unit without the help of extra object-based attention. To test this, we examined whether the object-based attention hypothesis of binding applied to integral-feature bindings (generality), and those results enabled us to then check whether previously revealed results had been due to an artifact of the testing process (reality). In Experiments 1, 2, 3, 5, and 7, our prediction was supported: A secondary task consuming object-based attention did not selectively impair participants’ binding performance. Our experiments showed that this absence of selective binding impairment was not due to an invalid secondary task (Experiment 4), tapping the incorrect type of attention (Experiment 6), the feasibility of feature categorization (Experiment 7), or poor task performance (Experiment 7). Overall, these results suggested that the object-based attention hypothesis did not extend to integral-feature bindings and that previously reported selective binding impairments were not an artifact of the testing process. 

Although any one set of data alone could be explained in other ways, the fact that all were derived from one hypothesis and tested in a number of different circumstances should lend them more weight when taken together than any individual finding would have on its own. It was hence critical to vary the secondary tasks as widely as possible in order to maximize the gain from converging operations. We previously developed three different secondary tasks (mental rotation task in Experiments 1, 2, and 3; transparent motion task in Experiment 4; and Duncan's object-feature report task in Experiment 5) (cf. Shen, Huang, & Gao, 2015) to test the object-based attention hypothesis of binding. In line with Shen et al. (2015), the current study and that by He et al. (2020) used a transparent motion task and mental rotation task to tax object-based attention. Each secondary task on its own might allow other interpretations. For example, there is no direct evidence showing that object-based attention plays a key role in mental rotation, so we can only deduce that object-based attention is critical in maintaining the representations during mental rotation according to previous studies (e.g., Barnes et al., 2001; Hyun & Luck, 2007; Matsukura & Vecera, 2009; Matsukura & Vecera, 2011; Prime & Jolicoeur, 2010; Woodman & Vecera, 2011). However, the fact that all of the three distinct tasks were derived as independent predictions from the same hypothesis should allow them, if confirmed, to strengthen the hypothesis more than any could individually. Because from the three tasks we reached the same conclusion as to the role of object-based attention in a set of experiments (cf. Z. Gao et al., 2017; He et al., 2020; Shen et al., 2015), we argue that object-based attention was the only feasible interpretation for the observed findings, and the manipulation of object-based attention in our study was effective. 

Taking the current study and previous studies together, we argue that there are dissociated mechanisms in retaining bindings in WM. In agreement with previous studies on separable-feature binding (e.g., Allen et al., 2006; Johnson et al., 2008), we demonstrated that domain-general attention did not play a key role in retaining integral-feature bindings in WM, but object-based attention was critical. However, in contrast to other separable-feature binding studies (e.g., Z. Gao et al., 2017; He et al., 2020; Shen et al., 2015) and Experiment 4 of the current study, we found evidence that object-based attention was not critical in retaining integral-feature binding. These findings are broadly in line with the assumptions of FIT (Treisman & Gelade, 1980), suggesting that integral features may be conjoined automatically and follow a unitary processing fashion, but separable features are processed separately and require attention for integration. The dissociated attentional mechanisms may be rooted in their underlying neural mechanisms. It has been suggested that separable features are processed through largely independent neurons during perception, whereas integral features are processed via a group of overlapping neurons (e.g., Cant et al., 2008; Garner, 2014). It is possible that the binding formed via neural synchronization or convergence (Hommel & Colzato, 2009) is effortless for overlapped neurons, resulting in a holistic processing of integral features and an analytic processing for separable features in not just perception but also WM (Attneave, 1950; Jones & Goldstone, 2013; Shepard, 1964; for a review, see Kemler Nelson, 1993). 

The current study helps rule out the possibility that previously observed selective binding impairments were some artifact of the testing process. Using similar settings for our memory array and dual tasks, we demonstrated in Experiment 4 that object-based attention tasks selectively disrupted separable-feature bindings (Z. Gao et al., 2017; He et al., 2020; Lu et al., 2019; Shen et al., 2015), and such results did not occur with integral-feature bindings (Experiments 1, 2, 3, 5, and 7), which should have disappeared according to the nature of integral features. These findings together suggest that the testing procedure used was sensitive in revealing the role of object-based attention. Therefore, we argue that suggestions of the pivotal role of object-based attention in retaining separable-feature bindings are reliable. On the other hand, we had to consider an alternative: Both the tested width and height were processed as a type of filled rectangle that had been employed in the current study; thus, it is not surprising to see the observed null effects reported in the current study. Here we consider that this alternative does not hold. Particularly, we used a thin line (1 pixel thick, 0.03°) to convey the value of width and height, which is a common and well-accepted manner in the research of length perception (e.g., Avery & Day, 1969; Brosvic & Cohen, 1988; Charras & Lupiáñez, 2009; Charras & Lupiáñez, 2010; Lipshits & McIntyre, 1999). Moreover, our post hoc interview suggested that no participants subjectively treated these lines as rectangles (see footnote 2). To be safe, however, we suggest that further study is necessary to verify these findings—for example, by using an unfilled rectangle with a gap between the constituent elements⁴ or by testing a new type of integral feature pair (e.g., hue and color saturation). 

The current findings suggest that the relationship between feature combinations has an impact on WM processing. Because WM studies have suggested that WM and perception share similar processing manners (e.g., the sensory recruitment hypothesis in visual WM) (Ester, Serences, & Awh, 2009; Z. Gao, Li, Yin, & Shen, 2010; Harrison & Tong, 2009), and because the way we encode information in visual perception determines how perceptual information is encoded into visual WM (T. Gao, Gao, Li, Sun, & Shen, 2011; Z. Gao & Bentin, 2011; Z. Gao et al., 2010; Shen, Tang, Wu, Shui, & Gao, 2013; Yin, Zhou, Xu, Liang, Gao, & Shen, 2012), it is possible that there are dissociated attention mechanisms for retaining bindings composed of separable features and for retaining bindings composed of integral features. However, although studies on perception have revealed distinct processing mechanisms for integral and separable features (e.g., Garner, 2014; Garner & Felfoldy, 1970), only two previous studies have paid attention to this revelation with respect to WM. Researchers found that objects constituted by integral features were stored stably as one integrated unit in WM, whereas objects constituted by separable features were stored in an unstable manner that led to independent storage failures (Bae & Flombaum, 2013; Fougnie & Alvarez, 2011). The current study is the third to investigate the processing mechanisms of integral features in WM. Our results are in line with those of the previous perceptual studies and the two aforementioned WM studies. Moreover, the current findings add new evidence suggesting that the processing mechanisms in perception have a considerable influence over the processing in WM. This highlights the need to explore WM mechanisms from the perspective of feature relations. 

It is worth noting that, although we did not reveal a selective impairment to the integral-feature bindings by adding a secondary task consuming object-based attention, we did observe a significant impairment led by the secondary task. Therefore, retaining integral-feature bindings in visual working memory requires the help of central attention, which is consistent with Zokaei, Heider, and Husain (2014). This observation implies that integral-feature bindings in WM may fall apart when central attention is consumed, which would lead to correlated fall-apart traces between the integral feature pair (for an example, see Fougnie & Alvarez, 2011). 

The current study also sheds light on WM models in general and the episodic buffer concept in particular. Baddeley (2000) added the episodic buffer as a newly added storage component of his original multiple-component model of WM (Baddeley, 1986; Baddeley & Hitch, 1974). The original multiple-component model posited that the WM system consists of a central executive supervisory component controlling information flow to and from two slave subsystems for storage (i.e., phonological loop and the visuospatial sketchpad). However, there were a few limitations to this model (for a review, see Baddeley, 2000). One critical limitation is the binding problem in WM—for example, how information from the phonological and visuospatial subsystems gets bound into one unit. The limited-capacity episodic buffer was posited to explain how the subsystems’ information gets linked into integrated units of temporal, visual, spatial, and verbal information (Baddeley, 2000). In this model, the episodic buffer is a slave of the central executive, which sends the buffer information from the phonological and visuospatial subsystems. With the help of the central executive, the episodic buffer constructs and maintains the binding of multiple-coding information in WM. This would make the episodic buffer an active buffer in which domain-general attention is assumed to play a key role. However, after extensive explorations, Baddeley and his colleagues determined that the binding in WM did not require extra domain-general attention for the constituent single features (Allen, 2015; Allen et al., 2006; Allen et al., 2009; Allen et al., 2012). 

Currently, Baddeley and colleagues have given up the idea of an active episodic buffer and suggest that the episodic buffer only passively receives the bindings, which are formed elsewhere (e.g., visuospatial sketchpad) (Baddeley, 2012; Baddeley, Allen, & Hitch, 2010; Baddeley et al., 2011). However, based on the findings that object-based attention plays a key role in binding maintenance in WM (Z. Gao et al., 2017; Lu et al., 2019; Shen et al., 2015), the episodic buffer does not seem to be a slave buffer under the control of the central executive; rather, like a visuospatial sketchpad, it is an independent storage buffer supported by object-based attention. Furthermore, in the hierarchical structure between the episodic buffer and the phonological and visuospatial subsystems, the latter two subsystems serve as lower level storage buffers and have direct access to the episodic buffer (cf. Z. Gao et al., 2017). 

The current findings appear to reconcile the two different views of the episodic buffer. Specifically, we suggest that the integral-feature binding is processed in a way suggested by Baddeley et al. (2011); that is, the integral-feature binding occurs in a specific storage buffer (the visuospatial sketchpad, in the current study) before entering the episodic buffer. However, separable-feature binding is processed differently, in the manner suggested by Z. Gao et al. (2017), when the constituent elements of the binding come from different sources. 

Finally, the current findings, together with those of Bae and Flombaum (2013), have certain implications to systems design. It is well accepted that engineers and systems designers should pay attention to the different perceptual mechanisms of feature combinations when designing, for example, air traffic control systems (Wickens et al., 2015); care should be taken to use proper feature combinations to match task requirements. We suggest that this principle can be generalized from perception to WM. Because integral-feature binding retention is more stable and effortless compared to separable-feature binding in WM, designers should adopt integral features to ensure the conveyance of binding-related information when the systems are in operation. 

Acknowledgments

Supported by the Science and Technology Innovation 2030 Program (2018AAA0101605), National Key R&D Program of China (2019YFB1600504), National Natural Science Foundation of China (31771202), and Key Program of Natural Science Foundation of Zhejiang Province (LZ20C090001). 

Commercial relationships: none. 

Corresponding authors: Zaifeng Gao; Mowei Shen. 

Email: [e-mail]; [e-mail]. 

Address: Department of Psychology and Behavioral Sciences, Zhejiang University, Zhejiang, China. 

Footnotes

1  Although Duncan's object-feature report task is a classic task for evaluating object-based attention (Duncan, 1984), the composed stimuli (rectangular and line) in this task are similar to the current memorized items. To avoid the secondary task stimuli overwriting the representation of memorized items (cf. Ueno, Mate, Allen, Hitch, & Baddeley, 2011), we did not use the Duncan task as the secondary task.

Footnotes

2  No participant treated this line as a rectangle according to post hoc interviews with the participants. Moreover, there are three considerations with regard to displaying black lines (isolated single features) instead of rectangles as probes. First, such a setting allows direct comparison with previous binding studies on this issue. Second, this setting encourages participants to just memorize target features. Third, this setting avoids potential contamination from task-irrelevant information. For example, according to the nature of integral features, even when storing only height, the height judgments might be better if the full rectangle is shown rather than just the height information.

Footnotes

3  Participants in Experiment 1 could have retained the same information between the feature and binding blocks; therefore, even though the memory array settings are similar for Experiments 1 and 4, the nature of memorized content may not be comparable. To this end, we did not compare performance between Experiments 1 and 4, although a similar result would be obtained if the mixed ANOVA reported here were performed.

Footnotes

4  We thank an anonymous reviewer for this suggestion.