Object-based implicit learning in visual search: Perceptual segmentation constrains contextual cueing

Abstract

Abstract

Abstract:

Abstract  In visual search, detection of a target is faster when it is presented within a spatial layout of repeatedly encountered nontarget items, indicating that contextual invariances can guide selective attention (contextual cueing; Chun & Jiang, 1998). However, perceptual regularities may interfere with contextual learning; for instance, no contextual facilitation occurs when four nontargets form a square-shaped grouping, even though the square location predicts the target location (Conci & von Mühlenen, 2009). Here, we further investigated potential causes for this interference-effect: We show that contextual cueing can reliably occur for targets located within the region of a segmented object, but not for targets presented outside of the object's boundaries. Four experiments demonstrate an object-based facilitation in contextual cueing, with a modulation of context-based learning by relatively subtle grouping cues including closure, symmetry, and spatial regularity. Moreover, the lack of contextual cueing for targets located outside the segmented region was due to an absence of (latent) learning of contextual layouts, rather than due to an attentional bias towards the grouped region. Taken together, these results indicate that perceptual segmentation provides a basic structure within which contextual scene regularities are acquired. This in turn argues that contextual learning is constrained by object-based selection.

Introduction

Natural environments provide a manifold and complex input to the visual system. In order to ensure that goal-relevant aspects of a scene are registered and processed by capacity-limited attentional mechanisms, the large amount of available visual information must be appropriately structured. Several processes can realize structure: On the one hand, perceptual grouping can support the integration of fragmentary parts into coherent units (objects), which may then be attended for further processing (e.g., Driver, Davis, Russell, Turatto, & Freeman, 2001, for review). On the other hand, previously acquired knowledge may also facilitate attentional selection through scene memory (e.g., Chun, 2000, for review). The aim of the present study was to investigate whether attentional selection is affected by the structure provided conjointly by perceptual grouping and memory for visual scenes. 

A number of studies have shown that perceptual grouping can integrate cluttered parts of a visual scene into coherent objects, while segregating a given object from the background and from other objects. For instance, it has been shown that grouped component parts may be detected faster than their isolated features (Pomerantz, Sager, & Stoever, 1977; Rensink & Enns, 1995), and search for an integrated, grouped target configuration is more efficient than search for corresponding fragmentary parts (Conci, Müller, & Elliott, 2007a, 2007b). On this view, perceptual grouping establishes coherent wholes on the basis of a number of Gestalt principles, such as closure (e.g., Donnelly, Humphreys, & Riddoch, 1991) or collinearity (e.g., Jingling & Tseng, 2013). These integrated objects then constrain subsequent attentional processing and serve as basic units for selection. 

Besides perceptual structure provided by grouping, scene memory may also facilitate attentional selection. In particular, the covariation of objects with other objects provides a rich source of contextual information for efficient processing (Oliva & Torralba, 2007, for review). For example, Biederman, Mezzanote, and Rabinovitz (1982) have shown that detection of a target object (e.g., a car) is facilitated if it is presented within a typical context (e.g., a street scene), compared to a surround that is not typically related to the target (e.g., a kitchen). Thus, both perceptual grouping and contextual scene memory may establish a relational structure supporting attentional guidance. 

Contextual relations between objects may not only facilitate target detection on the basis of object identities, but can also occur for invariant spatial relations between objects in visual search. For instance, Chun and Jiang (1998) have shown that search is facilitated when a spatial layout of nontargets is repeatedly paired with a given target location—a finding referred to as “contextual cueing.” In their paradigm, observers were typically asked to search for a target T among eleven nontarget Ls and to indicate the orientation of the target shape (to either the left or the right). Displays differed in that targets either appeared within “old” or “new” layouts: Old configurations always presented the target within the same layout of nontargets, whereas new configurations presented novel nontarget arrangements on every trial. Consequently, comparisons between old and new context conditions indicate whether invariant spatial layout (i.e., the repetition of a given search layout throughout the experiment) influences target detection. The results showed that repeated (old) spatial arrangements lead to a benefit in mean reaction time (RT), as compared to new spatial layouts: the contextual-cueing effect. Since observers were not able to explicitly distinguish repeated displays from novel arrangements (in a recognition test at the end of the experiment), this finding was interpreted in terms of a mechanism that implicitly encodes the spatial associations between display items, guiding visual attention and facilitating search. 

In order to investigate whether the relational structure provided by contextual learning is affected by perceptual grouping, Conci and von Mühlenen (2009) combined the contextual cueing paradigm with a systematic grouping manipulation of the search items. In a baseline condition, reliable contextual-cueing effects were obtained for search displays that were presented in random arrangement (i.e., with a nonsystematic layout) of search items, replicating previous studies (e.g., Chun & Jiang, 1998). However, no contextual cueing was observed when four nontarget Ls were arranged and oriented such that they appeared at neighboring positions forming a square grouping. This lack of contextual cueing was manifest even though the square always appeared at the same (global) location in a given repeated display and was thus predictive of the invariant target location. Subsequent experiments replicated this effect for several types of regular groupings—that is, there was no evidence of contextual cueing when search displays contained closed squares or symmetrically arranged cross shapes. Moreover, a comparable reduction of contextual cueing was observed when subsets of items in repeated displays were grouped by similarity (e.g., same color or same size; Conci & von Mühlenen, 2011). A potential account for this absence of contextual cueing is that grouping reduces the amount of available information for guiding attention to the target: The grouping together of individual items effectively reduces the number of separate objects in the scene context, with this reduced variability of the invariant context in turn decreasing the predictability of the target location. 

This “contextual variability” account might well explain the observed reduction in contextual cueing for layouts that contain grouped items. However, contextual cueing of the target location has been shown to primarily rely on the local context of only a few neighboring nontarget items (Brady & Chun, 2007). On this view, a regular grouping of four items (like the square in Conci & von Mühlenen, 2009) would not be expected to interfere with cueing, as the remaining invariant context would still allow a reliable prediction of the target location. An alternative account might be that contextual cueing is in itself “object-based.” On this view, the observed reduction of contextual cueing in search displays that contain a regular group of items might be due to contextual learning being biased towards the grouped region, with reliable contextual cueing occurring within, but not outside, of the grouped objects. The reduction in contextual cueing observed by Conci and von Mühlenen (2009) would then be due to the fact that the target was never presented within, but only outside, the grouped region. 

It should be noted that the presence of grouped regions, in addition to affecting the strength of contextual cueing, may also modulate overall target detection performance. Consistent with this, it has been shown that the deployment of attention is modulated by salient configurations, with a preference for processing the grouped region (e.g., see Conci et al., 2007a, 2007b; Donnelly et al., 1991; Kimchi, Yeshurun, & Cohen-Savransky, 2007; Riccardielli, Bonfiglioli, Nicoletti, & Umilta, 2001; Yeshurun, Kimchi, Sha'shoua, & Carmel, 2009, for findings on global shape precedence). For instance, Yeshurun et al. (2009) demonstrated that a square grouping captures attention and facilitates detection of a target located inside the square. Accordingly, perceptual grouping might provide units for attentional selection, and as a result, contextual cueing may occur only for the attended units. Alternatively, perceptual grouping might have independent effects on attention and contextual cueing—that is, salient objects primarily attract attention, but learning of regularities may at the same time also occur for more inconspicuous units or parts of a display layout. 

The present experiments were designed to further explore the interaction between perceptual grouping and contextual cueing in order to decide between two alternative, contextual variability and object-based cueing accounts and to determine how the structure provided by the context interacts with attention. To this end, in four experiments, we presented search displays with a square grouping that covered a relatively large subregion of the search display: Approximately half of the display was enclosed within the grouped region (and half outside), permitting a systematic comparison of contextual cueing for targets located within versus outside the boundaries of the grouped region. 

Based on previous findings (Conci & von Mühlenen, 2009, 2011), we expected that contextual cueing would be overall reduced given that grouping, or perceptual structure, essentially reduces the amount of variability of the invariant context—that is, it limits the amount of learnable context. However, if true, one would expect the reduction of contextual cueing to occur equally for all targets, whether they appear inside or outside the square's boundaries. An alternative explanation might be that contextual cueing is object-based—that is, primarily confined to segmented regions. In this case, reliable contextual cueing should primarily occur when the target is located inside, rather than outside, the grouped square. 

General method

The present experiments employed a variant of the contextual-cueing paradigm (Chun & Jiang, 1998) in which search displays were presented that contained a group of four nontarget items placed at the corners of an imaginary square, and the target could be presented either inside or outside the square's boundaries. Experiments 1, 2, and 3 were essentially the same, except that the grouping strength of the square arrangement varied systematically across the experiments. Experiment 4 was performed to examine whether the sudden removal of the square grouping, following initial contextual learning with the grouping present, would affect the pattern of cueing effects. Figure 1 presents example display layouts. 

Figure 1

 

Example search displays: Each display contained a target (T), pointing to the left or right, and a set of nontargets items (Ls), four of which induced a regular, square-like grouping (as indicated by the dashed lines). For square displays (A), four nontarget items were arranged to form a collinear square grouping (Experiment 1). Nonsquare displays (B) again contained a square-like form, but with the square elements pointing toward the center of the square configuration (Experiment 2). For random-square displays (C), four nontargets were presented in square-like form, but this time with random orthogonal orientations (Experiment 3). Finally, Experiment 4 presented square displays (A) for the initial part of the experiment, followed by a transfer phase where the square was removed from the displays (D). Note that the dashed lines were not presented in the actual experiments (see also Figure 4).

Participants

Ten different observers with normal or corrected-to-normal visual acuity participated in Experiment 1 (mean age = 31.3 years), Experiment 2 (mean age = 27.0 years), Experiment 3 (mean age = 23.4 years), and Experiment 4 (mean age = 28.8 years). Observers were paid eight Euro per hour for participating in the experiment. Participants gave their informed consent prior to performing the experiment. The experimental procedure was approved by the ethics committee of the Department of Psychology at LMU Munich, in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki). 

Apparatus and stimuli

The experiment was conducted with an IBM-PC compatible computer (Dell Inc., Round Rock, TX) using Matlab routines and Psychophysics Toolbox extensions (Brainard, 1997; Pelli, 1997). Stimuli subtended 0.7° × 0.7° and were presented in gray (8.5 cd/m²) against a black (0.02 cd/m²) background on a 17-in. CRT monitor. A search display always consisted of twelve items, one target T and eleven nontarget Ls, which were positioned within the cells of an invisible 8 × 6 matrix (cell size 2.9°; see Figure 2). Within each cell, the positions of the stimuli were randomly jittered horizontally and vertically in steps of 0.1° within a range of ±0.6°. The target was a T shape rotated 90° either to the left or to the right with random probability. Nontargets were L shapes rotated randomly in one of four orthogonal orientations. For each search display, 4 of the 11 nontarget items were placed at the corners of a square (i.e., the corner positions in a 5 × 5 submatrix, subtending 11.6° × 11.6°) without item jitter. The target T and the remaining 7 (of the 11) nontarget Ls were then randomly positioned within the remaining cells of the display matrix, comparable to the procedure in the “standard” contextual cueing paradigm. 

Figure 2

 

Example search display, depicting the invisible 8 × 6 matrix of possible item locations. Each square grouping covered an area of 5 × 5 cells. The cells with gray shading were excluded as potential target locations because they were either only available for target-on (central positions), or for target-off conditions (peripheral positions).

In each experiment, the positioning of the square (5 × 5 matrix) was varied randomly within the larger display (8 × 6 matrix), with all possible locations being equally likely. Two types of displays were generated: target-on displays presented the target within the boundaries of the 5 × 5 square (i.e., at one of the 21 cells inside the square, including the square's border, but excluding the square's corners; see Figure 2). In target-off displays, the target was presented at a position outside the square boundaries (i.e., at one of the 23 cells outside the square). Thus, the number of possible target locations was approximately the same for target-on and target-off displays. Furthermore, to control for possible systematic differences in target eccentricity between conditions, target positions were further restricted such that they had to be possible for both target-on and target-off displays. This procedure eliminated the four corner positions and the four central positions of the display as target locations (see gray shaded matrix cells in Figure 2). The remaining seven nontargets that did not form the square were distributed randomly across all available display positions. Thus, on the whole, the item density surrounding the target did not differ between target-on and target-off displays. Example displays are shown in Figure 1. 

Throughout the experiments, we varied the strength of grouping of the nontarget square-arrangements: In Experiment 1, four L shapes were rotated such that the four items formed a closed, collinear square (Figure 1A). In Experiment 2, four Ls were presented in square arrangement such that each L corner pointed towards the center of the configuration, forming a symmetrical, nonclosed (“nonsquare”) cross shape (Figure 1B). In Experiment 3, the four Ls were arranged in regular square-arrangement but with each item pointing in a random orthogonal direction (“random square”). Thus, there could be partial groupings (e.g., collinear line segments between neighboring items), but the overall square configuration was neither closed nor symmetric (Figure 1C). The final experiment (Experiment 4) used the same square displays as Experiment 1 for an initial training phase (Figure 1A), but in a subsequent transfer phase the four Ls defining the square were removed, leaving only the target and the seven randomly distributed nontargets (Figure 1D). 

Trial sequence

Each trial started with the presentation of a central fixation cross for 500 ms. The fixation cross was followed by the search display, to which participants responded with a speeded response via mouse keys. The task was to search for an oriented T among Ls and to decide as quickly and accurately as possible whether the T was oriented to the left or to the right. Displays remained on-screen until a response was recorded. In case of an erroneous response, feedback was provided by an alerting sign (“–”) presented for 1000 ms at the center of the screen. The intertrial interval was 1000 ms. 

Design and procedure

A three-factors within-subjects design was used. The independent variables were context, target, and epoch. Context had two levels, old and new. For the old-context condition, the arrangement of nontarget items was the same on every presentation. In the new-context condition a new, random arrangement of nontarget items was generated on every presentation. To rule out location probability effects, the target appeared equally often at 24 possible locations throughout the experiment. The orientation of the target was determined randomly for each trial while the orientations (and identities) of the nontarget items were preserved for the old-context condition. The second variable, target, also had two levels, on and off. In target-on conditions, the target was presented within the boundaries of the 5 × 5 square (e.g., in Figure 1A, C). In the target-off condition, the target was positioned outside the square boundaries (e.g., in Figure 1B). Note, that the old-context condition preserved the position of all nontarget items, including the square. Thus, for a given old context, the square would always be presented at the same location and should thus, in principle, be predictive of the target location. For new-context trials, the square location was chosen randomly but with the constraint that the square either included or excluded a given target location depending on the variable target (on or off, respectively). Finally, the third variable, epoch, simply divided the experiment into six consecutive bins that allowed possible learning effects to be assessed over the course of the experiment. Note that in Experiment 4, epochs 1 to 5 constituted the training phase, followed by the transfer phase (epoch 6) in which the four Ls forming the square were removed from the display (as described above). During transfer, displays were generated just as for the training phase, however, with the items that defined the square being removed and the respective positions remaining empty (for both old and new contexts), so as to assess whether the previous square–target relation still affected performance after removal of the square. 

At the beginning of each experiment, participants completed one block of 24 randomly generated practice trials to become familiarized with the task. All subsequent experimental blocks contained the same 12 old-context displays and 12 newly generated new-context displays in randomized order. In each block, old- and new-context displays were further subdivided into target-on and target-off displays (i.e., six old- and six new-context target-on displays, and six old- and six new-context target-off displays). There were 30 blocks in the experiment, with 720 experimental trials in total. 

Recognition test

After completing the search task, participants were asked to perform a recognition test. They were informed that certain display configurations had been repeated throughout the experiment and they had to decide whether a given presented display had previously been shown or not. A total of 24 displays were shown to the participants. Half of them were old-context displays that were used in the experiment; the other half were newly generated displays with random item arrangement. Note that in Experiment 4, the final recognition test presented displays from the initial training set (i.e., including the square). The trial sequence was identical to the search task, except that no error feedback was given. Participants had to indicate whether the display was new or whether they had seen it before. Nonspeeded responses were recorded via left (new) and right (old) mouse keys. 

Experiments 1, 2, and 3

Experiments 1, 2, and 3 investigated whether the segmentation of a square shape influences the ability to learn the context of a repeatedly encountered search array. Therefore, old- and new-context displays were always presented with a square grouping that covered approximately half of the search display, and the target could appear either inside or outside the square's boundaries. The experiments were near-identical, except for a systematic variation of grouping strength: Experiment 1 presented a (salient) closed and collinear “square” arrangement. Next, Experiment 2 presented a less salient “nonsquare” (i.e., cross-shaped) arrangement that was symmetrical but lacked closure. In Experiment 3, the grouping was a relatively subtle “random-square” configuration that was not defined by closure or symmetry (see Figure 1A through C for example displays). 

Results

Search task

Mean error rates were calculated for each participant and variable combination, separately for each experiment. The overall error rates were very low: 1.35%, 1.68%, and 1.47% in Experiments 1, 2, and 3, respectively. Examination of the error rates by repeated-measures analyses of variance (ANOVAs) with the factors context (old, new), target (on, off), and epoch (1–6) revealed no significant effects for Experiments 1 and 2. In Experiment 3, there was a significant main effect of target, F(1, 9) = 5.45, p < 0.05: slightly fewer errors were made for target-off than for target-on conditions (1.3% vs. 1.7%). 

Individual mean RTs were computed for each variable combination excluding error responses and RTs longer than 3 s. Less than 1.5% of all trials were excluded by this outlier criterion. Figure 3 presents the mean correct RTs, averaged across participants, as a function of epoch for Experiments 1 (panel A), 2 (panel B), and 3 (panel C), with separate graphs for target-on (left) and target-off (right) conditions. For each experiment, mean correct RTs were subjected to a three-way ANOVA with main terms for context (old, new), target (on, off), and epoch (1–6). 

Figure 3

 

Mean RTS (with SE bars) as a function of epoch in the target-on (left) and target-off (right) display conditions of Experiments 1 (square groupings), 2 (nonsquare groupings), and 3 (random-square groupings). Filled and unfilled symbols correspond to old- and new-context conditions, respectively.

Figure 4

 

Mean contextual-cueing effects (with SE bars) for target-on and target-off conditions averaged across epochs 1 to 6 (black and white bars, respectively) in Experiments 1 (square), 2 (nonsquare), and 3 (random-square). The dashed red lines present the mean attentional capture effect by the square, that is, the on-target benefit for baseline displays (i.e., the RT-difference in ms between target-off and -on conditions in new-context display layouts). For each experiment, an example display is shown.

In Experiment 1 (square), this analysis revealed significant main effects of context, F(1, 9) = 12.45, p < 0.007, target, F(1, 9) = 117.37, p < 0.001, and epoch, F(5, 45) = 7.84, p < 0.001. Old-context trials were on average 84 ms faster than new-context trials, and targets presented within the square boundaries were detected 196 ms faster than targets outside the square. The main effect of epoch was due to search becoming faster with increasing epoch (RTs were 138 ms faster in epoch 6 than in epoch 1). Interestingly, there was also a significant interaction between context and target, F(1, 9) = 12.25, p < 0.007, indicating that the contextual-cueing effect was stronger when the target was presented on the square (on-targets), as compared to off-the-square positions (off-targets; 135 ms vs. 29 ms, respectively). Follow-on t tests revealed contextual cueing to be reliable only for target-on conditions, t(9) = 5.54, p < 0.001, but not for target-off conditions, t(9) = 0.91, p = 0.4—that is, contextual cueing was limited to the regions enclosed by the square. The context by epoch interaction did not reach significance (p > 0.2), indicating that observers already showed a robust contextual-cueing effect in epoch 1. In order to determine in which block of the initial epoch significant contextual cueing occurred first, RTs for old and new contexts were compared for each block. The first significant difference emerged in block 4, t(9) = 2.4, p < 0.04, which is in line with findings of fast contextual learning in previous studies (e.g., Chun & Jiang, 1998; Conci & von Mühlenen, 2009). A similar pattern of rapid contextual learning was also observed in the subsequent experiments. The three-way interaction was not significant (p > 0.8). 

The RT analysis for Experiment 2 (nonsquare) revealed a similar pattern of results: There again were significant main effects of context, F(1, 9) = 12.33, p < 0.007, target, F(1, 9) = 85.77, p < 0.001, and epoch, F(5, 45) = 5.66, p < 0.001. RTs were on average 47 ms faster for old compared to new contexts, and search became faster with increasing epoch (RTs were 163 ms faster in epoch 6 than in epoch 1). The main effect of target was due to targets presented inside the non-square being detected 179 ms faster than targets outside the grouping. There was also, again, a significant interaction between context and target, F(1, 9) = 8.21, p < 0.02: contextual cueing was evident only when the target was presented “on,” but not when it was presented “off,” nonsquare positions, target-on: 115 ms, t(9) = 5.16, p < 0.002; target-off: −22 ms, t(9) = 0.67, p = 0.5. This replicates Experiment 1 in showing that contextual cueing was limited to the region within the grouping. 

The RT analysis for Experiment 3 (random-square) also revealed a pattern of contextual cueing consistent with Experiments 1 and 2. The ANOVA yielded significant main effects of target, F(1, 9) = 22.89, p < 0.001, and epoch, F(5, 45) = 5.18, p < 0.001, though this time no main effect of context (p > 0.11). Targets were detected 127 ms faster overall when they were presented inside, rather than outside, the random-square. In addition, search became faster with increasing epoch (RTs were 109 ms faster in epoch 6 than in epoch 1). Furthermore, the target × epoch interaction was significant, F(5, 45) = 2.73, p < 0.05, due to the decrease in RTs from epoch 1 to 6 being greater for target-off (144 ms) than for target-on conditions (74 ms). Importantly, as in the previous experiments, there was a significant interaction between context and target, F(1, 9) = 30.02, p < 0.001, with contextual cueing occurring only when the target was presented inside the random-square, benefit of 168 ms, t(9) = 7.17, p < 0.001; by contrast, there was a marginally significant cost (rather than a benefit) for targets presented at “off” positions, cost of −84 ms, t(9) = 2.07, p = 0.07. Thus, as in Experiments 1 and 2, contextual cueing was limited to the segmented region, with a comparable modulation for square, nonsquare, and even for random-square groupings. 

Recognition test

Overall, the mean accuracy in the recognition tests was 53%, 52%, and 46% for Experiments 1, 2, and 3, respectively. In Experiment 1 (square), participants correctly identified old patterns in 51.6% of all trials; however, without their hit rate differing significantly from their false-alarm rate of 57.5%, t(9) = 1.04, p = 0.3. The same was true for Experiment 2 (nonsquare), with hit and false-alarm rates of 42.5% and 48.3%, respectively, t(9) = 1.07, p = 0.3, and for Experiment 3 (random square), with hit and false-alarm rates of 55.8% and 48.3%, respectively, t(9) = 1.05, p = 0.3. Overall, these nonsignificant differences indicate that participants were not aware that some displays were repeated. 

Effects of grouping strength

In a subsequent step, we examined the influence of grouping strength on performance, directly comparing search for displays that contained a square (Experiment 1), a nonsquare (Experiment 2), or a random-square (Experiment 3) configuration. 

First, mean contextual-cueing effects (averaged across all six epochs) were entered into a mixed-design ANOVA with experiment (1, 2, 3) as between- and target (on, off) as within-participant term. This analysis revealed a significant main effect of target, F(1, 27) = 46.17, p < 0.001, but no effect of experiment (p > 0.3). As illustrated in Figure 4, across all experiments, contextual cueing was stronger for target-on than for target-off conditions (140 ms and −26 ms, respectively)—demonstrating that contextual cueing was modulated by the square configurations, with comparable region-based modulations whatever the strength of the particular grouping. 

Next, based on the assumption that the strength with which display segmentation mechanisms parse a cluttered scene affects where attention is engaged, we examined whether attentional capture by the segmented region differed as a function of grouping strength across Experiments 1, 2, and 3. As an index of attentional capture, we calculated the difference in the processing of targets presented inside versus outside a given configuration (e.g., Kimchi et al., 2007; Yeshurun et al., 2009)—importantly, for new-context, baseline displays only, as these are not affected by contextual learning. A mixed-design ANOVA with experiment (1, 2, 3) and target (on, off) as between- and within-participants terms, respectively, yielded a significant main effect of target, F(1, 27) = 28.86, p < 0.001, which was due to an overall RT benefit of 85 ms for target-on relative to target-off conditions (the main effect of experiment was again not significant, p > 0.3). In addition, the experiment × target interaction was significant, F(2, 27) = 7.54, p < 0.005, with an on-target benefit (see red dotted lines in Figure 4) manifesting only with the square (143 ms, t[9] = 5.01, p < 0.001) and nonsquare (110 ms, t[9] = 4.95, p < 0.001) configurations, but not with the random-square configuration (1 ms, t[9] = 0.006, p = 0.9). 

Taken together, these two result patterns show that attentional capture effectively vanished as grouping strength was reduced (replicating previous findings), whereas contextual cueing continued to be modulated even by relatively subtle groupings. 

Discussion

Experiments 1, 2, and 3 replicated previous findings on contextual cueing in visual search: Observers were, in general, faster in detecting a target within an old (i.e., repeated) search layout, as compared to a novel item arrangement. In agreement with Chun and Jiang (1998), these results indicate that a given learned contextual layout can be used to guide attention more quickly (and more efficiently) to the target location. 

However, all three experiments also showed that contextual cueing strongly depended on the spatial relation of the target to the square grouping: Reliable contextual-cueing effects occurred only when the target was presented within the boundaries of the segmented region (these effects were in the range of 115 to 168 ms). By contrast, no reliable cueing effects were evident when the target was presented outside the groupings (29 to −84 ms; see Figure 3). Thus, the region-based modulation of contextual cueing occurred independently of grouping strength, with a comparable pattern in all three experiments (see Figure 4). The difference between targets positioned on and off the segmented region occurred despite controlling for target eccentricity (see General Method and Figure 2), and despite the fact that contextual cueing in the standard paradigm is reliably observed for targets presented in peripheral vision (van Asselen & Castelo-Branco, 2009). In sum, this pattern of results confirms that contextual cueing is modulated by perceptual segmentation (Conci & von Mühlenen, 2009, 2011), however, with the modulation being independent of the strength of a given grouping. Surprisingly, even relatively subtle spatial regularities (as in Experiment 3) turned out to influence the formation of contextual associations. 

Despite an influence of the grouping on contextual cueing, the grouped region also affected the overall target detection performance: Targets presented inside the grouping were in general detected faster than targets outside the grouping. However, grouping strength modulated the degree to which attention was prioritized to the grouped region. As can be seen from Figure 4 (red lines), there was a strong preference for processing the grouped region with the square and, to a somewhat lesser extent, the nonsquare configurations, but none for the random-square configuration (attentional capture effects were 143, 110, and 1 ms, respectively). This modulation of attention by grouping strength was robust (despite of our relatively small sample sizes of ten participants per experiment, potentially limiting the statistical power for between-experiment comparisons), illustrating that salient groupings are particularly likely to attract attentional processing (in line with, e.g., Conci et al., 2007a, 2007b; Donnelly et al., 1991; Kimchi et al., 2007; Riccardielli et al., 2001; Yeshurun et al., 2009). Thus, attentional capture was modulated by grouping strength, with no capture effect whatsoever for the most subtle grouping. 

Contextual cueing, by contrast, was unaffected by this factor: The cueing effect was modulated even by the most subtle grouping. Why might this be the case? Conceivably, contextual cueing in itself reflects some form of “below-threshold” grouping, formed by (learned) associative links among neighboring search items. On this view, the extraction of statistical regularities from a search display is re-enforced by contextual memory. However, learning might also be particularly sensitive to grouping cues (e.g., collinear line segments between neighboring search items; see, e.g., the upper border of the random square in Figure 1C). This could explain why contextual cueing was influenced by the random square arrangements, even though there was no effect of attentional capture on search performance by the same type of grouping. This is consistent with previous evidence that perceptual grouping can affect performance in the absence of attention. For instance, grouped objects in triangle form may emerge and reach awareness despite significant masking by noise patterns (Wang, Weng, & He, 2012). Or, in neglect patients, fairly subtle collinear line segments may facilitate detection of target objects within the impaired, unattended visual hemifield (Conci et al., 2009; Pavlovskaya, Sagi, Soroker, & Ring, 1997), suggesting that such groupings may provide a preattentive structure of the scene for guiding perceptual scanning. 

It should be noted that in our previous study (Conci & von Mühlenen, 2009), contextual cueing was not significantly influenced by relatively small random-square arrangements (each square subtended ∼3° × 3°, compared to ∼12° × 12° in the current study); however, cueing under this condition nevertheless exhibited a numerical reduction by more than 50% compared to displays without grouping—a pattern consistent with the present experiments. Cueing in the random-square condition may have been more robust in the current study because the large square groupings connected individual items across larger portions of the display, which in turn might have had stronger implications for contextual learning. That is, “long-range” groupings might have stronger potential for the formation of contextual associations compared to more “local” neighborhood relations. Indeed, long-range connections between search items have already been shown to influence contextual cueing (Olson & Chun, 2002). 

Experiment 4

Experiments 1, 2, and 3 demonstrated that perceptual grouping affects contextual cueing such that targets presented within a segmented region benefit from contextual cueing, whereas targets outside the grouped region do not. In Experiment 4, we further investigated the processes underlying this modulation of contextual cueing. Previous work has shown that the allocation of selective attention primarily affects the expression of learning, while preserving latent learning for unattended display layouts (Jiang & Leung, 2005). In Jiang and Leung's (2005) study, observers were asked to attend to a color-defined subset of search items while ignoring a second, differently colored subset presented simultaneously within a given display. Contextual cueing manifested only for the attended set. However, when, in a subsequent transfer phase, the previously ignored subset was to be attended, contextual cueing was observed immediately for this subset, whereas the previously attended subset (which was now to be ignored) no longer produced contextual cueing. This pattern was taken to indicate that while observers are capable of acquiring the invariant regularities even in the absence of attention (i.e., they show latent learning for the unattended subset), attention is required for recalling (i.e., for the expression of) the learned information during search. 

While attention thus affects only the expression of the learned information, perceptual grouping primarily influences the learning of a given display layout (Conci & von Mühlenen, 2011). The search displays in Conci and von Mühlenen's (2011) study consisted of two segregated subsets that grouped individual items into clusters based on (size) similarity. Compared to a baseline condition (without groupings), contextual cueing was substantially reduced for the size-grouped displays. This reduction occurred despite overall slowed search RTs for grouped relative to the ungrouped displays. Moreover, when the grouping cues were removed in a subsequent transfer phase, contextual cueing still failed to become manifest. This suggests that, whereas latent learning of invariant contexts may occur for top-down controlled attentional selection (as in Jiang & Leung, 2005, where the display was “parsed” into one or the other color group based on the instructed attentional set), no learning occurs when bottom-up perceptual segmentation constrains search (Conci & von Mühlenen, 2011). 

Experiment 4 followed this distinction between initial learning and subsequent recall of the learned information. We therefore adopted the procedure introduced by Jiang and Leung (2005) to test whether learning itself or the expression of the learned information is influenced by the square grouping. To this end, we first presented during five epochs displays with a square configuration (as in Experiment 1) for training. Then, in the final epoch (transfer), we presented the same old displays without the square arrangement while otherwise preserving the repeated contexts (see Figure 1A and 1D for examples of a square display and a corresponding transfer display [without grouping], respectively). 

If the segmented region primarily affects the expression of learning, then the removal of the square should enable contextual cueing during the transfer epoch (because the boundaries of the global object are no longer there). However, if region segmentation affects learning itself, then the contextual benefit should be exclusively observable for target-on conditions even in the transfer epoch—as reliable associations between the target and the context were initially only acquired within the square boundaries. 

Results

Search task

Mean error rates were again low (1.57%). A repeated-measures ANOVA performed on the training set with the factors context (old, new), target (on, off), and epoch (1–5) revealed a significant main effect of context, F(1, 9) = 12.93, p < 0.006, with slightly higher error rates for old as compared to new contextual layouts (2.0% vs. 1.4%). Next, a second two-way ANOVA was performed for the transfer set (epoch 6) with the factors context (old, new) and target (on, off). This analysis yielded a significant main effect of target, F(1, 9) = 5.99, p < 0.04: More errors were made for target-off than for target-on conditions (1.3% vs. 0.7%). No other significant effects were obtained (all ps > 0.2). 

Next, individual mean RTs were computed for each variable combination, excluding error responses and outlier RTs longer than 3 s (0.4% of all trials). Figure 5 depicts the mean correct RTs, averaged across participants, as a function of epoch for target-on (left panel) and target-off (right panel) conditions. For the training set, mean correct RTs were entered into a three-way ANOVA with main terms for context (old, new), target (on, off), and epoch (1–5). This analysis yielded significant main effects of target, F(1, 9) = 63.68, p < 0.001, and epoch, F(4, 36) = 5.55, p < 0.002. There was a 171-ms benefit for target-on (relative to the target-off) conditions, and search became faster with increasing epoch (RTs were 80 ms faster in epoch 5 than in epoch 1). As in all previous experiments, there was a significant interaction between context and target, F(1, 9) = 18.32, p < 0.003: Contextual cueing was evident only when the target was presented on the square, 184 ms, t(9) = 5.56, p < 0.001; by contrast, for target-off positions, there was no contextual cueing, but rather nonsignificant contextual costs of −84 ms, t(9) = 1.71, p = 0.11. This outcome again shows that the square grouping modulates contextual cueing. 

Figure 5

 

Mean RTs (with SE bars) as a function of epoch in the target-on (left) and target-off (right) display conditions of Experiment 4. Epochs 1 to 5 presented search displays with a square grouping. In epoch 6 (transfer), the items forming the square were removed from the displays. Filled and unfilled symbols correspond to old- and new-context conditions, respectively.

Analysis of the transfer epoch (epoch 6) revealed that there was no sudden enhancement of contextual cueing when the square grouping was removed (see Figure 5). This observation was confirmed by a training-transfer comparison, which showed comparable contextual-cueing effects in epoch 6 as compared to epochs 1–5 (contextual-cueing effects were 178 [184] ms and −40 [−84] ms for target-on and -off conditions in epochs 1−5 [epoch 6], respectively), without significant differences in the contextual-cueing effect between epochs 5 and 6 for both target-on and -off conditions (all ps > 0.5). Even though contextual cueing did not change from training to transfer, the removal of the square nevertheless facilitated the overall search performance, with RT gains of 71 (134) ms for target-on (-off) conditions (for baseline, new-contexts, ts[9] > 2.54, ps < 0.04). This decrease of response latencies was most likely due to the reduction of the number of items in the display (from 12 to 8 items presented for a given display during transfer). Moreover, the change in overall RTs from epoch 5 to 6 was particularly pronounced for targets outside the square, revealing a significant interaction, F(1, 9) = 5.67, p < 0.05, with a larger RT benefit for target-off than for target-on conditions (154 ms and 69 ms, respectively). This indicates that targets presented outside the “prioritized” square region benefited most from the removal of the items that define the square. 

Recognition test

The mean accuracy in the recognition test was 47%. Participants correctly identified old patterns on 51.7% of all trials (hit rate), but this did not differ from their false-alarm rate of 45.0%, t(9) = 1.02, p = 0.3, indicating that participants were unaware of the display repetitions. 

Discussion

Results from the learning phase replicated Experiment 1 in showing a reliable (184 ms) contextual-cueing effect only for targets presented within the boundary of the square grouping (but not for targets presented outside the square: −84 ms). Moreover, in the transfer phase, contextual cueing continued to occur only when the target was previously (in the learning phase) presented inside the square (contextual cueing was 178 ms [−40 ms] for target-on [-off] positions). This maintained contextual-cueing effect suggests that grouping interfered with the (latent) learning of the display regularities, rather than with the recall of the learned context. That is, target positions outside the square are not encoded into contextual memory. The result concurs with Conci and von Mühlenen (2011), who similarly showed that perceptual grouping (based on color- or size-similarity) impedes contextual learning. By contrast, ignoring a subset of search items leaves latent learning of invariant display layouts intact (Jiang & Leung, 2005), suggesting that (top-down) attentional orienting and (bottom-up) perceptual segmentation mechanisms differentially affect the ability to engage in implicit contextual learning. 

General discussion

In the present study, we explored the common influence of perceptual grouping and contextual scene memory on the efficiency of attentional orienting. Four experiments revealed reliable contextual-cueing effects when a target was presented within the boundaries of a grouping, but no contextual cueing (or even costs) when the target was located outside of the grouping. Thus, contextual cueing was restricted to items within the grouped region emerging in a given display. As summarized in Figure 4, a modulation of contextual cueing was revealed for several types of grouping, including closed squares (Experiment 1), symmetrical but nonclosed groupings (Experiment 2), and randomly oriented items in square arrangement (Experiment 3). That is, contextual cueing reliably facilitated search for targets presented within all types of grouping—even for relatively subtle element arrangements; by contrast, no contextual facilitation occurred outside the grouped region. 

This pattern of results is incompatible with the idea that the formation of grouped clusters reduces the overall variability in contextual cueing (as surmised by Conci & von Mühlenen, 2009): If the groupings reduced the amount of available contextual information, there should have been no difference in contextual cueing for targets that appeared on versus off the square. Thus, efficient contextual cueing inside (but not outside) the grouped region indicates instead that contextual learning is, in a certain sense, “object-based,” with reliable associations between the target and its surrounding nontargets being formed primarily within the segmented region. According to this view, structural information provided by processes of (a) perceptual organization and (b) associative (contextual) learning may be combined, or integrated, to form a single ordered representation that guides attention: While grouping mechanisms provide units for subsequent attentional processing (Driver et al., 2001, for review), contextual cueing exploits statistical covariances given within a scene to support target selection (Chun, 2000, for review). Our results suggest that grouping and contextual learning are related hierarchically, with segmentation mechanisms first defining potential regions of interest, which then constrain the extraction of statistical covariances among the (search) items; restated, mechanisms of region segmentation automatically parse the visual field into units that are subsequently available for contextual learning. Both the perceptual and learned structure inherent in a given layout of search items might then be integrated to form a common priority signal (see, e.g., Fecteau & Munoz, 2006), based on inputs from both memory and perception, for guiding attention. 

While region-based contextual cueing occurred to a similar extent with all types of grouping, attentional capture by the square configurations was graded, varying with the strength of the grouping. Consistent with previous studies (e.g., Kimchi et al., 2007; Yeshurun et al., 2009), the present experiments revealed evidence of stimulus-driven attentional capture by the grouped object: Both square and nonsquare configurations yielded a substantial search benefit for targets presented within the boundaries of the configuration (when considering new-target displays only, i.e., baseline displays for which there could not be any contextual learning). This benefit for targets within the grouped region arose even though the (square/nonsquare) configuration was entirely task-irrelevant and nonpredictive with respect to the target location. In contrast, random squares showed no search benefit for targets within the grouped region (see Figure 4, red dotted lines). This pattern shows that attention was attracted to a lesser extent by the square configuration as grouping strength decreased. Configurations defined by closure and symmetry attracted attention, but configurations defined solely by spatial regularity did not. Nevertheless, contextual cueing was confined to the segmented region with all groupings (even relatively subtle regularities such as the random-square in Experiment 3), indicating that grouping has differential effects on contextual learning and attentional selection. 

Figure 6 presents a schematic illustration to account for the relation between perceptual grouping and contextual cueing in guiding attention. First, when considering a repeated “standard” display (i.e., without systematic grouping), it is assumed on the basis of a computational model by Brady and Chun (2007) that contextual cueing relies primarily on the learning of local contextual associations between a target and a limited number of (about three) surrounding nontarget items (red lines in Figure 6A; see also Olson & Chun, 2002). In order to encode these contextual associations, attention is not required—as evidenced, for instance, by the finding that an ignored subset of search items can nevertheless generate a reliable contextual-cueing effect (Jiang & Leung, 2005). This is reflected in our schematic model by a broad distribution of attention (gray attentional “spotlight” in Figure 6) by default, such that contextual associations can be learned and used to guide search (see also Conci & von Mühlenen, 2011). However, the picture is different when the displays presented contain grouped regions, in which case contextual learning will primarily operate within the grouped, segmented region. As a result, reliable contextual associations are only, or primarily, formed when the target is located within this region (Figure 6B, C), but not for targets outside of the groupings (Figure 6D, E). Note that the preference of contextual learning to evolve within object boundaries is independent of grouping strength (which is depicted in Figure 6 by the thickness of the gray dashed lines): Weak groupings (e.g., random squares; Figure 6C, E) impose comparable limits on contextual learning to strong groupings (e.g., squares; Figure 6B, D). Conversely, attention is only attracted by strong (e.g., square) groupings, whereas it remains distributed broadly when groupings are weak (e.g., random squares), illustrating that contextual learning is independent of the allocation of attention (Jiang & Leung, 2005), but dependent on the segmented groups. Thus, even subtle regularities in a given display that are too weak to attract attention may nevertheless interfere with contextual cueing (Figure 6E). Segmented regions are therefore not necessarily equally effective for learning and attention: Contextual learning may already be modulated by relatively subtle grouping cues, whereas attentional selection only occurs for relatively salient configurations possessing a certain degree of “object-hood” (Kimchi et al., 2007). In this view, salient groupings can directly influence the allocation of attention, while relatively nonsalient groupings exert only an indirect influence through the modulation of contextual learning. 

Figure 6

 

Examples of search display layouts with schematic illustrations of the relation between perceptual grouping (gray, dashed lines), contextual cueing (red lines), and attention (gray circles). See text for further details.

Grouping determines contextual learning

Experiment 4 further investigated the way in which memory is affected by closed square groupings, distinguishing a period of learning from a subsequent transfer phase. As in all previous experiments, during learning, a reliable contextual benefit was observed for targets inside, but not outside, the grouping (184 ms and −84 ms, respectively). Importantly, removal of the items that defined the square in the transfer phase did not change this pattern of results: Targets presented within the (previous) square region still exhibited a contextual benefit (which was undiminished, 178 ms), whereas there was still no contextual facilitation for targets presented outside of the (former) square region (−40 ms). This pattern indicates that grouping affects contextual learning, rather than the retrieval of learned contextual associations (as a region-based modulation of contextual cueing remained effective even though the grouping cues were removed; see also Conci & von Mühlenen, 2011). Conversely, Jiang and Leung (2005) demonstrated that (selective) attention affects the retrieval, but not the learning of contextual regularities. Thus, taken together, these studies show that while grouping affects learning itself, the allocation of attention only influences the recall of learned information. 

In explicit-learning tasks, attention and memory are generally closely linked—for instance, the allocation of attention influences what is encoded into memory (see, e.g., Chun & Turk-Browne, 2007, for review). However, in implicit-learning tasks, such as in contextual cueing, this relationship is more complex; for instance, an ignored context can nevertheless be learned (Jiang & Leung, 2005). In agreement with these findings, our results demonstrate that whether or not a grouped region attracts attention does not influence the extent of contextual learning; in addition, though, our results show that perceptual segmentation does constrain learning. 

In line with previous studies (e.g., Chun & Jiang, 1998), the current experiments provided evidence of contextual learning being implicit in nature: The repetition of certain search layouts was not noticed in the final recognition test. However, observers could nevertheless have noticed that some display items contained a square grouping, which could have affected the extent of on- versus off-square learning. Taking the occurrence of attentional capture (see Figure 4, red dotted line) as an indirect measure of whether observers did or did not notice the grouped arrangements, it would appear that at least the random squares went unnoticed while they nevertheless modulated contextual cueing. Moreover, a comparable modulation of contextual cueing was also evident in Conci and von Mühlenen (2009) who presented square groupings in only half of all trials, rather than on every single trial—suggesting that presenting a grouping more or less frequently does not affect its modulatory influence on contextual learning. Taken together, the modulatory effects of relatively unnoticed (subtle) and “infrequent” groupings show that contextual learning is, to some degree, independent of the engagement of attention but determined by the grouped display structure. 

Object segmentation structures attention and memory

In general, a large number of studies have demonstrated close links between perceptual grouping and attentional orienting (e.g., Driver et al., 2001; Roelfsema, 2006, for reviews). For example, search for a target grouping is more efficient than search for corresponding fragmented items (Conci et al., 2007b), with the speed of attentional orienting varying as a function of the grouping strength (Conci, Töllner, Leszczynski, & Müller, 2011). On the other hand, there is less evidence as to how the structuring of perceptual inputs governs learning and memory representations: It has been shown that short-term memory representations are affected by bottom-up perceptual grouping—items that are grouped together tend also to be stored together (Quinlan & Cohen, 2012; Woodman, Vecera, & Luck, 2003), with integrated objects reducing memory load (Patterson, Bly, Porcelli, & Rypma, 2007). Moreover, relational information between objects may also influence the degree of change blindness and change detection (Jiang, Chun, & Olson, 2004; Jiang, Olson, & Chun, 2000; Landman, Spekreijse, & Lamme, 2004). In addition, perceptual grouping also affects learning of environmental statistics (Baker, Olson, & Behrmann, 2004; Glicksohn & Cohen, 2011). For example, Baker et al. (2004) demonstrated that element connectedness affects implicit learning of object statistics, suggesting that integrated fragments are registered as units. In sum, these finding show that grouping not only affects the orienting of attention, but also the contents of (working) memory. 

In this regard, previous work on (long-term) scene memory has likewise demonstrated that object segmentation affects contextual cueing. Conci and von Mühlenen (2009) showed that region segmentation interferes with contextual cueing. For instance, no contextual cueing effect was found when four items of a search display grouped by, for example, closure and/or symmetry to form a square. It should be noted that the segmented region was smaller and never enclosed the target. Therefore, the pattern of interference was probably comparable to the target-off condition in the current experiments (which also revealed no indication of contextual learning). In general agreement with a modulatory influence of grouping on contextual cueing, grouping by color (and size) similarity also has (have) been shown to substantially reduce contextual cueing (Conci & von Mühlenen, 2011). In line with such a dependency of contextual cueing on the spatial segmentation of a display, investigation of contextual cueing in three-dimensional search layouts has shown contextual-cueing effects to evolve primarily for repetitions of invariant context within, but not between, segregated depth planes (Kawahara, 2003; see also Geyer, Shi, & Müller, 2010, for a comparable outcome for color-based subset formation in contextual cueing). In this regard, cueing effects have also been shown to vary as a function of the spatial proximity between a given target location and the invariant context (Olson & Chun, 2002). Similarly, contextual cueing may be limited by the temporal segmentation of subsets in (preview) search (Hodsoll & Humphreys, 2005). Taken together, these studies suggest that perceptual structure imposed by means of grouping, three-dimensional (3-D) disparity, or temporal segmentation constrains the formation of contextual cues in search. Our current results extend these findings and indicate that the influence of perceptual structure on contextual cueing is not dependent on the deployment of attention. Whether or not attention was attracted by a given perceptual structure did not influence contextual cueing: spatial-, temporal-, or 3-D–structure therefore appears to constrain contextual cueing to particular regions independently of attentional deployment. 

Conclusion

In summary, our results suggest that object grouping defines perceptual units within which contextual scene regularities are acquired. Grouping may be relatively subtle but nevertheless affect how contextual learning establishes associations between items in a search display. Conversely, only salient groups will attract attention, suggesting that grouping affects contextual learning before a particular grouped object attracts attention. It has recently been proposed that the visual system primarily derives predictions from the environment to infer adaptive behavior (Clark, 2013; see also Conci, Zellin, & Müller, 2012). Our current results show that learning of statistical regularities to predict the location of a target is largely confined to subsets of the environment that potentially integrate to form the regions or objects in a scene, independently of whether or not they are subsequently attended. 

Acknowledgments

This work was supported by German Research Foundation (DFG) project grants (FOR 480 and CO 1002/1-1). We thank Barbara Neumayer for help with data collection. 

Commercial relationships: none. 

Corresponding author: Markus Conci. 

Email: [e-mail]. 

Address: Allgemeine und Experimentelle Psychologie, Department Psychologie, Ludwig-Maximilians-Universität, München, Germany.