Transfer of contextual cueing in full-icon display remapping

Abstract

Abstract

Abstract:

Abstract  Invariant spatial context can expedite visual search, an effect that is known as contextual cueing (e.g., Chun & Jiang, 1998). However, disrupting learned display configurations abolishes the effect. In current touch-based mobile devices, such as the iPad, icons are shuffled and remapped when the display mode is changed. However, such remapping also disrupts the spatial relationships between icons. This may hamper usability. In the present study, we examined the transfer of contextual cueing in four different methods of display remapping: position-order invariant, global rotation, local invariant, and central invariant. We used full-icon landscape mode for training and both landscape and portrait modes for testing, to check whether the cueing transfers to portrait mode. The results showed transfer of contextual cueing but only with the local invariant and the central invariant remapping methods. We take the results to mean that the predictability of target locations is a crucial factor for the transfer of contextual cueing and thus icon remapping design for mobile devices.

Introduction

Invariant visual context provides an important spatial cue for the guidance of visual search and focal-attentional selection. Repeated exposure to the same arrangements of search displays facilitates reaction time (RT) performance, an effect that has been referred to as contextual cueing (Chun, 2000; Chun & Jiang, 1998; Chun & Nakayama, 2000). In their seminal paper, Chun and Jiang (1998) had their observers search for a target letter “T” embedded in a set of distractor letters “L”. Unbeknown to participants, half of the presented displays contained identical configurations of target and distractor items (i.e., old displays), whereas the other half contained novel configurations (i.e., new displays). The main result was that of faster RTs to old relative to new displays (i.e., contextual cueing), an effect that developed after a short period of training. Interestingly, when observers were queried about repeated displays at the end of the search task in an “old-new” recognition test, their performance was only at chance level. From these findings, Chun and Jiang (1998) concluded that (a) contextual cueing guides focal attention more rapidly to the target location (but see Kunar, Flusberg, Horowitz, & Wolfe [2007], for evidence that contextual cueing might also aid postperceptual processes) and (b) the cueing effect derives from an implicit memory for the items' spatial arrangement. Since then, the cueing effect has been elaborated in a number of further studies (Chun, 2000; Chun & Jiang, 1998; Chun & Nakayama, 2000; Conci, Sun, & Müller, 2011; Conci & von Mühlenen, 2009, 2011; Geyer, Shi, & Müller, 2010; Jiang & Wagner, 2004; Kunar, Flusberg, & Wolfe, 2006). Jiang and Wagner (2004; see also Brady & Chun, 2007, or Olson & Chun, 2002) showed that contextual cueing is supported by two distinct spatial memory systems for individual item locations (i.e., local learning) and, respectively, the entire configuration formed by the distractors (i.e., global learning). Further, Kunar et al. (2006) showed that nonspatial attributes, too, such as background color, can facilitate RT performance. Contextual learning is also influenced by selective attention: Only the arrangement of some items, in particular, those sharing the target color, are learned over the course of an experiment (e.g., Geyer et al., 2010; Jiang & Leung, 2005). 

However, the degree to which contextual cueing can adapt to changes in learned displays remains subject to debate. For example, Jiang and Wagner (2004) reported that contextual cueing was still reliable even when learned displays were shifted along the horizontal display axis, the vertical display axis, or presented in a different size (compressed or expanded). Other studies (Brady & Chun, 2007; Olson & Chun, 2002) showed that contextual cueing survived changes of approximately 50% up to 75% of the display items; that is, cueing was reliable even when only one half or one quadrant of the display was repeated across trials. On the other hand, Olson and Chun (2002) reported that the cueing effect was abolished when new distractors were presented in between the target and the old distractors, with the target being presented, for example, in the left half and the old distractors in the right half of the display. Several other studies confirmed that contextual cueing diminished when the target was repositioned in repeated displays and thus became unpredictable (Chun & Jiang, 1998; Manginelli & Pollmann, 2009; Olson & Chun, 2002; Wolfe, Klempen, & Dahlen, 2000). In contrast, the contextual cueing effect remained effective with predictable target location changes (Conci & Müller, 2012; Conci et al., 2011). Makovski and Jiang (2010) suggested that predictability based on invariant context is a key factor for contextual cueing, based on their finding that the cueing effect decreased as the target appeared further away from its learned location; in fact, there were even RT costs when the target swapped its location with a previous distractor. Similar findings have been reported in three-dimensional (3D) scene search (Chua & Chun, 2003), in which contextual cueing decreased with increasing angular difference between viewpoints in the training versus the test displays (the experiment was divided into a training and test phase, with the latter containing modified displays). 

Although most of the work on contextual cueing was conducted using consistent (i.e., spatially invariant) search displays with a fixed number of items (e.g., one target and 11 distractors presented at a total of 48 locations within an invisible 6 × 8 matrix), none of these studies has examined the influence of changes of the display orientation on the cueing effect. Although changing display mode (and accordingly remapping of the items) occurs rarely with standard (i.e., laboratory) displays, switching display mode is a normal routine in current touch-based mobile devices, such as the iPad. Interestingly, with these devices, there is only one type of item—or icon—remapping method available: The positions of icons in one display (e.g., landscape mode) are remapped to the other display (portrait mode) by keeping the positional order (left to right and up to down) constant across all icons (see Figure 1a, b). Although this remapping method preserves the positional order and 80% of the horizontal intericon relationships (in a 4 × 6 icon matrix, as shown in Figure 1a, b), it destroys almost all local icon relationships, in particular, when the display is arranged as a rectangle (as with almost all mobile devices). However, based on the contextual cueing studies reviewed above, it is possible that contextual cueing is reduced, if not entirely abolished, when display orientation changes from landscape to portrait mode and icons are remapped in the standard position-order manner. Given this, one intriguing question arises, namely, are there any other improved methods for icon remapping, such that the remapping could enhance users' performance in everyday situations of display mode changes? This question was addressed in the current study by using the contextual cueing effect as a tool to evaluate the effectiveness of various display-remapping techniques; that is, preserved contextual cueing from one to the other display mode was taken as an indicator for the value of a given remapping method. 

Figure 1

 

Schematic illustration of display layouts and remapping methods. (A) Display layout in landscape mode; each number denotes an individual icon. (B) Portrait display layout obtained by the position-order invariant remapping method; the arrow indicates the icon remapping sequential order from the landscape to portrait mode. (C) Portrait display layout obtained by the global rotation remapping method; the arrow indicates the rotation direction from the landscape to portrait mode. (D) Portrait display layout obtained by the local invariant remapping method; circled regions remain the same between the landscape and portrait mode. (E) Portrait display layout obtained by the central invariant remapping method; circled regions are invariant.

Besides the position-order remapping method, several other (simple) remapping methods are possible. For example, one of the most natural ways is to rotate the entire display by 90° in the clockwise direction (individual icons are rotated 90° in the counterclockwise direction to keep their appearance constant; see Figure 1c). Such a global rotation is similar to the rotation of an object in our physical world (e.g., imagine you rotate a key cabinet with many keys). Alternatively, and motivated by the above mentioned studies on contextual cueing (e.g., Brady & Chun, 2007), one could also try to preserve local associations within the entire configuration as completely as possible. There are two ways to maximize such local invariants. One is to subdivide displays into several local regions and preserve the placement of these local regions in the entire configuration after icon remapping (Figure 1d). Another method is to keep the display center constant in remapped displays (Figure 1e). 

To investigate how these various display-remapping methods influence memory performance, we examined contextual cueing effects in four separate experiments. Each experiment examined one display remapping method. To simulate touch-based icon displays and observers' active touch action, we used real desktop icons as search items and presented them on a touch monitor in the four experiments. 

Methods

Participants

A total of 40 observers took part in the experiments (10 in each experiment, mean ages: 27.9, 26.2, 25.5, and 27.3 years and number of females: 7, 6, 6, and 5 for Experiments 1–4, respectively). All had normal or corrected-to-normal visual acuity (including color vision). They gave written consent prior to the experiment and were paid at a rate of €8/hour for taking part. Participants were naive as to the intention of the study. 

Apparatus and stimuli

The experiments were conducted in a dimly lit cabin (ambient light: 4.36 cd/m²). Visual stimuli were presented on a 23-inch multitouch LCD monitor (HP2310ti) with spatial resolution set to 1920 × 1080 pixels. To make touch pointing comfortable for the participants, the screen panel was placed on the table tilted by 45°. The viewing distance was approximately 40 cm, with participants' head position fixed by a chin rest. Twenty-four typical computer icons (randomly selected from 48 candidate icons¹ for each observer) were presented within an invisible 6 × 4 horizontal grid (subtending 24° × 16° of visual angle) or a 4 × 6 vertical grid (subtending 24° × 16°). The target was the icon with a top overlay of a compound letter “T” (subtending 1.6° × 1.6°; luminance 35.67cd/m²; see Figure 2). Such a compound target letter was used for two reasons: first, to avoid interference between the target and some other (distractors) letters, and second, to make the compound letter and the icon comparable in terms of their luminance level. The background of the search displays was set to gray (16.56 cd/m²). To enhance the global spatial “Gestalt” (i.e., perception of the display as landscape or, respectively, portrait mode), we added one array of six upright white triangles (130.5 cd/m²) with a gray background (19.62 cd/m²) below the landscape mode (Figure 2a) or to the left side of the portrait mode (Figure 2b). The triangle array was meant to serve as a global landmark in the experiments, indicating display mode changes. The experimental program was developed with and controlled by Matlab (Mathworks Inc., Natick, MA), in addition to the Psychtoolbox (Brainard, 1997; Pelli, 1997). Response times were recorded via the touch screen. To determine the onset of a response, an additional input button (connected to a NI PXI system) was placed in between the touch screen and the participants, which was used for initiating the task and pointing movement. 

Figure 2

 

Example displays in the experiments. (A) Example of a landscape display. In this example, the “Apple” icon (second row, right-most column) is the search target. (B) Example of a portrait display. In this case, icons are remapped from the landscape mode by keeping the position order constant in the left-to-right and up-to-down manner (Experiment 1). (C) The top overlay for the target icon (a compound letter “T”).

Design and procedure

A three-factorial within-subject design was used with display mode (landscape, portrait), context (old, new), and experimental epoch (1–9) as independent variables. From the 24 possible target locations, we randomly selected 12 target positions for old and the other 12 positions for new displays. In this way, the target appeared equally likely at any of the 24 possible locations. To have enough difference between old and new configurations and to control the similarity of icon identities, we selected 24 icons from 48 typical icon candidates and assigned to random locations. Each of the new target locations was paired with newly generated distractor icons for every new-display trial, whereas each of the old target locations was paired with randomly selected distractor icons at the beginning of each experiment and served as old landscape displays. These old landscape displays were also used to define the remapped old portrait displays. Remapping was one as follows: 

•  
  Experiment 1 (position-order invariant). The positional order (left to right and top to bottom) of the icons in the portrait mode was the same as that in the landscape mode (Figure 1b). This method is used in most of the present mobile devices for the rearrangement of icons.
•  
  Experiment 2 (global-rotation). The landscape display was, as a whole, rotated by 90° clockwise into the portrait mode, while preserving the (upright) orientation of the individual icons. With this global rotation, the global and local relationships of the icons are rotated by 90° across display changes (Figure 1c).
•  
  Experiment 3 (local invariant). To preserve the local (and global) spatial configuration as much as possible, in Experiment 3, the display was divided into four peripheral and one central region, each consisting of four icons (see circled regions in Figure 1d). The positioning of these four “corners” and the central region were kept constant across display mode changes. Only four remaining items (i.e., Icons 3, 4, 21, and 22 in Figure 1d) changed their relative positions. Similar to the global rotation, with the local-invariant transformation, the local relationships between all icons are preserved across display changes.
•  
  Experiment 4 (central-invariant). Instead of dividing the display into multiple regions, in Experiment 4, we preserved the central display region as much as possible (i.e., preserving the central maximum square region). As shown in Figure 1e, icons in the central 4 × 4 matrix were positioned at identical locations across display mode changes. In addition, the four outermost (corner) icons were also unchanged. Only the remaining four icons (7, 12, 13, and 18 in Figure 1e) changed their positions.

Each experiment comprised three consecutive sessions: learning, test, and recognition. In the learning session, there were five epochs of three blocks, with each block consisting of 24 search trials. To keep the experiment as short as possible, the learning session contained only 12 old-landscape displays to foster learning effect (each of the old display repeated twice per block). The transfer session had four epochs, with each epoch consisting of 24 trials (i.e., one block only). In half of these trials, an old display was presented and new displays in the other half. New displays were randomly generated at the beginning of each trial. The order of display modes in the transfer epochs was fixed: landscape (L), portrait (P), portrait (P), and landscape (L). The first transfer epoch with the landscape mode (i.e., nontransformed) was intended to test for a standard contextual cueing effect. The last transfer epoch was intended for examining whether contextual cueing is still manifested by two intervening epochs containing different display modes. To avoid confounding by repetition effects, we randomly presented trials in such a way that the same old display was never repeated within three consecutive trials. 

In the learning and test sessions, each trial started with a cross-fixation presented in the center of the display. Participants had to press the input button (also serving as the initial hand position) to trigger the presentation of the search display. Participants were instructed to detect the target and touch its location with their index finger as rapidly and as accurately as possible. A blank screen was presented after the localization response, or 4.5 s when no response was made. When participants made an erroneous response, an additional feedback display containing a stop warning sign was presented for 1.0 s. After 1.0 to 1.2 s of intertrial interval, the next trial started. 

In the recognition session, participants were asked if they had realized any display repetitions during the learning and transfer sessions and, if so, when they had first noticed the repeated displays (note that a similar protocol was used by Chun & Jiang, 1998). Following this, they had to judge a total of 24 displays, including 12 new displays (six landscape and six portrait displays) and 12 old displays (six landscape and six portrait displays), in an “old-new” recognition test. In this test, the chance rate for recognizing a repeated display was 50%. 

Prior to the experiment, participants practiced the experimental task in one training block of 24 trials (data not recorded). The search displays used in the practice trials were not shown later in the experiment. Participants were allowed to take a break in between successive blocks of the experiment. The break between the learning and transfer sessions was similar to other between-block breaks. 

Results

Accuracy performance

Error rates were overall small (<1%) and were comparable across all experiments. For further RT data analyses, we excluded trials with erroneous responses and RTs outside the range of 200 to 3000 ms. Such outliers were also low in general (<3%). 

Perceptual learning

The mean RTs for the learning sessions are shown in Figure 3 (Epochs 1–5). For each experiment, the mean RTs were examined by repeated-measures analysis of variance (ANOVA) with the single-factor epoch. The main effect was significant for all four experiments (all p's < 0.05); further Bonferroni tests revealed a significant perceptual learning effect, defined as the difference in RTs between Epoch 5 (i.e., the end of the training session) and Epoch 1 (i.e., the beginning of the training session; Table 1). In addition, to examine interference by the introduction of new (both landscape and portrait) displays in the transfer session, RTs for the old displays in the first epoch of the transfer session (Epoch 6) were compared with RTs in the last epoch of the learning session. Although RTs were numerically longer in Epoch 6 compared with Epoch 5, the slowing was significant only for Experiment 2 (Table 2). This suggests that introducing novel displays had only some moderate influence on the search task response. 

Figure 3

 

Mean correct response times (RTs) as a function of epoch for the learning (Epochs 1–5) and transfer (Epochs 6–9) sessions; for the latter, mean RTs are shown separately for old displays (denoted by diamonds and solid lines) and new displays (denoted by dots and dashed lines). (A) Experiment 1, position-order invariant remapping. (B) Experiment 2, global rotation remapping. (C) Experiment 3, local invariant remapping. (D) Experiment 4, central invariant remapping.

Table 1

 

Mean learning effect in the training sessions and interference by the addition of new displays in the transfer session, for each experiment.

	Perceptual learning		Interference associated with the presentation of new displays
Experiment	Facilitation (ms)	ANOVA	Cost (ms)	ANOVA
1	364	p < 0.01	60	p = 0.08
2	185	p < 0.05	154	p < 0.05
3	385	p < 0.01	36	p = 0.21
4	380	p < 0.01	98	p = 0.11

Table 2

 

Contextual cueing effects in the transfer session. The reported significance values are as follows: *p < 0.05; **p < 0.01; ***p < 0.001.

Experiment	Contextual cueing effect (ms)					ANOVA test with F value
	Average (Epoch 6–9)	Epoch 6	Epoch 7	Epoch 8	Epoch 9	Context (old/new)	Epoch (6–9)	Interaction
1	86	202***	5	18	120*	9.25*	9.23**	4.49*
2	120	262***	7	28	197**	12.48**	1.40	8.80***
3	156	223**	50	147*	205**	21.49***	5.16**	3.07*
4	163	286**	43	142*	190**	22.42***	5.63**	3.91*

Transfer of contextual cueing effect

The mean RTs, separately for old and new contexts, as a function of epoch for the test phase are presented in Figure 3 (Epochs 6–9). To examine the contextual cueing effect, mean RTs were subjected to a repeated-measures ANOVA with epoch (6–9) and context (old vs. new) as factors, separately for each experiment. The results are summarized in Table 2. The RTs were significantly faster for old displays compared with new displays in all four experiments, indicating robust contextual cueing benefits. The main effect of epoch was also significant for Experiments 1, 3, and 4, indicating that some perceptual learning also occurred in the transfer session. Finally, the context × epoch interaction was significant for all experiments, reflecting differential cueing effects in the different epochs. Post hoc tests revealed significant contextual cueing to be significant for all landscape displays (Epochs 6 and 9). By contrast, for portrait displays (in Epoch 8), significant contextual cueing was evident only in Experiments 3 and 4 (see Table 2). Note that each epoch in the transfer session contained only 24 trials, suggesting that the contextual cueing effect could be quickly transferred with the local invariant and central invariant remapping methods when the display mode was changed. 

To examine whether contextual cueing effects were comparable among the different experiments, a repeated-measure ANOVA was conducted on the cueing effect in the first transfer Epoch 6 (with landscape mode), with the single-factor experiment. The effect of experiment was nonsignificant, F(3, 27) = 0.55, p = 0.65, suggesting that the contextual cueing effects were comparable among experiments. Thus, any differences in the subsequent transfer epochs are likely attributable to the particular method of display (icon) remapping. 

Recognition test

Based on participants' postexperimental reports, we determined the percentages of participants who noticed display repetitions during the search task and who attempted to explicitly learn the displays; the times (in terms of the number of blocks performed) at which these participants first noticed the repetitions were also calculated. We then further calculated participants' mean hit and false alarm rates as well as their discrimination sensitivities (d′) for landscape (L) and portrait (P) displays. The results are summarized in Table 3. 

Table 3

 

Results of recognition test. The reported significance values are as follows: *p < 0.05; **p < 0.01.

Experiment	Noticed repetition	Explicit learning	When (blocks)	Hit rates	False alarms	d′L	d′P	d′L = d′P
1	90%	60%	6.89	76.3%	38.3%	2.17**	1.17*	p = 0.17
2	80%	30%	7.25	72.9%	43.3%	1.33**	1.17*	p = 0.68
3	90%	20%	5.85	65.1%	28.3%	1.56	2.02*	p = 0.51
4	80%	60%	4.75	80%	28.3%	3.42**	1.52*	p < 0.05

In all experiments, participants exhibited high proportions of recognized displays. The recognition sensitivities (d's) were significantly larger than zero for both landscape and portrait displays (p < 0.05), except for one marginally significant effect for the landscape display in Experiment 3 (p = 0.066), which was mainly due to one observer who showed an extreme negative d_H′ score (−1.40). When excluding this participant, d_H′ was also significant: p < 0.05. Taken together, the significantly positive d′ scores suggest that after learning, participants recognized not only the old landscape displays but also the remapped portrait displays in all four experiments. Moreover, there was no significant difference in recognition sensitivity between the landscape and portrait displays, at least for the first three experiments (see the last column in Table 1), indicating that remapping did not hamper explicit recognition. Although recognition accuracy was lower for portrait than for landscape displays in Experiment 4, the effect was mainly due to the very high recognition sensitivity in the landscape mode (Table 1). Nevertheless, even in Experiment 4, the sensitivity for the portrait displays was still significantly greater than zero, supporting the idea that the transformed old portrait displays can be recognized explicitly. The lack of differential recognition sensitivities between landscape and portrait displays in Experiments 1 and 2 is in contrast to the differential contextual cueing effects with landscape versus portrait displays. This suggests that recognition and visual search may involve different memory processes, with the former recruiting more complex information-matching processes that do not benefit the search processes. 

Discussion

The present study examined the transfer of learned contextual cues in full-icon display remapping. The main goal was to investigate whether contextual cueing continues to facilitate icon localization (RT) performance following display mode changes. We compared four different types of icon remapping: position-order invariant, global-rotation, local-invariant, and central-invariant remapping. In all experiments, robust learning effects were found in the training session for the landscape displays. The RTs were faster at the end relative to the beginning of the training session. This practice effect is likely attributable to general learning of the localization task (Schneider & Shiffrin, 1977). 

In the test session, in which new displays were introduced (in addition to the old displays), we established a contextual cueing effects in all experiments, at least when the display mode was kept the same. This suggests that icon identities and spatial configurations among icons could serve as context cues to facilitate the localization task. Note that the facilitation effect might also be partially due to position-based learning, given that only old displays were used in the training session. However, the transfer effects found in the portrait displays (Experiments 3 and 4) cannot be fully explained by position-based learning, because the positions were changed in the portrait displays and positional repetitions were equated between the old and new displays. Interestingly, contextual cueing was evident for landscape displays even after the insertion of two epochs of portrait displays. This may be taken to indicate that the cueing effect is relatively robust against interference within the same set of old configurations, consistent with previous studies (Chun & Jiang, 1998, 2003; Conci et al., 2011; Conci & Müller, 2012; Jiang, Song, & Rigas, 2005; Song & Jiang, 2005; Zellin, Conci, von Mühlenen, & Müller, 2011). However, contextual cues acquired with landscape displays were transferred to portrait displays only under certain remapping conditions (those of Experiments 3 and 4), suggesting that contextual cueing is relatively inflexible and that transfer is confined to specific remapping situations. 

The differential pattern of effects revealed among the four experiments raises the question as to the factors that modulate the transfer of learned displays. The position-order invariant method maintained icons in their same left-to-right and up-to-down manner. Although 80% of the horizontal relationships are preserved with this transformation, it destroys almost all vertical relationships. It also changes the absolute positions of the icons dramatically; for instance, Position 5 is shifted from the left side in the landscape display (Figure 1a) to the right side in the portrait display (Figure 1b). As a result, the target location might become unpredictable in remapped displays, abolishing the contextual cueing effect (Conci et al., 2011; Manginelli & Pollmann, 2009). Note that in the current terms, predictability refers to both the target's absolute location on the screen as well as its placement within the entire configuration (given that we did not vary the target's absolute and relative location independently). 

When comparing the position-order invariant to the global-rotation method, the latter maintains all local icon neighborhood relationships, but the overall Gestalt is rotated by 90° from the landscape to portrait mode. With this type of remapping, repeated displays failed to facilitate RT performance in portrait displays. Possibly, the contextual associations learned in the landscape displays were quite instance specific and too weak for the global-rotation remapping. As shown in mental rotation studies (Böckler, Knoblich, & Sebanz, 2011; Borst, Kievit, Thompson, & Kosslyn, 2011; Ionta & Blanke, 2009; Shepard & Metzler, 1971; Shomstein & Yantis, 2004), RTs increase linearly with increasing angular disparity when participants were asked to decide whether two presented objects are the same. Those paired objects were normally rotated objects or mirrored objects, and participants had to carry out mental rotation (rotating one object into the other) to solve the task. Applied to the current Experiment 2, although the global rotation maintains the local icons' neighborhood relationships, the mapping of a new portrait onto an old landscape display may likewise be a demanding (i.e., time-consuming) process, which diminishes any performance gains brought about by contextual cueing. In a previous study using 3D visual search, Chua and Chun (2003) also showed that contextual cueing decreased with increasing angular difference between viewpoints of training and test displays. Thus, demanding mental rotation might be the main reason why we failed to find any transfer of contextual cueing from the landscape to the portrait in Experiment 2. It should be noted, however, that in our setup, the experimental program presented the rotated portrait display automatically. That is, participants passively viewed the search displays, rather than carrying out the rotation actively. It would be interesting to examine the transfer of contextual cueing when participants rotate the displays themselves (i.e., actively). 

In contrast to Experiments 1 and 2, we found significant transfer of contextual cueing in Experiments 3 and 4, in which the portrait display was remapped from the landscape display using the local-invariant (Experiment 3) or central-invariant methods (Experiment 4). Both experiments disclosed numerical contextual cueing benefits already in the first epoch with portrait displays (50.8 and 44 ms for Experiments 3 and 4, respectively), although these effects were not significant. No contextual cueing in the first portrait epoch is likely due to the orientation change of the whole display. Mapping old landscape to portrait displays may engage additional mental processes, diminishing the contextual cueing effect. In addition, interobserver variability was large because both the old and new displays were presented only once in this epoch. Interestingly, transfer of contextual cueing was highly reliable for both remapping methods (147.4 and 142.0 ms for Experiments 3 and 4, respectively) in the second epoch. The local-invariant remapping method keeps five of seven local regions unchanged, and the global topological relationship of these five local regions also remains the same. This means that local regions appear at the very same positions (quadrants) in the entire configuration after the remapping. Likewise, the central-invariant remapping method maintains the absolute icon positions of the four outermost corners and the central region (83% in total). In both cases, after the remapping, the target position is much more predictable compared with both the position-order invariant and the global rotation methods. In contrast to previous investigations of contextual cueing, suggesting that only three to four repeated items (among some eight novel items) can produce the effect (Song & Jiang, 2005), the results of the present Experiments 1 and 2 suggest that merely preserving some local invariant information does not guarantee transfer of contextual cueing. Instead, the three to four items would have to appear at the very same positions within the global configuration to observe contextual cueing (Experiments 3 and 4; see also Brady & Chun, 2007, for a related proposal, albeit using different approach). 

The recognition tests showed that in all experiments, participants were well able to discern repeated from nonrepeated displays. This contrasts with standard contextual cueing studies in which recognition accuracy was typically at chance level (Chun & Jiang, 1998). Explicit memory effects may be due to the heterogeneous and, importantly, realistic icons used as distractors in our experiments (see also Brockmole, Castelhano, & Henderson, 2006). Interestingly, in all the experiments of the present study, recognition accuracy was larger than chance for all landscape and, importantly, remapped portrait displays. In contrast, transfer of contextual cueing was observed only in Experiments 3 and 4. This argues that merely recognizing a repeated display as an old one does not necessarily mean that this also facilitates RT performance. Of interest in this regard, it has been reported that explicit learning of repeated displays engages neural processes that are distinct from those concerned with implicit configural learning (Geyer, Baumgartner, Müller, & Pollmann, 2012; Preston & Gabrieli, 2008; Westerberg, Miller, Reber, Cohen, & Paller, 2011). Along these lines, we suggest that recognition and visual search are supported by different memory processes. Further, the dissociation between the transfer of contextual cueing (Experiments 3, 4) and explicit recognition (Experiments 1–4) suggests that the memory underlying explicit learning is more flexible than that underlying implicit configural learning. 

Conclusion

In sum, our experiments suggest that when display orientation switches and icons are rearranged, the traditional position-order remapping method used in current mobile touch devices is suboptimal in aiding search performance. Comparing and contrasting three alternative methods of icons remapping, we found that when using local-invariant or central-invariant remapping, contextual cueing continues to enhance (target) icon localization performance. Although the global-rotation method may be intuitive for users, it might introduce additional mental-rotation processes that are detrimental to localization performance. Our findings thus have implications for alternative interface design guidelines for icon rearrangement in mobile devices. Open questions awaiting further research concern how to optimize local invariance regions and what the effects of active manual rotation are. 

Acknowledgments

This work is supported in part by German Research council (DFG) project grant SH166 to Z. S. and by Chinese Scholarship Council (CSC) to X. Z. and L. J. 

Commercial relationships: none. 

Corresponding author: Zhuanghua Shi. 

Email: [e-mail] 

Address: Experimental Psychology, Department Psychology, Ludwig-Maximilians-Universität München, 80802, Munich, Germany. 

Footnotes

¹  All icons were selected from www.softicons.com, available under a Creative Commons Attribution license.