Open Access
Article  |   August 2020
Working memory is corrupted by strategic changes in search templates
Author Affiliations
  • Garry Kong
    Department of Psychology, New York University, Abu Dhabi, United Arab Emirates
    kong.garry@gmail.com
  • Jessica Meehan
    Department of Psychology, New York University, Abu Dhabi, United Arab Emirates
    jm5755@nyu.edu
  • Daryl Fougnie
    Department of Psychology, New York University, Abu Dhabi, United Arab Emirates
    darylfougnie@nyu.edu
Journal of Vision August 2020, Vol.20, 3. doi:https://doi.org/10.1167/jov.20.8.3
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Garry Kong, Jessica Meehan, Daryl Fougnie; Working memory is corrupted by strategic changes in search templates. Journal of Vision 2020;20(8):3. https://doi.org/10.1167/jov.20.8.3.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

When searching for a specific object, we often form an image of the target, which we use as a search template. This template is thought to be maintained in working memory, primarily because of evidence that the contents of working memory influences search behavior. However, it is unknown whether this interaction applies in both directions. Here, we show that changes in search templates influence working memory. Participants were asked to remember the orientation of a line that changed every trial, and on some trials (75%) search for that orientation, but on remaining trials recall the orientation. Critically, we manipulated the target template by introducing a predictable context—distractors in the visual search task were always counterclockwise (or clockwise) from the search target. The predictable context produced a large bias in search. Importantly, we also found a similar bias in orientation memory reports, demonstrating that working memory and target templates were not held as completely separate, isolated representations. However, the memory bias was considerably smaller than the search bias, suggesting that, although there is a common source, the two may not be driven by a single, shared process.

Introduction
We are often faced with the problem of finding a particular object in a crowded visual world. In looking for our keys on a desk or a friend in a crowd, we likely represent the object as a template to guide search. Influential theories suggest that this template recruits the working memory system (e.g., Desimone & Duncan, 1995). The most commonly suggested mechanism is that templates are maintained directly in working memory and thus are equivalent to working memory representations (Soto, Hodsoll, Rotshtein, & Humphreys, 2008), at least when targets change on a per trial basis (Woodman, Luck, & Schall, 2007). Versions of this theory vary in the nature of the relationship between the two constructs. According to some theories, being stored in working memory may be necessary but not sufficient for a representation to be a template (Dube & Al-Aidroos, 2019; Hollingworth & Hwang, 2013), that is, templates require some additional top–down process, such as attention (Gunseli, Meeter, & Olivers, 2014; van Driel, Gunseli, Meeter, & Olivers, 2017). There is evidence that attentional templates have independent properties from working memory representations, suggesting that the two can be dissociated (Carlisle & Woodman, 2011, 2013; Kerzel, 2019). However, most theories tend to favor a strong link between the two constructs. 
Evidence for the overlap of templates and working memory representations comes largely from studies showing that the contents of working memory influence attention, commonly referred to as memory-driven attentional capture (Downing, 2000; Olivers, Meijer, & Theeuwes, 2006; Soto, Heinke, Humphreys, & Blanco, 2005). Here we examine this interaction in the other direction. Previous research has found that memory representations improve due to visual search (Rajsic, Ouslis, Wilson, & Pratt, 2017; Williams, Henderson, & Zacks, 2005). However, it is difficult to disentangle the memory improvement because of an item becoming the search template from the general memory improvement that comes from attending to an item for longer, either while present or within memory (i.e., the retro-cue effect; Griffin & Nobre, 2003). Instead, we will directly ask whether changes in the target template will influence memory reports. If target templates are equivalent to memory representations, then biases in one will produce equivalent biases to the other. Alternatively, it is possible for working memory representations to influence search, without a reciprocal interaction in the other direction. For example, bias may originate from attention-based rehearsal (Awh, Jonides, & Reuter-Lorenz, 1998), such as, a byproduct of activating feature channels to maintain a working memory representation is that activation biases search toward those same features. 
The present work tests an important assumption of theories that view target templates and working memory representations as one and the same. Previous work has shown that search context (or other factors) can influence target reports (Geng, DiQuattro, & Helm, 2017; Scolari, Byers, & Serences, 2012; Scolari & Serences, 2009; Yu & Geng, 2019). For example, one study (Yu & Geng, 2019) found that a block of visual search trials that encouraged participants to tune an attentional template away from a target would in turn also bias subsequent memory reports away from the target. However, because the attentional manipulation and memory reports were displaced in time, it was not possible to directly compare the impact of the attentional manipulation on memory. Furthermore, because the target and distractors were constant across the experiment, overtraining and familiarity with those specific stimuli may have played a role in their results. Here, we manipulate participants’ expectations of the distractors to manipulate the search template, allowing us to present a different search target and distractors on every trial. Furthermore, because the search template must be recreated every trial, we can directly investigate whether a shift in the search template brings a concomitant change to the memory representation of the target. 
Experiment 1
Participants were asked to remember the orientation of a standard and either search for that orientation (75% of trials) or report that orientation from memory (25% of trials). To manipulate the search template, we introduced a predictable context—distractors in the visual search task were drawn from orientations that were counterclockwise (or clockwise for some participants) from the target, encouraging a shift in the target template away from expected distractors (e.g., clockwise), improving search performance (Becker, 2010; Becker, Folk, & Remington, 2013; Navalpakkam & Itti, 2007). 
Critically, the orientation of the standard changed on every trial, which necessitated a change in both the search template and the memory representation. Furthermore, because participants did not know whether the trial was a search or memory recall trial until the trial started, we can be sure that neither search behavior nor any stimuli after the preview display would affect memory performance. If participants’ underlying working memory representations are independent of the target templates, then strategically altering target templates should not degrade memory performance. However, if the target template is inextricably linked to the underlying memory representation, then we would expect memory responses to be biased away from the expected search context, matching the shift in search template. 
Methods
Participants
Twenty-eight participants, 18 female, mean age 20.2 years, range 18 to 29 years took part in this experiment. This number was chosen because a power analysis found that 27 people or more would give us a greater than 80% chance of finding an existing effect (assuming a Cohen's d of 0.5 or greater). Participants had normal or corrected-to-normal vision. All participants were naïve as to the purpose of the experiment. Participants were recruited at New York University Abu Dhabi and took part in exchange for course credit or alternatively received a subsistence allowance of 50 AED per hour. 
Written consent was obtained from each participant before the experiments. The experiments were approved by the New York University Abu Dhabi Institutional Review Board in accordance with the ethical principles of the Belmont Report. 
Apparatus and stimuli
Stimuli were presented on a 24-inchBenQ XL2411 monitor (60 Hz refresh rate, 1920 × 1080 pixels) that was placed 57 cm away from the participant. The stimuli were generated in Matlab using the Psychophysics Toolbox extensions (Kleiner, Brainard, & Pelli, 2007). The background of the screen was grey (33.0 cd/m2) and kept constant during the experiment. 
As seen in Figure 1a, stimuli in the experiment were arrangements of oriented lines. All lines used were white (100.0 cd/m2), 1.2° in length and Gaussian blurred along its length to hide pixel artifacts that could be used to indicate orientation. During target preview, a standard stimulus line was presented for 500 ms at the center of the screen, with a random orientation, re-randomized every trial. During visual search trials, four lines (three distractors and one odd-one-out search target) were presented 2.4° above, below, left, and right of the center. The search target had an equal chance of being at any of the four locations, and its orientation was randomly jittered from the standard. The amount of jitter was drawn from the following values: −46.5°, −25.6°, −20.7°, −16.8°, −13.5°, −10.5°, −7.7°, −5.1°, −2.5°, 0°, 2.5°, 5.1°, 7.7°, 10.5°, 13.5°, 16.8°, 20.7°, 25.6°, or 46.5° (1st, 99th and every 5th percentile between 10th and 90th of a normal distribution with a standard deviation [SD] of 20). Distractors were set to be between 25° and 40° clockwise or counterclockwise of the search target, values that were chosen to be far enough such that the odd-one-out was immediately noticeable, but with a small enough pop-out effect that there was a benefit of top–down knowledge of the distractors (Foster & Ward, 1991). Note that the orientations of the distractors were pegged to the search target, not the standard, and as such all search displays were geometrically identical, with the exception of the slight amount of orientation jitter in the distractors. During memory trials, a response line with a randomized starting orientation was placed at the center to be manipulated by the participant. 
Figure 1.
 
(A) Trial sequence for Experiment 1. Participants were instructed which direction distractors will be oriented relative to the targets (here, counterclockwise). A preview of the true target is followed by a visual search display on 75% of trials, or a memory recall task on 25% of trials. Note that only one visual search display is shown on any one search trial. The search displays shown here represent two possible search displays in which a different and same response would be the correct answer. (B) Probability of a same response in the visual search task, as a function of the amount that the search target was jittered from the true target. Orientations reported have been realigned such that positive values were away from the distractors. Data are averaged across all participants. Error bars represent 95% CIs. (C) Mean attentional and memory bias, averaged across participants. Error bars represent 95% CIs, and dots represent each participant's bias.
Figure 1.
 
(A) Trial sequence for Experiment 1. Participants were instructed which direction distractors will be oriented relative to the targets (here, counterclockwise). A preview of the true target is followed by a visual search display on 75% of trials, or a memory recall task on 25% of trials. Note that only one visual search display is shown on any one search trial. The search displays shown here represent two possible search displays in which a different and same response would be the correct answer. (B) Probability of a same response in the visual search task, as a function of the amount that the search target was jittered from the true target. Orientations reported have been realigned such that positive values were away from the distractors. Data are averaged across all participants. Error bars represent 95% CIs. (C) Mean attentional and memory bias, averaged across participants. Error bars represent 95% CIs, and dots represent each participant's bias.
Design and procedure
A summary of the procedure can be seen in Figure 1a. At the beginning of the experiment participants are given instructions about the possible orientation values of search distractors relative to the target (e.g., in Figure 1a, counterclockwise from the target). This was done by telling the participant, both verbally and in written words, that all distractors in the search trials would be at a fixed rotation from the search target (clockwise or counterclockwise, counterbalanced across subjects). The purpose of this instruction was to modify participants’ expectations to create a search context. 
Each trial began with a 500-ms preview of the standard orientation (the orientation that participants were to use to classify search responses), followed by a 500-ms blank screen. On 75% of trials, this would be followed by a visual search task. The visual search display consisted of an odd-one-out search target (with orientation jittered from the true target), and three distractors (with orientations matching the context expectation, e.g., in Figure 1a, counterclockwise of the search target) was presented for 500 ms. Participants were instructed to respond as quickly as possible to whether the odd-one-out search target was the same or at a different orientation to the standard shown during the preview (the response screen was present until response). The search task is essentially the same as the target present/absent visual search tasks typically used in visual search (e.g., Treisman & Gelade, 1980), with same and different replacing present and absent responses, respectively. The other major difference is that we imposed a time limit of 500 ms on display presentation. This limit makes it so that participants must guess if they have not found the target within the allotted time and therefore makes accuracy the primary measure of performance instead of reaction time (Foster & Ward, 1991; Kong, Alais, & Van der Burg, 2017). High levels of performance (responding same when the target matches the standard, responding different when the target does not match the standard) can only be achieved when the search template is efficient enough to filter out the distractors in time to find the target. 
Responses were given using a computer keyboard (right arrow for same, left arrow for different). No immediate feedback was provided for visual search trials, because we did not want feedback to affect the search template manipulation. Instead, to incentivize good performance, participants were given an accuracy score for their visual search trials on each break screen. Because the search target and true target were only the same when there was 0° jitter (approximately 1 in 19 trials), we counted all trials with ±12° jitter as requiring a same response for a correct answer, and all trials with a jitter of greater than ±12° jitter as requiring a different response for a correct answer, only for the purpose of computing an accuracy score for giving feedback to participants (i.e., this was irrelevant to all results described elsewhere in this article). Twelve degrees was chosen as the cut-off point because it would divide trials into an approximately even number of same and different trials. 
On the remaining 25% of trials, participants performed a memory recall task. A randomly oriented line would appear along with the question, “What orientation was the target?” Participants would then have unlimited time to rotate the line to match their memory and confirm their response with a mouse click. Participants were instructed to match the orientation as precisely as possible from memory, and that responses were unspeeded. For response feedback, the fixation dot would turn green if the target was within 20° of the true value or red if it was not. The next trial would then commence after 500 ms. Participants completed 800 trials, with a break every 40 trials. 
Results and discussion
The mean response time in the search task was 0.72 ± 0.26 seconds. To confirm that we were successful in inducing a biased search template, we looked at the proportion of same responses as a function of the difference between true target and the search target, as shown in Figure 1b. For ease of reporting, all orientations reported have been realigned such that positive values are away from the distractors. As can be seen in Figure 1b, participants were more likely to judge a search target that is jittered away from the distractors as the same, suggesting that distractor context altered the search template. This finding is consistent with past results from visual search suggesting that participants modulate a search template to be more distinct from expected distractors to improve search efficiency (Becker, 2010; Navalpakkam & Itti, 2007). To quantify this finding, we multiplied the magnitude of the search target jitter value with the proportion of same responses for each value, effectively creating a weighted vector for each jitter orientation. We then took the circular mean of these vectors, for each subject, giving an overall mean search template bias of 2.90° ± 0.86°. A one-sample t test found that this was significantly different from zero, t(27) = 17.75, p < 0.001, Cohen's d = 3.35. This finding confirms that participants’ search templates had shifted away from expected distractors. 
Having found evidence that the predictive context successfully biased search templates away from distractor orientations, we then analyzed the memory trials to see if we could find a commensurate bias there. To analyze the memory data, we first computed the circular mean of each participant's memory errors (orientation of the adjusted line minus the orientation of the standard), giving a mean of 0.92° ± 1.60°. A one-sample t test found that this was significantly different from zero, t(27) = 3.05, p = 0.005, Cohen's d = 0.58. To provide another estimate of bias, one that takes into account trials that might be random guess trials (which could influence bias estimates), we performed a mixture modeling analysis. This analysis estimates the proportion of guess responses and attempts to measure the distribution and bias of true memory responses (Zhang & Luck, 2008). Specifically, we fit the memory error to the sum of a uniform and von Mises distribution, corrected for the 180° feature space, using MemToolbox (Suchow, Brady, Fougnie, & Alvarez, 2013). We found estimates of guess rate (5.93%) and SD (12.42°) consistent with those of past studies with similar parameters. Critically, we still observed a significant bias (0.79°) away from the expected distractors, t(27) = 2.97, p = 0.006, Cohen's d = 0.56, when compared with 0°. This analysis accords with the model free analysis in showing a change in the search template due to expectations. 
A follow-up analysis was performed to examine whether this bias could be explained by participants responding to the target template on a proportion of trials, rather than a biased response distribution. Specifically, we used a mixture model analysis (Zhang & Luck, 2008) to determine whether the pattern of memory reports were better explained by a model where participants were not biased, but misreported the responses centered on the template (rather than the stimulus) on some proportion of trials. This model was implemented as a swap model (Bays, Catalao, & Husain, 2009), with the template center as a potential “swapped” response. The search template bias for each participant (mean of 2.90°; calculated using the circular mean of weighted same responses, as discussed elsewhere in this article), was used as the center of the search template responses. For completeness, we fit the data to a swap model with and without a bias parameter and compared the fits with the standard mixture model of a guess rate, a memory precision, and a bias parameter (Zhang & Luck, 2008). Using Memtoolbox (Suchow et al., 2013), we found no support that the data can be explained without a bias toward the prior. The best fitting model, evaluated using an Akaike information criterion (AIC) (Burnham, Anderson, & Huyvaert, 2011) to account for the number of free parameters, was the model with bias but without a swap parameter. The model with a swap and bias parameter fit worse for all 28 participants with an AIC penalty of 2.00. The model with only a swap parameter fit worse for 21 of 28 participants with an AIC penalty of 2.04. The distribution of memory errors and the fit to the best model is shown in Figure 2a. 
Figure 2.
 
Distribution of memory errors (red bars) and model predictions of the best-fitting model (blue line and shaded area) for Experiments 1 (A) and 2 (B). Positive errors represent a response away from the distractors. Shaded blue areas represent ± 1 standard error of the mean from the mean model prediction.
Figure 2.
 
Distribution of memory errors (red bars) and model predictions of the best-fitting model (blue line and shaded area) for Experiments 1 (A) and 2 (B). Positive errors represent a response away from the distractors. Shaded blue areas represent ± 1 standard error of the mean from the mean model prediction.
Importantly, because we induce and measure a bias in both search and in working memory, our approach lets us compare the size of the bias for the two responses. This point is important because, although many theories suggest that two are related, there is debate over the degree to which target templates and working memory representations are equivalent. We observed a bias in the attention task (2.90°) that was about three times that of the memory task (0.92°). Is this evidence for a significantly larger attention bias? A paired-samples t test suggested that this was indeed the case, t(27) = 5.34, p < 0.001, 95% confidence interval of the difference, 1.22 to 2.75, Cohen's d = 1.01. However, this does not tell us how much bigger the measured attention bias is than the memory bias, in terms of a ratio (e.g., Carlisle & Woodman, 2013). To compute the confidence interval for the ratio of attention and memory biases, we used a bootstrapping procedure (Efron, 1981). For each resample, we sampled with replacement 600 search trials and 200 memory trials within each participant, to generate a ratio of the two biases. This process was repeated 100,00 times to give a 95% confidence interval of 2.09 to 6.24, with a median of 3.14. Note that this confidence interval does not cross 1, which accords with the previous t test that suggested that attention bias was greater than the working memory bias. This finding seems to accord with the idea that, although there is a shared bias, it is too simplistic to consider the two as being equivalent constructs. However, caution is necessary in interpreting the relative size of the two effects given the differences in how the effects were measured. 
Note that this difference between the attention and memory biases was not found in a recent study (Kerzel, 2020), where the attentional bias was measured by taking advantage of the effect of cueing a target varying with the similarity between the cue and the target. By presenting only distractors on one side of the feature space from the target in a visual search task, they found that the peak cuing effect would shift away from the distractors, and that this shift roughly matched the shift of subsequent memory reports for the target. However, there are numerous factors as to why our results may differ. The memory stimulus in the Kerzel study was held constant throughout the study and always presented with the same distractors. The influence of long-term memory, both for the distractors and the targets, could have played a role in the increased bias away from the distractors relative to our study. Or perhaps the common bias between attention and working memory takes time to build up, and the fact that we changed our target on a per-trial basis may have reduced the effect. Additionally, the Kerzel study specifically avoided relational guidance of search (Becker, 2010; Becker, Folk & Remington, 2013), whereas we encouraged it in this study. This difference could impact the results, because recent research has questioned whether relational guidance is the same as the shifting of a search template (York & Becker, 2020). Finally, the tasks measured bias in distinct ways and the differences in methodology could explain the discrepant results. For one, it is often difficult to disentangle search from postsearch processes (Mruczek & Sheinberg, 2005; Zhang & Onyper, 2020) and, therefore, caution is needed to interpret our results as being purely driven by changes in the search template. 
We measured attention bias using a search target in the presence of distractors presented to one side of the stimulus space, whereas the memory stimulus was always presented in isolation. Could these distractors influence search responses in the absence of changes to the attentional template? Perhaps the results reflect a perceptual bias caused by the presence of the distractors in the search display, similar to a tilt illusion (Gibson, 1937), or perceptual contamination from memory (Kang, Hong, Blake, & Woodman, 2011). We address this possibility in Experiment 2
Experiment 2
In this experiment, we measure whether there is a bias in search performance caused by including distractors that were all either clockwise or counterclockwise from the target. To do this, we will remove the search context, that is, the instruction that the distractors would always be rotated in the same direction from the target. Further, the standard stimulus was no longer relevant for the search task and was removed. Instead, we presented an array of four oriented lines and asked participants to report the orientation of the deviant line using an adjustment task (Figure 3). This array was equivalent to the search displays in Experiment 1. The three distractors were shifted in a consistent direction from the target as in Experiment 1 (e.g., a participant could have clockwise-oriented distractors throughout the study). Although the participants may be able to learn this pattern implicitly, critically, they had no information about the likely orientation of the target and, thus, could not explicitly create a target template. An absence of bias (or at least an absence of bias away from the distractors) would be evidence against the concern that the bias observed in Experiment 1 was due to the display structure and not due to participants altering the target template in response to the expected search context. 
Figure 3.
 
Trial sequence for Experiment 2. Participants were instructed to remember the orientation of the line that is the odd one out, then adjust the orientation of the response bar to match the orientation of the deviant line.
Figure 3.
 
Trial sequence for Experiment 2. Participants were instructed to remember the orientation of the line that is the odd one out, then adjust the orientation of the response bar to match the orientation of the deviant line.
Methods
Participants
Twenty-eight participants, 17 female, mean age 21.0 years, range 18 to 29 years, took part in this experiment. 
Design and procedure
Trials began with the presentation of an array of four oriented line consisting of one target and three distractors. These displays were as in Experiment 1. The three distractors were oriented away from the target by 25° to 40° in a consistent manner across blocks (e.g., all clockwise). Participants knew to identify the target as the odd one out and were to subsequently report its orientation. After 500 ms, the orientation adjustment screen (equivalent to that for Experiment 1) was displayed. Participants adjusted the bar to match the orientation of the search target. Feedback was given in the same way as in the memory trials of Experiment 1. One-half of the participants saw distractors that were clockwise from the target in each display, whereas for the other one-half, distractors were counterclockwise. Participants performed 200 trials of this task. 
Results and discussion
The circular mean of each participant's memory errors (orientation of the reported orientation relative to the true orientation of the search target) was calculated with positive error values reflecting responses toward the distractor values. The resulting error was 7.55° ± 5.91°, revealing a strong bias toward the distractors. A t test suggested that this was significantly different from zero, t(27) = 5.52, p < 0.001, Cohen's d = 1.28. 
Whereas Experiment 1 memory responses were biased away from distractors, here we find the opposite pattern. However, this bias toward distractors likely reflects contamination from the search task on memory reports. In Experiment 1, the bias is away from expected distractors. The to-be-reported stimulus is shown in isolation at trial start. In the current study, participants are shown the target and distractors at the same time. Further, the short presentation of the stimulus creates the possibility that participants may have failed to isolate the target in time. On these trials they may have reported the orientation of a nontarget item, thereby impacting bias measures. To address this factor, we fit the responses to the same three models used in Experiment 1, a standard mixture model with precision, guess rate, and bias parameters; a swap model with precision, guess rate, and swap parameters (with swaps centered on the orientation of each of the three distractors); and a combination of the two, with precision, guess rate, bias, and swap parameters. 
We found that the model with both bias and swap parameters fit better than the models with bias only (an AIC change of 15.46, with a better fit for 23 of 28 participants) and swap only (an AIC change of 4.64, with a better fit for 27 of 28 participants) (this outcome is in contrast with Experiment 1, which did not find evidence for a superiority for swap models). The estimate of the guess rate was 2.68%, SD was 14.26°, and swap rate was 22.42%. Importantly, the bias after accounting for guesses and swaps was 0.24° toward the distractors, which was not significant, 95% confidence interval −1.37 to 1.96, t(27) = 0.36, p = 0.719, Cohen's d = 0.07. The memory error distribution and model fits are shown in Figure 2b. These findings demonstrate that the bias away from the distractors observed in Experiment 1 is no longer observed once search context expectations are removed. Indeed, a direct comparison of the attentional bias in Experiment 1 and the bias in this experiment (using a nonparametric Mann-Whitney U test because our data violated the assumptions of an independent sample t test) found a significant difference, Z = 4.42, p < 0.001. Furthermore, the observed bias is less than 1°, and is therefore too small to explain the larger effect size difference between the two tasks. The Mann-Whitney U test also found a significance difference between this bias in this experiment and the memory bias in Experiment 1, Z = 2.40, p = 0.016, demonstrating that the orientation adjustment task is sensitive enough to detect the difference between a biased and nonbiased distribution. These findings reinforce the conclusion that the results of Experiment 1 likely reflect (at least) a partially overlapping mechanism responsible for influencing search template and memory contents. 
General discussion
How do we find things in our visual world? Several theories propose that representations of the target item are placed in working memory to act as a target template. Evidence for this includes demonstrations that working memory contents influence attention, for example, it is drawn toward items in working memory (Downing, 2000; Olivers et al., 2006; Soto et al., 2005), at least under certain conditions (Woodman et al., 2007). This work has established a link between working memory and target templates, but does this link work in both directions? In this study, we show that strategic changes to target templates also results in a bias to working memory reports, even though such biases would be detrimental to memory accuracy. Furthermore, the combination of interleaved search and memory trials and different target orientations every trial ensured that the bias in memory reports was due to the mere creation of a search template, without the requirement to actually use it. This corruption from attention onto memory reports is further evidence of a relationship between the two constructs. By establishing a bidirectional relationship, this study provides strong support of a link between search templates and working memory. Moreover, it challenges the possibility that memory-driven capture is caused by a strategy to refresh memory (Woodman & Luck, 2007) or that it arises from top–down processes involved in maintaining working memory filtering down to attention, without effects in the other direction. 
Another advantage of our approach is that we can measure the relative bias from the predictable context on both the search task and the changes on memory report. If target templates are reducible to working memory representations and do not have separate existence or properties, then the resulting bias should be equivalent for both. However, the effect of a working memory bias was smaller than the search template bias. Admittedly, the attention and memory biases in our study were measured using two different techniques, which may add some unforeseen noise to the comparison. However, this study, in conjunction with existing evidence (Carlisle & Woodman, 2013; Kerzel, 2019), provides a challenge to the theory that a single, shared mechanism is responsible for both search templates and memory representations. 
How do we reconcile the bidirectional relationship between search templates and memory representations with the difference in the magnitude of their biases? One possibility is that there is a shared mechanism responsible for both search templates and memory representations, but also an extra process that adds to the search template, without affecting the memory representation. Evidence for two components to memory-driven attentional capture has been previously observed, usually in the form for an automatic component and a strategic component. The argument for an automatic component is supported by evidence that memory-driven attentional capture happens when it is detrimental to visual search performance (Soto et al., 2005; Soto et al., 2008; Soto, Humphreys, & Heinke, 2006). In contrast, the strength of attentional capture owing to a memory stimulus is modulated by how predictive it is of the subsequent search target (Carlisle & Woodman, 2011; Kawashima & Matsumoto, 2017; Kiyonaga, Egner, & Soto, 2012; Woodman & Luck, 2007), and that the ability to modulate the capture effect seems to require time and, presumedly, cognitive effort (Han & Kim, 2009). Therefore, one possibility is that one of these two components of memory-driven capture is affecting the working memory representation, whereas the other does not. Future work will be necessary to determine whether the bias in working memory reports the automatic or strategic component of attentional capture and to establish that the difference does indeed reflect only a partially overlapping mechanism. 
Another possibility is that the bias in search task responses arises both from the strategic corruption of a memory-based search template as well as a decisional component that is independent of the stored template, resulting in an overestimation of our measured search bias. For example, one possible decisional component is a bias in search reports that arises because of a greater willingness to respond “same” when the target was jittered away from the expected orientation of the distractors, which would explain why reports continued to be biased toward same even when the search target was jittered 46.5° away from the standard. Importantly, the fact that we observed significant biases in both memory and search suggests that the effect on search cannot arise solely owing to changes in decision thresholds and still points to a shared mechanism. Indeed, this account may not be wholly distinct from the automatic versus strategic account, because the strategic component of memory-driven capture may plausibly arise at decision stages. 
Regardless of the exact mechanism, this study has found that inducing a bias in a search template changes reports in a memory task in a manner consistent with a (at least partially) shared source. Thus, the interaction between search templates and the underlying working memory representation works in both directions. However, a much larger bias for search responses suggests that neither are attention and working memory biases wholly dissociable or driven by a single, shared source. 
Acknowledgments
The authors thank Christian Olivers and Joy Geng for their comments on an earlier version of this manuscript. This research was supported by a capstone grant awarded to JM by New York University Abu Dhabi. 
Commercial relationships: none. 
Corresponding author: Garry Kong. 
Email: gk426@cam.ac.uk. 
Address: University of Cambridge, Downing Site CB2 3DT, Cambridge, UK. 
References
Awh, E., Jonides, J., & Reuter-Lorenz, P. A. (1998). Rehearsal in spatial working memory. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 780–790, doi:10.1037/0096-1523.24.3.780. [CrossRef]
Bays, P. M., Catalao, R. F. G., & Husain, M. (2009). The precision of visual working memory is set by allocation of a shared resource. Journal of Vision, 9(10), 7–7, doi:10.1167/9.10.7. [CrossRef]
Becker, S. I. (2010). The role of target-distractor relationships in guiding attention and the eyes in visual search. Journal of Experimental Psychology. General, 139(2), 247–265, doi:10.1037/a0018808. [CrossRef]
Becker, S. I., Folk, C. L., & Remington, R. (2013). Attentional capture does not depend on feature similarity, but on target-nontarget relations. Psychological Science, 24(5), 634–647, doi:10.1177/0956797612458528. [CrossRef]
Burnham, K. P., Anderson, D. R., & Huyvaert, K. P. (2011). AIC model selection and multimodel inference in behavioral ecology: Some background, observations, and comparisons. Behavioral Ecology and Sociobiology, 65(1), 23–35, doi:10.1007/s00265-010-1029-6. [CrossRef]
Carlisle, N. B., & Woodman, G. F. (2011). Automatic and strategic effects in the guidance of attention by working memory representations. Acta Psychologica (Amst), 137(2), 217–225, doi:10.1016/j.actpsy.2010.06.012. [CrossRef]
Carlisle, N. B., & Woodman, G. F. (2013). Reconciling conflicting electrophysiological findings on the guidance of attention by working memory. Attention, Perception & Psychophysics, 75(7), 1330–1335, doi:10.3758/s13414-013-0529-7. [CrossRef]
Desimone, R., & Duncan, J. (1995). Neural mechanisms of selective visual attention. Annual Review of Neuroscience, 18(1), 193–222, doi:10.1146/annurev.ne.18.030195.001205. [CrossRef]
Downing, P. E. (2000). Interactions between visual working memory and selective attention. Psychological Science, 11(6), 467–473, doi:10.1111/1467-9280.00290. [CrossRef]
Dube, B., & Al-Aidroos, N. (2019). Distinct prioritization of visual working memory representations for search and for recall. Attention, Perception & Psychophysics, 81(5), 1253–1261, doi:10.3758/s13414-018-01664-6. [CrossRef]
Efron, B. (1981). Nonparametric estimates of standard error: The jackknife, the bootstrap and other methods. Biometrika, 68(3), 589–599, doi:10.1093/biomet/68.3.589. [CrossRef]
Foster, D. H., & Ward, P. A. (1991). Asymmetries in oriented-line detection indicate two orthogonal filters in early vision. Proceedings. Biological Sciences, 243(1306), 75–81, doi:10.1098/rspb.1991.0013. [CrossRef]
Geng, J. J., DiQuattro, N. E., & Helm, J. (2017). Distractor probability changes the shape of the attentional template. Journal of Experimental Psychology. Human Perception and Performance, 43(12), 1993–2007, doi:10.1037/xhp0000430. [CrossRef]
Gibson, J. J. (1937). Adaptation, after-effect, and contrast in the perception of tilted lines. II. Simultaneous contrast and the areal restriction of the after-effect. Journal of Experimental Psychology, 20(6), 553–569, doi:10.1037/h0057585. [CrossRef]
Griffin, I. C., & Nobre, A. C. (2003). Orienting attention to locations in internal representations. Journal of Cognitive Neuroscience, 15(8), 1176–1194, doi:10.1162/089892903322598139. [CrossRef]
Gunseli, E., Meeter, M., & Olivers, C. N. (2014). Is a search template an ordinary working memory? Comparing electrophysiological markers of working memory maintenance for visual search and recognition. Neuropsychologia, 60, 29–38, doi:10.1016/j.neuropsychologia.2014.05.012. [CrossRef]
Han, S. W., & Kim, M. S. (2009). Do the contents of working memory capture attention? Yes, but cognitive control matters. Journal of Experimental Psychology. Human Perception and Performance, 35(5), 1292–1302, doi:10.1037/a0016452. [CrossRef]
Hollingworth, A., & Hwang, S. (2013). The relationship between visual working memory and attention: Retention of precise colour information in the absence of effects on perceptual selection. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 368(1628), 20130061, doi:10.1098/rstb.2013.0061. [CrossRef]
Kang, M. S., Hong, S. W., Blake, R., & Woodman, G. F. (2011). Visual working memory contaminates perception. Psychonomic Bulletin & Review, 18(5), 860–869, doi:10.3758/s13423-011-0126-5. [CrossRef]
Kawashima, T., & Matsumoto, E. (2017). Cognitive control of attentional guidance by visual and verbal working memory representations. Japanese Psychological Research, 59(1), 49–57, doi:10.1111/jpr.12141. [CrossRef]
Kerzel, D. (2019). The precision of attentional selection is far worse than the precision of the underlying memory representation. Cognition, 186, 20–31, doi:https://doi.org/10.1016/j.cognition.2019.02.001. [CrossRef]
Kerzel, D. (2020). Direct evidence for the optimal tuning of attention. Journal of Experimental Psychology: Human Perception and Performance, 46(7), 716–728, [CrossRef]
Kiyonaga, A., Egner, T., & Soto, D. (2012). Cognitive control over working memory biases of selection. Psychonomic Bulletin & Review, 19(4), 639–646, doi:10.3758/s13423-012-0253-7. [CrossRef]
Kleiner, M., Brainard, D., & Pelli, D. (2007). What's new in Psychtoolbox-3. Perception, 36 ECVP Abstract Supplement.
Kong, G., Alais, D., & Van der Burg, E. (2017). Orientation categories used in guidance of attention in visual search can differ in strength. Attention, Perception, & Psychophysics, 79(8), 2246–2256, doi:10.3758/s13414-017-1387-5. [CrossRef]
Mruczek, R. E. B., & Sheinberg, D. L. (2005). Distractor familiarity leads to more efficient visual search for complex stimuli. Perception & Psychophysics, 67(6), 1016–1031, doi:10.3758/BF03193628. [CrossRef]
Navalpakkam, V., & Itti, L. (2007). Search goal tunes visual features optimally. Neuron, 53(4), 605–617, doi:10.1016/j.neuron.2007.01.018. [CrossRef]
Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of experimental psychology. Human perception and performance, 32(5), 1243–1265, doi:10.1037/0096-1523.32.5.1243. [CrossRef]
Rajsic, J., Ouslis, N. E., Wilson, D. E., & Pratt, J. (2017). Looking sharp: Becoming a search template boosts precision and stability in visual working memory. Attention, Perception & Psychophysics, 79(6), 1643–1651, doi:10.3758/s13414-017-1342-5. [CrossRef]
Scolari, M., Byers, A., & Serences, J. T. (2012). Optimal deployment of attentional gain during fine discriminations. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 32(22), 7723–7733, doi:10.1523/jneurosci.5558-11.2012. [CrossRef]
Scolari, M., & Serences, J. T. (2009). Adaptive allocation of attentional gain. Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 29(38), 11933–11942, doi:10.1523/jneurosci.5642-08.2009. [CrossRef]
Soto, D., Heinke, D., Humphreys, G. W., & Blanco, M. J. (2005). Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance, 31(2), 248–261, doi:10.1037/0096-1523.31.2.248. [CrossRef]
Soto, D., Hodsoll, J., Rotshtein, P., & Humphreys, G. W. (2008). Automatic guidance of attention from working memory. Trends in Cognitive Sciences, 12(9), 342–348, doi:10.1016/j.tics.2008.05.007. [CrossRef]
Soto, D., Humphreys, G. W., & Heinke, D. (2006). Working memory can guide pop-out search. Vision Research, 46(6), 1010–1018, doi:10.1016/j.visres.2005.09.008. [CrossRef]
Suchow, J. W., Brady, T. F., Fougnie, D., & Alvarez, G. A. (2013). Modeling visual working memory with the MemToolbox. Journal of Vision, 13(10):9, doi:10.1167/13.10.9. [CrossRef]
Treisman, A., & Gelade, G. (1980). A feature-integration theory of attention. Cognitive Psychology, 12(1), 97–136, [CrossRef]
van Driel, J., Gunseli, E., Meeter, M., & Olivers, C. N. (2017). Local and interregional alpha EEG dynamics dissociate between memory for search and memory for recognition. NeuroImage, 149, 114–128, doi:10.1016/j.neuroimage.2017.01.031. [CrossRef]
Williams, C. C., Henderson, J. M., & Zacks, R. T. (2005). Incidental visual memory for targets and distractors in visual search. Perception & Psychophysics, 67(5), 816–827, doi:10.3758/bf03193535. [CrossRef]
Woodman, G. F., & Luck, S. J. (2007). Do the contents of visual working memory automatically influence attentional selection during visual search? Journal of experimental psychology. Human perception and performance, 33(2), 363–377, doi:10.1037/0096-1523.33.2.363. [CrossRef]
Woodman, G. F., Luck, S. J., & Schall, J. D. (2007). The role of working memory representations in the control of attention. Cerebral Cortex, 17(Suppl. 1), i118–i124, doi:10.1093/cercor/bhm065. [CrossRef]
York, A., & Becker, S. I. (2020). Top-down modulation of gaze capture: Feature similarity, optimal tuning, or tuning to relative features? Journal of Vision, 20(4), 6–6, doi:10.1167/jov.20.4.6. [CrossRef]
Yu, X., & Geng, J. J. (2019). The attentional template is shifted and asymmetrically sharpened by distractor context. Journal of experimental psychology. Human perception and performance, 45(3), 336–353, doi:10.1037/xhp0000609. [CrossRef]
Zhang, W., & Luck, S. J. (2008). Discrete fixed-resolution representations in visual working memory. Nature, 453, 233, doi:10.1038/nature06860. [CrossRef]
Zhang, Y., & Onyper, S. (2020). Visual search asymmetry depends on target-distractor feature similarity: Is the asymmetry simply a result of distractor rejection speed? Attention, Perception, & Psychophysics, 82(1), 80–97, doi:10.3758/s13414-019-01818-0. [CrossRef]
Figure 1.
 
(A) Trial sequence for Experiment 1. Participants were instructed which direction distractors will be oriented relative to the targets (here, counterclockwise). A preview of the true target is followed by a visual search display on 75% of trials, or a memory recall task on 25% of trials. Note that only one visual search display is shown on any one search trial. The search displays shown here represent two possible search displays in which a different and same response would be the correct answer. (B) Probability of a same response in the visual search task, as a function of the amount that the search target was jittered from the true target. Orientations reported have been realigned such that positive values were away from the distractors. Data are averaged across all participants. Error bars represent 95% CIs. (C) Mean attentional and memory bias, averaged across participants. Error bars represent 95% CIs, and dots represent each participant's bias.
Figure 1.
 
(A) Trial sequence for Experiment 1. Participants were instructed which direction distractors will be oriented relative to the targets (here, counterclockwise). A preview of the true target is followed by a visual search display on 75% of trials, or a memory recall task on 25% of trials. Note that only one visual search display is shown on any one search trial. The search displays shown here represent two possible search displays in which a different and same response would be the correct answer. (B) Probability of a same response in the visual search task, as a function of the amount that the search target was jittered from the true target. Orientations reported have been realigned such that positive values were away from the distractors. Data are averaged across all participants. Error bars represent 95% CIs. (C) Mean attentional and memory bias, averaged across participants. Error bars represent 95% CIs, and dots represent each participant's bias.
Figure 2.
 
Distribution of memory errors (red bars) and model predictions of the best-fitting model (blue line and shaded area) for Experiments 1 (A) and 2 (B). Positive errors represent a response away from the distractors. Shaded blue areas represent ± 1 standard error of the mean from the mean model prediction.
Figure 2.
 
Distribution of memory errors (red bars) and model predictions of the best-fitting model (blue line and shaded area) for Experiments 1 (A) and 2 (B). Positive errors represent a response away from the distractors. Shaded blue areas represent ± 1 standard error of the mean from the mean model prediction.
Figure 3.
 
Trial sequence for Experiment 2. Participants were instructed to remember the orientation of the line that is the odd one out, then adjust the orientation of the response bar to match the orientation of the deviant line.
Figure 3.
 
Trial sequence for Experiment 2. Participants were instructed to remember the orientation of the line that is the odd one out, then adjust the orientation of the response bar to match the orientation of the deviant line.
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×