Look before you seek: Preview adds a fixed benefit to all searches

Abstract

Knowing in advance what to look for speeds up search, but how this knowledge guides search is poorly understood. The little available evidence suggests that previewing the target produces larger reductions in search times for harder searches. To investigate this issue further, we performed seven experiments in which subjects searched for an oddball target after previewing the target, distracter, or an unrelated square. Consistent with previous studies, harder searches showed bigger reductions in search time for an informative preview. However, the same data replotted using the reciprocal of search time showed a remarkably different result: The informative preview showed a fixed additive increase in reciprocal search time across all searches regardless of difficulty. This is a nontrivial outcome because it cannot be explained using a simple relationship between search times in the informative and uninformative preview conditions. We interpret our findings by proposing that the reciprocal of search time reflects the strength of an underlying accumulating signal related to the distinctiveness or salience of the target over the distracters and that preview additively increases this signal for all searches. This in turn implies that the top-down signals related to target preview and bottom-up signals related to target-distracter salience sum linearly.

Introduction

We search for visual objects with varying specificity: We might search for anything (“find something interesting”), an item from a category (“find an animal”), or for a very specific item (“find the meerkat”). How does knowing what to look for help in search? This is a fundamental question because it concerns how top-down knowledge influences visual processing. Classic studies of this issue have shown that previewing a target before the onset of the search display speeds up search (Wolfe et al., 2004; Vickery, King, & Jiang, 2005; Schmidt & Zelinsky, 2009, 2011; Maxfield & Zelinsky, 2012; Maxfield, Stalder, & Zelinsky, 2014). These studies find that the preview benefit depends on the relationship between the cue and the target. It reduces when the cue and target differ in size or orientation (Vickery et al., 2005) and more generally as they become more dissimilar (Schmidt & Zelinsky, 2009; Lamy, Zivony, & Yashar, 2011). Previewing the category to which the target belongs also produces systematic effects (Maxfield & Zelinsky, 2012). Previewing the target as a word has a weaker benefit that arises only when presented well in advance, suggesting that lexical representations become available only later in time (Wolfe et al., 2004; Vickery et al., 2005; Schmidt & Zelinsky, 2011). 

Despite these insights, we understand relatively little about how target preview influences search in terms of the mechanisms. Previewing a target is generally thought to bias attention to its features throughout the visual field, facilitating its detection (Ipata et al., 2006; Khayat, 2006; Suzuki & Gottlieb, 2013; Wilschut, Theeuwes, & Olivers, 2013, 2014; Sridharan, Schwarz, & Knudsen, 2014). But precisely how attentional bias leads to reduced search time is not clear. A potential clue to the underlying mechanism comes from observations made in a few studies, all showing that the preview benefit is small for easy searches and large for hard searches. The preview benefit is stronger not only when the target is more similar to the distractors (Chen & Zelinsky, 2006), but also with increasing set size (Vickery et al., 2005; Yang & Zelinsky, 2009). A similar observation has been made for repetition priming (Lamy et al., 2011). 

Why should target preview have a larger benefit for harder searches? There are two possibilities. The first possibility is that easy searches are dominated by bottom-up factors and therefore do not experience a preview benefit whereas hard searches are benefited more by top-down factors, such as repetition (Lamy et al., 2011). Although this is plausible, this account does not make explicitly testable predictions. This issue is complicated by the fact that there is no consensus in the literature about how the preview benefit should be measured: Should it be the absolute change in search time, percentage change in search time, or something else? 

Our solution to this issue is to develop a measure that is a natural index of the underlying biophysical processes that drive visual search. This is based on an accumulator model for search that has broad empirical support from neurophysiology (Schall et al., 2011). According to this model, a local salience signal generated at each location accumulates toward a threshold and triggers a response on reaching it. When a target is previewed, we hypothesize that this biases feature processing throughout the visual field, resulting in an increase in distinctiveness of the target relative to the distractor. Specifically, we propose that this bias is an additive increase in the signal, but this benefit accumulates longer for harder searches, resulting in a bigger reduction in search time. This possibility predicts the same preview benefit for all searches, provided they are characterized in terms of the underlying distinctiveness signal. 

Testing this possibility requires an estimate of the salience signal, which we obtain as follows: Consider a constant signal S that accumulates toward a threshold set (without loss of generality) to one. The accumulated signal at the time of threshold crossing must satisfy S × RT = 1, where RT is the threshold crossing time or Reaction Time. Therefore, the underlying signal S = 1/RT. Thus, 1/RT is an estimate of the underlying accumulating signal. This remains largely true even in the presence of motor delays and noise in the accumulator (Experiment 9). We therefore predicted that, when preview conditions are compared using 1/RT, we would find a fixed benefit due to preview in terms of 1/RT. 

The reciprocal of search reaction time (1/RT) not only has a plausible physical interpretation, but in a series of studies, we have found that it explains a variety of visual search phenomena compared to models based on search reaction time (RT). 1/RT increases linearly with feature differences (Arun, 2012; Pramod & Arun, 2014) and behaves like a mathematical distance (Arun, 2012). It explains complex searches involving multiple types of distracters (Vighneshvel & Arun, 2013), searches involving targets differing in multiple types of features (Pramod & Arun, 2014), and dissimilarities between objects differing in local and global attributes (Pramod & Arun, 2016). In this study too, we have measured the preview benefit in terms of RT as well as reciprocal RT (1/RT) and found a qualitatively different insight using reciprocal search time. 

Overview of this study

Our goal was to systematically investigate the impact of target preview on visual search across a large variety of searches. We conducted eight experiments in all. In each experiment, subjects searched for an oddball target after previewing an informative cue (the target or distractor of the upcoming search) or an unrelated white square. Across all experiments, we obtained essentially the same result: Although harder searches showed larger decreases in search time when the target was previewed, the same data replotted in terms of 1/RT showed a fixed offset across all searches. This is a nontrivial result because it cannot be explained by a simple transformation of search times. For instance, if RTu and RTi denote the search time in the unrelated and informative priming conditions, then halving all the unrelated preview search times (RTi = RTu/2) would cause a doubling of the reciprocal search times (1/RTi = 2/RTu) but not a fixed offset. Across all experiments, we also quantitatively evaluated these two models by comparing their residual error and found that the 1/RT offset model has a consistently smaller error compared to the RT scaling model. In Experiment 9, we show through simulations that these results can be explained by an additive but not multiplicative increase in distinctiveness even after taking into account fixed effects such as motor response latency. Taken together, our results show that previewing a target or a distractor additively increases the underlying dissimilarity signal in visual search. 

Methods

Our experimental procedures were nearly identical across all experiments and are collectively described below. 

Subjects

All subjects were 20–30 years old with normal or corrected-to-normal vision. Subjects were naïve to the purpose of the experiment and gave written consent to a protocol approved by the Institutional Human Ethics Committee of the Indian Institute of Science. 

Stimuli

We chose stimuli consisting of similar and dissimilar shapes to ensure a broad range of search difficulty. All images were normalized to have the same brightness and contrast to avoid low-level cues from guiding search. 

Procedure

Subjects were seated approximately 60 cm from a computer monitor, which was under control of custom Matlab programs in Psychtoolbox (Brainard, 1997). In each trial, subjects saw a 3° × 3° noise mask that appeared for 0.5 s at the center of a screen followed by the cue (Figure 1). The cue was either the oddball target of the subsequent search display or an unrelated white square (3° wide) that appeared at fixation for 1 s. This was followed by a search array with 32 items arranged in a 6 × 6 grid with 4.5° spacing between the item centers. All items were identical except for one oddball target that appeared within the central 4 × 4 grid. The position of each item in the grid was randomly jittered according to a uniform distribution with range ±0.675° to prevent alignment cues. The task of the subject was to indicate using a key press (“M” for right, “Z” for left) the side of the screen on which the oddball target was located. A red vertical line (width 0.11°) was displayed along the middle of the screen to facilitate left/right judgments. The search array was displayed for 10 s or until the subject made a response, whichever was sooner. Timed-out trials and error trials were repeated randomly later on during the task. 

Figure 1

 

Schematic of the visual search task used in all experiments. Each trial began with a noise mask followed by either an unrelated preview (a white square) or an informative preview (target or distracter) for 1 s. This was followed by a search array containing one oddball. Subjects had to indicate, using a key press, whether the target was on the left or right of the screen. In the actual experiment, stimuli were shown on a black background in a 6 × 6 search array.

In most experiments, subjects performed searches involving all possible pairs of 16 images (i.e., ¹⁶C₂ = 120 pairs). For each pair of images (A, B), subjects performed eight correct trials (four with A as target, which comprised two trials with A on the left and two with A on the right, and four with B as target). Thus, for each priming condition (target preview and unrelated preview) there were 120 × 8 = 960 trials, resulting in a total of 1,920 correct trials. These trials appeared in random order. 

The above details were common to most experiments. The different details in each experiment are described below. 

Experiment 1

A total of nine subjects (five male) participated in the experiment. 

Experiment 2

A total of eight subjects (five male) participated in the experiment. We chose a random set of 80 pairs from the 120 pairs and ran them each for 16 correct trials (eight with A as target, among which four trials had A on the left and four with A on the right, and eight with B as target) giving rise to a total of 2,560 trials. All the other details remained the same. 

Experiment 3

A total of eight subjects (five male) participated in the experiment. Stimuli measured 2° × 2°. All the other details remained the same. 

Experiment 4

A total of eight subjects (four male) participated in the experiment. 

Experiment 5

A total of eight subjects (six male) participated in the experiment. 

Experiment 6: Present/absent searches

A total of eight subjects (four male) participated in the experiment. A total of 60 searches were randomly chosen from the 120 searches of Experiment 1. Targets were present in half the trials and absent in the other half. Similarly, the target was previewed in half the trials and a white square was shown in the other half of the trials. This gave rise to a total of 60 × 4 × 8 = 1,920 trials. The task in this experiment was to report the presence or absence of the oddball in the search by pressing either the “P” or “A” key on the keyboard. Searches with an accuracy less than 65% were removed from further analysis. All the other details remained the same. 

Experiment 7: Homogeneous search with varying set size

A total of eight subjects (seven male) participated in the experiment. A total of 120 searches were performed at two set sizes (14 and 32 items) with subjects performing four correct trials per unique search. This gave rise to a total of 120 searches × 2 set sizes × 2 preview conditions × 4 repetitions = 1,920 trials. All the other details were the same as in the common methods. 

Experiment 8: Heterogeneous search with varying set size

A total of eight subjects (three male) participated in the experiment. Here, searches were heterogeneous, i.e., search arrays contained a target among two types of distractors. In each trial, the search array was preceded by a 300-ms preview of either the exact target image or the word corresponding to the target. The subject had to respond using a key press to indicate whether the target was on the left or right side of the screen. We chose nine unique animals (bird, cat, cow, dog, goat, horse, monkey, snake, and rhino) for this experiment, which were used to create 36 target/distractor triplets. Each stimulus appeared as a target in four searches such that it occurred only once with each of the four remaining pairs of stimuli. The four distractor pairings for the searches for each stimulus were chosen randomly, without replacement, among the remaining eight stimuli. Search arrays contained either 14 or 32 items in a 6 × 6 grid where, in searches with 14 items, stimuli appeared in the middle 4 × 4. Subjects had to perform eight correct trials of each search. Thus, in all there were a total of 36 searches × 2 set sizes × 2 preview conditions × 8 repetitions = 1,152 trials. Searches with an accuracy less than 80% were removed from further analysis. All the other details were the same as the common methods. 

Data analysis

During post hoc analyses, search times greater than 5 s were excluded from further analysis. This was done for all experiments except for Experiment 8, which contained harder searches and set size variations. This removed no more than 0.6% of all trials. We obtained similar results upon changing these thresholds. To estimate the dissimilarity signal for a particular search, we calculated the average search time across trials (with either item in a pair as target) and subjects and then took the reciprocal of the average search time. Confidence intervals for slopes and intercepts were obtained using built-in linear regression functions in Matlab (regress). 

Results

We performed a total of eight visual search experiments with a very similar design (Figure 1), which yielded extremely similar results. Therefore, for ease of exposition, we have described all experiments together grouped into sections and tabulated the experiment-specific details separately. 

Across all experiments, subjects were extremely consistent in their search times across searches, highly accurate, and showed no speed–accuracy trade-off in their performance (Table 1). Average search times across searches with preview were strongly correlated with search times with unrelated preview, suggesting that preview modulates but does not fundamentally alter intrinsic search difficulty (Table 1). 

Table 1

 

Consistency, accuracy, and speed–accuracy trade-offs. Notes: ^#correlations that did not reach statistical significance (p > 0.05). All other correlations were significant (p < 0.05). P = Informative preview; NP = no (uninformative) preview.

 

Experiment 1: Target preview in oddball search

In Experiment 1, we took searches involving all possible pairs of a set of 16 animals (Figure 2A). On plotting the average search time in the preview versus the unrelated preview condition, we obtained a strong correlation (Figure 2A; Table 1). To assess whether a particular image pair (A, B) showed a priming benefit, we performed an ANOVA on the search time for that pair with subject, preview condition, and target (A or B as target) as factors. For these pairs, the relationship between the target preview and unrelated preview search times was not just a change in slope (i.e., slope significantly different from one) but also a clear nonzero intercept (Table 2). 

Figure 2

 

Target preview speeds up search (Experiment 1: animals). Top row: Animal stimuli used in this experiment. (A) Search times in the target preview plotted against search times in the unrelated preview conditions. Plus symbols represent object pairs for which the search time was significantly different (p < 0.05, see text) between the two preview conditions. Pairs with no significant difference (p > 0.05) are represented by gray circles. The equation of the best-fitting straight line for the object pairs benefited by preview is shown. (B) Same data as in panel A but replotted using the reciprocal of search times (1/RT), a measure of the distinctiveness of the target relative to the distracters. The equation of the best-fitting straight line for the object pairs benefited by preview is shown. The symbol Δ represents the average salience benefit due to preview, i.e., the distance from the unit line.

Table 2

 

Relationship between target versus unrelated preview for significantly primed pairs. Notes: The best-fitting slope and intercept are shown for the RT plot and the reciprocal RT plot. In each case, the numbers within square brackets represent the 95% confidence interval.

 

On replotting the same data in terms of the reciprocal RT, we saw a strikingly different pattern that suggests a qualitatively different explanation: Searches that were significantly benefited by preview had a reciprocal search time that differed by a fixed constant from the unrelated preview condition (Figure 2B). In other words, the slope of this relationship was not significantly different from one and the intercept was nonzero (Table 2). This effect remained qualitatively similar even when including all searches rather than the ones with a significant benefit (Table 3). 

Table 3

 

Relationship between target versus unrelated preview across all pairs. Notes: Conventions remain the same as in Table 2.

 

To verify that the 1/RT preview benefit does not depend on search difficulty, we sorted the searches into two groups according to the average search time in the unrelated priming condition and asked whether the increase in reciprocal RT was different between the easy and hard search groups. This revealed no significant difference (Table 4). Thus, the reciprocal of search time increased by a constant offset due to target preview. To investigate whether the priming benefit interacts with search asymmetry, we identified for each image pair (A, B) the easy target (e.g., A in B) and hard target (e.g., B in A) based on the average search times in the unrelated priming condition. We then compared the priming benefit (i.e., difference in 1/RT) for the easy and hard targets. We found a greater priming benefit for easy targets compared to hard targets, but in both cases, we obtained a constant offset in 1/RT (Table 4). Thus, the preview benefit shows the same effect for both easy and hard searches. 

Table 4

 

Priming benefit as a function of search difficulty and search asymmetry. Notes: Here we have compared the priming benefit (difference in 1/RT between target preview and unrelated preview) by separating all significant searches into two equal groups (low RT and high RT). The column denoted by p represents the statistical significance of a paired comparison (paired t test) performed between the easy and hard searches. Likewise, to investigate whether the priming benefit is influenced by search asymmetry, we divided the searches for each significantly primed image pair (A, B) into those corresponding to the easy target or the hard target in the unrelated priming condition. The priming benefit for the easy and hard targets is shown with column p denoting the statistical significance of the comparison (paired t test). Entries corresponding to easy and hard targets in the words experiment are marked N/A because search asymmetries are no longer meaningful for heterogeneous searches with one target and two distractors.

 

Experiment 2: Are some image pairs benefited more by preview than others?

The above analyses were based on image pairs that showed a significant priming benefit. Why do some pairs show a systematic preview benefit but not others? There are two possible explanations: The simplest one is that all searches are benefited by preview but only a few are detected due to random noise. Alternatively, only some searches may be consistently benefited, which can happen, for instance, if some targets are easier to hold in working memory. To investigate this issue, we performed Experiment 2, in which we selected a subset of 80 of the 120 searches from Experiment 1 with twice the number of trials per condition. This allowed us to ask the question: Are the same searches benefited by preview more frequently than expected by chance? 

For ease of exposition, we describe the result of dividing the trials into odd- and even-numbered trials, but we obtained similar results for other random splits of the data. For the odd-numbered trials, we found that 19 of the 80 pairs (i.e., 24%) showed a significant benefit in priming. Among the even-numbered trials, 23 of the 80 pairs (29%) showed a significant benefit. Only five searches were benefited in both groups. We then asked whether the distribution of searches that did or did not overlap in each group was significantly different from the number of searches expected had the incidence in each group been independently distributed. This revealed no statistically significant difference, suggesting that the incidence of benefit was independently distributed in the two groups (observed distribution of effects: 5, 18, 14, and 43 searches that were benefited in both groups, in the odd group only, in the even group only, and in neither group, respectively; predicted distribution given the 24% and 29% rate of occurrence in odd and even groups: 5.5, 17.5, 13.5 and 43.5; p = 0.98, chi-squared test). Thus, there was no significant tendency for some searches to be consistently benefited by preview across independent sets of trials. In other words, the target preview benefit is present in some pairs but not others due to random noise and not because some pairs show a consistent preview benefit compared to others. In any case, the constant offset in 1/RT was present in both searches with significant preview benefit and across all searches (Tables 1 and 2). 

Experiments 3 and 4: Generality of the fixed priming benefit

To establish the generality of the effect, we performed several additional experiments, which yielded the same results as detailed above. In Experiment 3, we selected English letters as another disparate set of shapes to test the generality of the effect (Figure 3). In Experiment 4, we tested the same animals as in Experiment 1 but inverted them to reduce their semantic strength (Figure 4). In all these experiments, priming produced a fixed constant offset in 1/RT and yielded results extremely similar to those detailed above (Tables 1 through 4). 

Figure 3

 

Experiment 2 (letters). (A) Search times in the target preview plotted against search times in the unrelated preview conditions. All conventions are as in Figure 2. (B) Same data as in panel A but replotted using reciprocal search time (1/RT).

Figure 4

 

Experiment 3 (inverted animals). (A) Search times for target and unrelated preview. All conventions are as in Figure 2. (B) Same data as in panel A but replotted using reciprocal search time (1/RT).

Experiment 5: Distractor preview

Next, we asked whether previewing the distractor of an upcoming search can also lead to a priming benefit, and if so, whether it will also yield a fixed benefit across all searches. To this end, we repeated Experiment 1 with a preview of the distractor instead of the target. We obtained qualitatively similar results, i.e., a fixed preview benefit in 1/RT (Figure 5; Tables 1 through 4). Importantly, the preview benefit did not depend on the statistical criterion used to detect significantly primed pairs (Table 3) and did not depend on search difficulty or on search asymmetry (Table 4). 

Figure 5

 

Effect of distractor preview on visual search (Experiment 4). (A) Search times for distractor and unrelated preview. All conventions are as in Figure 2. (B) Same data as in panel A but replotted using reciprocal search time (1/RT).

To compare the magnitude of the preview benefit for target versus distractor preview, we compared the increase in 1/RT across all 120 searches for distractor preview with the same searches with target preview in Experiment 1. The benefit was significantly larger for target compared to distractor preview (average benefit across 120 searches: 0.05 in this experiment; 0.13 in Experiment 1; p < 0.00005, paired t test). Although this could, in principle, reflect differences between subjects, it is unlikely because search times in the two experiments in the unrelated preview conditions were strongly correlated (r = 0.94, p < 0.00005 across 120 searches). 

We conclude that distractor preview also produces a fixed increase in distinctiveness just like target preview. 

Experiment 6: Present/absent searches

In all preceding experiments, subjects had to indicate the location of a target that is always present. We therefore wondered if our results would generalize to the more common form of visual search used in the literature in which a target is either present or absent. To this end we performed Experiment 6 with the target present or absent with equal probability in each trial. We obtained extremely similar results: There was a fixed preview benefit in 1/RT across all searches (Figure 6; Tables 1 through 4). Importantly, the preview benefit did not depend on the statistical criterion used to detect significantly primed pairs (Table 3) and did not depend on search difficulty or on search asymmetry (Table 4). 

Figure 6

 

Effect of target preview in a present/absent task. (A) Search times for target and unrelated preview. All conventions are as in Figure 2. (B) Same data as in panel A but replotted using reciprocal search time (1/RT).

Although not the main focus of this study, we obtained an incidental insight into target-absent searches using this data. We note that it is meaningless to look for a preview benefit in a target-absent search because there is no target in the first place. However, there could still be systematic effects of target preview: We reasoned that, when a “target” is previewed, it might take longer to confirm its absence in an upcoming search array if it is similar to the distractors than when it is dissimilar. Specifically, when item A is previewed and then a search array containing Bs is shown, we predict that the response time will be systematically related to the search time for finding the target A among an array of Bs in the unrelated preview condition. Indeed, these two search times had a significant positive correlation (r = 0.72, p < 0.00005). We conclude that confirming the absence of a previewed target in an array of distractors is driven by the dissimilarity between the previewed target and the distractors. 

Experiment 7: Set size

We performed this experiment to investigate how the preview benefit changes with set size. To this end, subjects searched for oddball targets in either 14- or 32-item arrays in both target and unrelated preview conditions. We obtained qualitatively similar results at both set sizes: Target preview led to a fixed benefit that appeared to be independent of set size (Figure 7; Tables 1 through 4). Again, the preview benefit did not depend on the statistical criterion used to detect significantly primed pairs (Table 3) and did not depend on search difficulty or on search asymmetry (Table 4). 

Figure 7

 

Effect of set size on target preview. (A) Search times in the target preview plotted against search times in the unrelated preview conditions for 14-item searches (red) and 32-item searches (blue). Plus symbols represent object pairs that showed a main effect of preview condition (p < 0.05, see text). Circles represent pairs with no significant difference (p > 0.05). Squares depict averaged data for search pairs corresponding to quartiles of the search time in the unrelated priming condition (set size 14). Red and blue lines indicate best-fitting straight lines to the significantly primed pairs. (B) Same data as in panel A but replotted using reciprocal search time (1/RT). All conventions are as in panel A. The symbol Δ represents the average salience benefit due to preview, i.e., the distance from the unit line. (C) Reaction time versus set size plot for the significant searches in hard (solid lines) and easy (dashed lines) groups divided according to the sorted RTs across 120 searches in the unrelated preview condition. There were 18 hard and 15 easy significant searches.

To visualize the effect of set size on searches across varying levels of difficulty, we separated the search times in the unrelated priming condition at set size 14 into four equal groups sorted by difficulty. For each group, we calculated the mean search time and the mean reciprocal search time in both the target preview and unrelated preview conditions at each set size. The resulting points are shown in Figure 7A and B. It can be seen that, for each level of difficulty, a change in the set size increases the search time in both the exact and unrelated preview conditions, but the net benefit due to priming remains unchanged (Figure 7B). 

To visualize the priming benefit in the more conventional RT versus set size plot, we divided the image pairs that were benefited significantly by preview into two groups (based on the average search times in the unrelated preview condition with set size 14): easy searches (n = 18) and hard searches (n = 15). We can see that the hard searches are benefited more by target preview, but again, the same data plotted using 1/RT show a fixed preview benefit (Figure 7A, B). 

In sum, we conclude that the target preview leads to a fixed benefit that is independent of set size. 

Experiment 8: Word versus picture preview

The preceding experiments all involved displays containing one target and multiple identical distractors with the target itself directly as the relevant preview. Here we asked whether our results regarding the preview benefit would generalize in two ways: (a) preview benefit for a target presented as a word versus an exact picture and (b) preview benefit for heterogeneous searches in which there is one target among two types of distracters (e.g., a bird among dogs and rhinos). 

In Experiment 8, subjects searched for a target image that was always cued using a word or the exact image. The search display consisted of the target image embedded among two types of distractors. We then repeated all the analyses as before and compared the picture preview condition with the word preview condition (Tables 1 through 4). Based on previous literature, we expected that picture preview would lead to a stronger benefit compared to word preview. Indeed this is what we observed: Search times for picture preview were faster in general (Figure 8A), but when the same data was plotted using the reciprocal of search time, we observed a fixed preview benefit (Figure 8B; Table 2). Importantly, the preview benefit did not depend on the statistical criterion used to detect significantly primed pairs (Table 3) and did not depend on search difficulty (Table 4). The same data plotted using the conventional RT versus set size plots revealed that the preview benefit in terms of the absolute change in RT was larger for harder searches (Figure 8C). It can also be seen that previewing the exact target image has a stronger effect compared to previewing its word. 

Figure 8

 

Target preview in heterogeneous search (Experiment 8). (A) Search times in the target preview plotted against search times in the word preview conditions for 14-item searches (red) and 32-item searches (blue). Plus symbols represent object pairs that showed a main effect of preview condition (p < 0.05, see text). Circles represent pairs with no significant difference (p > 0.05). Red and blue lines indicate best-fitting straight lines to the significantly primed pairs. (B) Same data as in panel A but replotted using reciprocal search time (1/RT). All conventions are as in panel A. (C) RT versus set size plot for the 18 hard (solid lines) and 18 easy (dashed lines) searches divided according to the sorted RTs across 36 searches in the word preview condition.

In sum, we conclude that target preview leads to a fixed benefit in distinctiveness across all searches regardless of difficulty even for heterogeneous searches with verbal or pictorial preview. 

Is the preview benefit a scaling of RT or an offset in 1/RT?

The essential finding of this study is that previewing a target or distractor leads to a fixed increase in 1/RT across both easy and hard searches. We have argued that this cannot be explained using a simple scaling of RT because this will result in a simple scaling of 1/RT as well. However, because these two accounts of the data are quantitatively different, we set out to examine these two possibilities by comparing how well they can explain the data. 

Specifically, we considered two models. Let the search time in the target and unrelated preview conditions be RTp and RTnp. The first model, which we call the RT scaling model, specifies that RTp = a. RTnp, where a is a scaling factor. The second model, which we call the 1/RT offset model, specifies that 1/RTp = 1/RTnp + c, where c is a constant offset. Note that these two models have exactly one free parameter, and therefore, their residual errors can be directly compared. We then fit the two models to the data in Experiment 1 and plotted the predictions of both models on a plot of RTp versus RTnp as before (Figure 9A). Across all searches with a preview benefit, the absolute residual error in search time for the 1/RT offset model was significantly smaller than for the RT scaling model (Figure 9A, inset). To perform a global comparison across all experiments, we repeated this procedure for each experiment and combined the absolute residual error for searches with preview benefit across all experiments; here too the 1/RT offset model had a smaller error (Figure 9B). To further confirm this effect, we replotted the data again using 1/RT (Figure 9C). Here too we observed a similar pattern: The absolute error in reciprocal search time was smaller for the 1/RT offset model compared to the RT scaling model for Experiment 1 (Figure 9C), and this pattern was true even across the data for Experiments 1 through 8 put together (Figure 9D). 

Figure 9

 

Comparing RT scaling with 1/RT offset. (A) Search RTs for the target preview condition plotted against those in the unrelated preview conditions (black dots) with the best-fitting predictions of the RT slope model (blue) and the predictions of the 1/RT offset model (red) for searches that are significantly benefited by preview (n = 46). Inset: Mean absolute error for the RT slope model (blue) and 1/RT offset models (red). (B) Mean absolute error for the RT slope model (blue) and the 1/RT offset model (red) across all searches with a preview benefit across Experiments 1 through 8. This was calculated by combining the mean absolute error across experiments for each model. (C) Similar plot as in panel A but using 1/RT. (D) Similar plot as in panel B but using 1/RT, showing that the 1/RT offset model has a smaller error regardless of whether the error is measured using RT or 1/RT.

Thus, the relationship between search times in the target and unrelated preview conditions is better explained as a fixed increase in 1/RT than a scaling of RT. We conclude that preview leads to a fixed benefit in 1/RT or distinctiveness across all searches. 

Experiment 9: Simulations

In all our results, we have interpreted the reciprocal of search time as an estimate of the underlying distinctiveness signal in visual search and its fixed increase due to preview as arising from a fixed increase in the underlying distinctiveness. Here, we critically evaluated these two interpretations using computational modeling. First, we asked whether a fixed 1/RT increase could be explained by an additive or a multiplicative change in the underlying accumulating signal. This issue is not straightforward if there is a motor delay between target selection and the motor response. Second, we asked whether these two interpretations change in a more realistic process of stochastic accumulation toward threshold. 

Can the fixed 1/RT increase be explained by an additive or multiplicative gain?

The fixed 1/RT increase due to target preview implies that the underlying distinctiveness or salience signal undergoes a systematic transformation. We considered two simple possibilities: The fixed increase in 1/RT could arise from an additive or a multiplicative gain applied to the intrinsic bottom-up salience signal in the search array. These two possibilities are illustrated in Figure 10. 

Figure 10

 

Models for the fixed preview benefit (Experiment 9). (A) In the additive gain model, a bottom-up salience signal accumulates to threshold, and a response is generated after a short motor delay (denoted by B). In a target or distractor preview condition, the bottom-up salience signal is incremented by a fixed amount. (B) Plot of simulated RTs in the preview condition against the unrelated preview conditions for two levels of preview benefit and two motor baselines (see text). (C) Same data as in panel B but replotted using search salience, i.e., reciprocal of search time. (D) In the multiplicative gain model, all details are the same as in the additive model except that the top-down bias due to preview multiplicatively increases bottom-up salience. (E–F) Same plots as in panels B and C for the multiplicative gain model.

In the additive gain model (Figure 10A), the bottom-up salience signal arising from the search array itself is increased by a fixed amount by the top-down signal from target preview. The search RT is taken as a sum of the time required for the net salience signal to integrate to threshold and a fixed motor baseline delay due to the time required to make a key-press response after detecting the target. Thus, if the bottom-up salience is B and the motor delay is M, the reaction time in the no-preview condition is RTnp = M + 1/B. The RT in the target preview condition is then RTp = M + 1/(B + T), where T represents the top-down bias due to target preview. For a fixed level of motor baseline, we adjusted the top-down bias T to obtain a slope of either 0.5 or 0.75 between RTp versus RTnp. A slope of 0.75 is close to the relationship observed in Experiment 1 between the search times for target preview versus unrelated preview (Figure 2A). Next, as we did with the real data, we plotted the RT in the preview condition against the RT in the no-preview condition for a broad range of searches (i.e., for many values of the bottom-up salience B) for different levels of motor baseline (M = 0 or 0.2 s) and different levels of top-down bias (slope = 0.5 or 0.75). The resulting plot (Figure 10B) shows that increasing the motor baseline tends to reduce the effect of preview, particularly for easy searches in which the target selection time is small. This effect is more pronounced when the same data is replotted using 1/RT (Figure 10C). For zero baseline, as expected, the 1/RT for target preview differs by a constant offset from the 1/RT for unrelated preview. This offset increases for a larger preview effect. However, with a fixed motor baseline, the 1/RT for target preview tends to approach the 1/RT for unrelated preview. In other words, when 1/RT is large (i.e., for easy searches), the preview benefit should be small because the motor baseline dominates the RT. Thus, additive gain can explain the fixed increase in 1/RT only if the motor delay is either zero or relatively small compared to the target selection time. 

In the multiplicative gain model (Figure 10D), the bottom-up salience signal arising from the search array undergoes a proportional gain change due to the top-down signal from target preview. The search RT is taken again as a sum of the target selection time (from the accumulator model) plus a fixed motor delay. Following the same conventions as before, RTnp = M + 1/B and RTp = M + 1/(BT). We then plotted the relationship between RTp and RTnp (Figure 10E) and between 1/RTp and 1/RTnp (Figure 10F) as before for two levels of preview benefit (slope = 0.5 or 0.75) and motor baseline (B = 0 or 0.2 s). With no motor delay, the search times in the target preview and unrelated preview conditions are directly proportional (Figure 10E), and the 1/RT plot reveals no fixed increase due to target preview (Figure 10F, solid lines). Instead, the benefit of preview is larger for easy searches. This effect is mitigated by the presence of a motor delay (Figure 10F, dashed lines), which dominates the net RT for easy searches but still does not produce a fixed increase in the 1/RT plot. Although it is possible that this nonlinearity may not be observed due to noise in the data, achieving a fixed increase in 1/RT will require a different motor delay for each level of gain, which is not plausible. We conclude that a multiplicative gain change in the bottom-up salience signal cannot produce a constant offset in 1/RT with or without a motor baseline. 

To summarize, we have found that a fixed increase in 1/RT can be observed only with an additive gain change in the underlying salience signal but not a multiplicative gain change. Thus, top-down factors, such as knowledge of the target, add to bottom-up salience rather than multiplying it. 

Does 1/RT estimate the underlying salience signal for a stochastic accumulator?

As detailed in the Introduction, for a noiseless accumulation to threshold, 1/RT is directly proportional to the underlying salience signal. However, this may not be true in the presence of noise. To assess this possibility, we simulated a stochastic accumulation process toward threshold (Figure 11A). The level of the accumulator was incremented in each time step (5 ms each) by a fixed drift signal (i.e., the bottom-up salience signal) and by zero-mean Gaussian noise (σ = 0.033). A response was recorded when the accumulator level reached a value of one. The standard deviation of the Gaussian noise was chosen to match the linear relationship between mean and standard deviation observed in the real data in Experiment 1 (σ_RT = 0.75μ_RT − 0.24, where σ_RT and μ_RT are the standard deviation and mean of the observed RTs across subjects and trials). For the unrelated preview condition, we chose a total of 120 values of bottom-up saliences (ranging from 0.0025 to 0.01 in equal steps) and created 72 simulated search times for each search pair (corresponding to nine subjects with eight trials each as in Experiment 1). For the target preview condition, we incremented all bottom-up salience values in the unrelated preview condition by a fixed value of 0.00045 (this value was chosen to match the observed slope of 0.79 between target and unrelated priming RT in Experiment 1; see Figure 2A). 

Figure 11

 

Additive gain model with stochastic accumulators (Experiment 9). (A) In the stochastic accumulator model, a noisy bottom-up salience signal accumulates until it reaches threshold at which point a response is generated. (B) Plot of 1/RT against the magnitude of the underlying salience signal, showing that 1/RT is an excellent estimate of the salience signal. (C) Plot of simulated search times in the preview condition against search times in the unrelated preview condition. (D) Same data as in panel C but using search salience, i.e., reciprocal of search time.

We first asked whether the reciprocal of the simulated RT was indeed a reasonable estimate of the underlying salience signal. A plot of these two quantities revealed a striking correlation (r = 0.99, p < 0.00005; Figure 11B). We conclude that 1/RT provides an unbiased estimate of the underlying salience signal even for a stochastic accumulator at least within the range relevant to the observed data. 

To investigate whether the additive increase in bottom-up salience results in a fixed increase in 1/RT, we plotted the data for the two preview conditions as before. We found a strong correlation between the search times in the two preview conditions (r = 0.98, p < 0.00005; Figure 11C). But importantly, when the same data was replotted using 1/RT, we observed a clear constant offset (best-fitting slope = 0.98 with [0.94, 1.01] as the 95% confidence interval; offset = 0.11 with [0.07, 0.16] as the 95% confidence interval; Figure 11D). We conclude that a fixed increase in 1/RT due to target preview can be observed due to an additive gain in bottom-up salience even for a stochastic accumulator process. 

General discussion

In this study, we have systematically investigated how previewing a target or a distractor benefits visual search across a wide variety of search conditions. Our main finding is that preview produces larger reductions in search time for harder searches, but when the same data is replotted in terms of reciprocal search time (1/RT), this reveals a fixed benefit in 1/RT across all searches regardless of difficulty. This is a nontrivial finding because it cannot be explained using a simple scaling of the search time (scaling RT is equivalent to scaling 1/RT and thus cannot produce a constant offset in 1/RT). 

We have interpreted our findings in the framework of an accumulator model for visual search: We have shown that 1/RT is an unbiased estimate of the underlying salience signal and that a fixed increase in 1/RT can be explained by an additive (but not multiplicative) increase in the underlying salience signal. Our results imply that top-down knowledge in the form of target preview combines additively with bottom-up intrinsic factors. Below, we discuss our findings in the context of the literature. 

Our study addresses a major gap in the literature on how prior knowledge of the target influences search: Whether the preview benefit depends on search difficulty has never been systematically investigated in contrast to the extensively studied topic of cue–target relationships (Wolfe et al., 2004; Vickery et al., 2005; Schmidt & Zelinsky, 2009, 2011; Maxfield & Zelinsky, 2012; Maxfield et al., 2014). Only a few studies have observed that the strength of repetition priming depends on search difficulty (Lamy et al., 2011; Meeter & Olivers, 2014). How might search difficulty influence the preview benefit? One possibility is that easy searches are benefited less by preview because they are dominated by bottom-up factors or because the preview benefit might develop slowly in time (Lamy et al., 2011). 

This issue is complicated by the fact that there is no clarity in the literature about how to measure the preview benefit: Should it be the absolute reduction in search time or the percentage reduction? If the preview benefit is taken as the absolute reduction in RT, we have found it to be smaller for easy searches (e.g., panel A of Figures 2 through 8) as have others (Vickery et al., 2005; Chen & Zelinsky, 2006; Yang & Zelinsky, 2009; Lamy et al., 2011). But if the preview benefit is taken as the percentage reduction in search time, then we have found it to be the same for all searches. At this juncture, there is no clear reason to favor one measure over the other. However, neither measure offers any intuition about the underlying mechanisms. They also do not explain other aspects of the preview benefit: For instance, we have consistently observed a nonzero intercept in the relationship between search times in the two conditions (panel A of Figures 2 through 8; Tables 2 and 3). 

One solution to this quandary is to develop a measure for target preview that is a natural index of the processes that underlie target detection in visual search. Here, we have shown that the appropriate measure for preview benefit is the reciprocal of search time. This measure is based on a model for visual search with broad empirical support from neurophysiology (Schall et al., 2011). According to this account, viewing a search array sets off accumulators that integrate local salience toward a threshold. A response is made when the net salience or evidence at a particular location reaches threshold. The fixed preview benefit means that previewing the target increases the net salience of the target relative to the distractor by biasing feature representations throughout the visual field (Carrasco, 2011). This increased salience accumulates faster toward threshold, but the accumulation occurs for a short time in an easy search and for longer in a hard search, producing a bigger absolute reduction in search time for hard searches. This interpretation explains the data more parsimoniously and is grounded in a physiologically plausible account of visual search. This interpretation is also consistent with a model of guided search in which the feature guidance signal, which drives a diffusion process, is benefited additively by preview (Wolfe, 2006). Our finding that bottom-up salience is additively facilitated by top-down preview predicts that feature attention should act additively on target–distractor salience throughout the visual field although this prediction is consistent with additive gain changes due to attention the precise link between attentional facilitation, and its effect on salience signals in search need to be systematically investigated (Carrasco, 2011). 

We have also found that previewing the distractor produces a fixed but weaker increase in distinctiveness compared with previewing the target (Experiment 5). This finding is consistent with the weaker repetition priming observed for distractors (Watson & Humphreys, 1997; Wolfe et al., 2003). These findings imply that subjects can also suppress distractor features but not as strongly as they can enhance target features although this effect may become stronger with repeated distracters (Cunningham & Egeth, 2016). More generally, our results show that, although target enhancement and distractor suppression may be served by distinct mechanisms (Khayat, 2006; Suzuki & Gottlieb, 2013; Sridharan et al., 2014), both have the same effect on the underlying salience signal. 

Conclusions

In sum, we have shown that previewing the target or distractor of an upcoming search produces a fixed increase in distinctiveness or salience. In other words, the bottom-up salience of the search is additively modulated by top-down knowledge. The notion that these factors combine linearly is concordant with our recent findings that bottom-up salience signals related to target–distractor dissimilarity and distractor heterogeneity combine linearly (Vighneshvel & Arun, 2013) and that salience signals for object attributes combine linearly (Pramod & Arun, 2014, 2016). Taken together, these results indicate a general principle whereby a variety of salience signals combine linearly to guide visual search. 

Acknowledgments

This research was funded by an Intermediate Fellowship from the Wellcome Trust - DBT India Alliance and by funds from the Department of Biotechnology, Government of India. 

Commercial relationships: none. 

Corresponding author: S. P. Arun.