Adaptation improves performance on a visual search task

Abstract

Abstract

Abstract:

Abstract  Temporal context, or adaptation, profoundly affects visual perception. Despite the strength and prevalence of adaptation effects, their functional role in visual processing remains unclear. The effects of spatial context and their functional role are better understood: these effects highlight features that differ from their surroundings and determine stimulus salience. Similarities in the perceptual and physiological effects of spatial and temporal context raise the possibility that they serve similar functions. We therefore tested the possibility that adaptation can enhance stimulus salience. We measured the effects of prolonged (40 s) adaptation to a counterphase grating on performance in a search task in which targets were defined by an orientation offset relative to a background of distracters. We found that, for targets with small orientation offsets, adaptation reduced reaction times and decreased the number of saccades made to find targets. Our results provide evidence that adaptation may function to highlight features that differ from the temporal context in which they are embedded.

Introduction

The percept evoked by a sensory stimulus depends strongly on its spatial context. For instance, the perceived color, brightness, contrast, and orientation of a visual pattern can be altered by the co-presentation of surrounding stimuli (Fiorentini, 2004; Knoblauch & Shevell, 2004; Schwartz, Hsu, & Dayan, 2007). Such spatial contextual effects are thought to contribute to figure-ground segregation and to establishing stimulus salience (Bergen & Adelson, 1988; Series, Lorenceau, & Fregnac, 2003). Indeed, computational models of salience (Wolfe, 1994; Li, 1999; Itti & Koch, 2000) rely on detectors with suppressive spatial surrounds, which provide weaker responses to a preferred stimulus when it is surrounded by similar features. Such suppressive surrounds are a well-established property of neurons in visual cortex (Angelucci & Bressloff, 2006). In these models, a target stimulus is salient relative to a field of uniform distracters, because responses to distracters are reduced by such suppressive surrounds. 

Temporal context also alters the appearance of a stimulus. The prolonged presentation of a stimulus, or adaptation, can cause a repulsive bias in the perception of subsequently viewed stimuli. For instance, after viewing a vertical pattern, similarly oriented test patterns appear repelled away from vertical (the tilt aftereffect; Gibson & Radner, 1937; Clifford, 2002). Likewise, exposure to motion causes subsequent stimuli to appear to move in the opposite direction (motion aftereffect; Mather, Verstraten, & Anstis, 1998), and adaptation to textures can reduce their perceived density (Durgin & Proffitt, 1996). Perceptual aftereffects to complex stimulus attributes such as the attractiveness, gender, ethnicity, and emotion of faces have also been reported (Rhodes, Jeffery, Watson, Clifford, Nakayama, 2003; Webster 2011). 

Whereas there is a clear sense of how spatial contextual effects could contribute to vision, this is not the case for many temporal contextual effects. Studies testing for perceptual enhancement after adaptation have focused on the possibility of improved discriminability for stimuli similar to the adapter. However, the evidence for this has largely been negative (Kohn, 2007). For instance, detection thresholds are elevated after exposure to high contrast patterns (Blakemore & Campbell, 1969; Graham, 1989), and there is no consistent improvement in the discriminability of suprathreshold contrasts (Barlow, Macleod, & van Meeteren, 1976; Greenlee & Heitger, 1988; Maatanen & Koenderink, 1991; Abbonizio, Langley, & Clifford, 2002). The best established effect of adaptation on orientation and motion discrimination is reduced performance for test stimuli offset from the adapter (Regan & Beverley, 1985; Hol & Treue, 2001; see Clifford, 2002 for review). 

Schwartz et al. (2007) pointed out many similarities in the phenomenology of spatial and temporal contextual modulation, and recent neurophysiological work has indicated a strong interaction between these contextual influences on the response of neurons in the early visual system (Tailby, Solomon, Dhruv, & Lennie, 2008; Camp, Tailby, & Solomon, 2009; Wissig & Kohn, 2012). These findings raise the possibility that spatial and temporal contextual effects may serve a common purpose. Like spatial contextual effects, adaptation can alter the relative sensitivity of detectors, typically reducing most strongly the sensitivity of detectors similar to the adapter. Therefore, it may also influence salience, emphasizing stimuli that differ from their temporal context. Recent work showing that color adaptation can modulate salience (McDermott, Malkoc, Mulligan, & Webster, 2010) provides additional evidence for this suggestion. 

Here we test whether pattern adaptation can enhance salience. We measured the effects of prolonged (40 s) adaptation with a counterphase grating on performance in a search task in which subjects searched arrays of counterphase Gabors for targets defined by an orientation difference. Adaptation reduced reaction times and the number of saccades needed to find the target. These results suggest that adaptation may enhance the salience of targets that differ from their temporal context. 

Methods

Apparatus

Stimuli were displayed on a calibrated CRT monitor (1024 × 768 pixels; 60 Hz frame rate; ∼40cd/m² mean luminance). The display was viewed binocularly from a distance of 50 cm, subtending ∼35° of visual angle. Stimuli were generated with custom software based on OpenGL (EXPO), running on an Apple Macintosh MacPro. In a subset of experiments, eye movements were sampled at 1 kHz using a SR Research EyeLink 1000 system. Measurements of the right eye position were digitized and recorded in EXPO. Sessions involving eye tracking were run as described above except that a chin rest was used. 

Procedure

Subjects searched arrays of counterphase Gabor patches to locate a single target defined by an orientation offset (or orientation “feature contrast”; Treisman & Gelade, 1980) relative to distracters. Each stimulus was 1° in diameter (0.167° standard deviation of the envelope), had a spatial frequency of 2 cycles/deg. and temporal frequency of 6.25 cycles/s, and was presented at 50% contrast. Unless otherwise noted, arrays consisted of 12 stimuli. 

In the standard display, stimuli were arranged in a 4 × 3 grid spanning 14° × 8°, centered on the fixation point. The elevation and azimuth of each stimulus was shifted randomly and independently on each trial, by up to 2°, so that the grid structure was not perceived. Stimuli were separated by at least 1° and were presented at least 2° from the fixation point. All distracter stimuli had the same orientation, which was randomly assigned on each trial to be either parallel to the orientation of the adapter or orthogonal to it. The orientation of the target was offset from the distracters by 10°, 15°, 20°, 25°, 45°, or 90°. The orientation and location of the target were randomly assigned on each trial. Approximately 14% of trials were catch trials in which no target was presented (i.e., the target had the same orientation as the distracters). 

To assess the effect of distracter number on performance, subjects viewed arrays of 6, 12, and 24 stimuli. Arrays of 6 and 24 stimuli were presented in grids spanning 6° × 5° and 18° × 9°, respectively, and the elevation and azimuth of stimuli were jittered by up to 1°. The relative spacing between stimuli was the same as above: at least 1° between stimuli and 2° between stimuli and the fixation point. 

Subjects participated in adaptation and control test sessions. Adaptation sessions (Figure 1A) began with the appearance a central fixation point (a white circle 0.12° in diameter), which remained on the screen for the duration of the session. After 2 s a full contrast counterphase sinusoidal grating of fixed orientation (usually vertical; 18° diameter, spatial frequency 2 cycles/deg, temporal frequency 6.25 cycles/s) was presented for 40 seconds. This was immediately followed by the first test trial. Subjects were instructed to locate the target stimulus as fast as possible and to indicate with a key press whether it was on the right or left side of the fixation point. Subjects were free to move their eyes while searching, but were instructed to return their gaze to the fixation point after responding. Stimulus presentation was terminated by the subject's response or after 2 s. Trial length was capped at 2 s so that the effects of adaptation would not dissipate between tests. 5 s of ‘top-up' adaptation were presented between test trials. Each session consisted of 168 trials: 12 presentations (once at each possible location) of each target offset, on two different backgrounds. 

Figure 1

 

The search task. (A) Adaptation sessions began with a 40 second exposure to a vertical full contrast counterphase grating. Trials were separated by 5 s of ‘top-up' adaptation. Targets (highlighted by yellow circles for presentation purposes only) were defined by an orientation offset. Targets and distracters were counterphase Gabors presented at 50% contrast. Distracters were oriented either vertically (90°) or horizontally (0°). Target orientation offset ranged from 0° (catch trials) to 90° . The target orientation and location were randomly chosen on each trial. Trials were terminated by subjects' response or after 2 s. (B) Control sessions. As in (A) except that sessions began with the presentation of the first test trial and trials were separated by a 5 s noise mask.

Control sessions (Figure 1B) were similar except that the appearance of the fixation point was immediately followed by the first test trial and a 5 s noise mask composed of black and white random dot fields (i.e., 0 coherence random dot kinematograms; dot diameter = 0.08°, density = 150 dots/deg²/s) was interleaved between test trials. The mask served to clear the mental representation of the previously presented stimulus and to match the temporal structure of the adaptation sessions. 

Participants

Six subjects participated in these experiments: two authors (SW and CP) and four naïve subjects. All subjects participated in the main experiment, with eye tracking. Supplemental conditions were tested on different subsets of the subject pool. Daily sessions consisted of at least one control and one adaptation session. Subjects participated in 3–4 days of testing per test condition (consecutively whenever possible), plus 1–2 practice days to familiarize them with the task. All subjects had normal or corrected to normal vision. Naïve subjects received monetary compensation. All procedures followed protocols approved by the university's Institutional Review Board. 

Analysis

Reaction times were recorded for all trials on which subjects correctly located the target; error trials (< 3%) were excluded from analysis. Trials on which subjects failed to respond within 2 s accounted for about 7% of all trials and were included in the analysis as reaction times of 2 s. These trials occurred almost exclusively for targets orientation offsets of 10° (∼ 35% of trials) indicating that they represented instances when the subject required more time to find a weakly salient target. We calculated the effects of adaptation in our data set both including and excluding these trials and found nearly identical results. 

Eye position data were analyzed off-line. We smoothed the eye position data with a Gaussian kernel (five samples wide, standard deviation of 1.5 samples) and then calculated the velocity between successive samples. Saccades were defined by epochs above the 95^th percentile for velocity in a particular session, which were separated by at least 25 samples during which the eyes moved more slowly. These criteria were chosen so that the number of saccades counted by the algorithm matched that provided by a visual inspection of subjects' scan paths. 

We calculated the percent change in responses (either reaction times or numbers of saccades) following adaptation. To assess statistical significance, we calculated, by a bootstrap procedure, the 95% confidence intervals for the null hypothesis that the pre- and post-adaptation responses for each subject were drawn from a common distribution. For each target offset, we combined the responses from each subjects in both control and adaptation conditions. We then sampled with replacement from this common distribution to calculate “control” and “adapted” responses, with the number of observations for each case equal to that in the original data. We then averaged the percentage change across subjects. We repeated this procedure 1,000 times and defined the confidence interval by the 2.5 to 97.5 percentile ranked values. 

Indications of variation are standard error of the mean, except in our measurements of orientation discrimination thresholds. For these data, 95% confidence intervals were determined by a bootstrap procedure in which we calculated pre- and post-adaptation psychometric functions by resampling each subject's data 1,000 times from the measured responses with replacement. 

Results

We developed a task in which subjects searched an array of counterphase Gabor stimuli for a target. Targets were identical to distracters except their orientation was offset by 10°–90°. 

Performance as a function of display set size

Performance on search tasks is known to be affected by the number of distracting stimuli and the effect of display set size on performance is commonly used to assess the salience of stimuli and explore the role of attention in visual search (Verghese & Pelli, 1992; Palmer, 1995; Wolfe, 1998). Classically, search targets fall into two categories. The detection of highly salient “pop-out” targets is thought to arise from automatic, pre-attentive mechanisms operating in parallel across the visual display. Thus, search for these targets is unaffected by the number of distracters. In contrast, targets that do not pop out must be searched for serially, so adding more distracters delays responses (Treisman & Gelade, 1980). More recent work has demonstrated that pop-out targets and those requiring serial search form a continuum, rather than two discrete classes (Wolfe, 1998; Nakayama & Martini, 2011). We began our investigation by locating our stimuli along this continuum. 

Subjects searched for targets in displays of 6, 12, or 24 items and indicated the location of the target (right or left of the fixation point) with a button press. Distracters were oriented vertically. Targets were defined by orientation offsets of 10°, 20°, or 45° relative to distracters. Results for one subject are shown in Figure 2A. For small target offsets (10°; blue), the subject's reaction time (RT) increased with the display set size (mean RT of 1.09 ± 0.07 s for 6 stimuli vs. 1.47 ± 0.03 s for 24 stimuli; p < 0.001, Wilcoxon rank sum test). For large target offsets (45°; red), reaction time was unaffected by the number of display items (0.60 ± 0.02 s vs. 0.63 ± 0.02 s; p = 0.56). Intermediate offsets (20°; green) showed a mixture of these effects with no difference in reaction time for search among 6 and 12 items (0.67 ± 0.04 s vs. 0.68 ± 0.02 s) but slower reaction times for 24 items (0.78 ± 0.03 s, p = 0.03 difference with 6 items). 

Figure 2

 

Effect of number of display items on search latencies. (A) Example results from a single subject. Average reaction times to locate targets with orientation offsets of 10° (blue), 20° (green), and 45° (magenta), as a function of the number of distracters. (B) Mean difference in reaction times for search on displays of 24 and six items for five individual subjects (grey) and across subjects (black). Dashed line indicates no change.

The difference in reaction times for search in 6 and 24 display items is shown in Figure 2B for all subjects. The open circles represent data for four individual subjects; solid points represent the mean across subjects. The mean increase in reaction time for targets offset by 10° was 0.42 ± 0.05 s or ∼25 ms per distracter, indicating that finding these targets required serial search. For target offsets of 45°, the difference was nearly zero (0.001 ± 0.01 s), indicating these stimuli ‘pop-out'. Reaction times for targets with an intermediate orientation offset (20°) had a weak dependence on distracter number (difference of 0.09 ± 0.01 s), indicating that they fall between these two end points. 

Effects of adaptation on search

To determine the effects of adaptation on stimulus salience, we had subjects perform a search task similar to the one described above. Subjects viewed displays of 12 counterphase Gabor stimuli. All distracters shared the same orientation. The target was defined by an orientation offset relative to the distracters, and its orientation and location were randomly assigned on each trial. In the control condition, test trials were separated by a 5 s presentation of a noise mask. In the adaptation condition, test sessions started with a 40 s presentation of a vertically oriented counterphase grating that covered the area in which test stimuli were presented; test trials were separated by 5 s of top-up adaptation (Figure 1). To determine the orientation-specificity of any adaptation-induced changes in performance, we measured reaction times on backgrounds in which the distracters had the same orientation as the adapter (“parallel” condition) or were offset by 90 degrees (“orthogonal” condition; Figure 3, top). 

Figure 3

 

Effect of adaptation on visual search. (A) Example results from a single subject. Mean control (black) and adapted (red) reaction times for search on parallel (left panel) and orthogonal (right panel) backgrounds. (B) Mean percent change in reaction times following adaptation for six individual subjects (grey open circles) and across subjects (black filled circles) for search on parallel (left panel) and orthogonal backgrounds (right panel). Grey fill indicates 95% confidence interval for the null hypothesis. Adaptation reduces search times, particularly on parallel backgrounds.

Mean control reaction times for one subject (black) are shown in Figure 3B. For targets with orientation offsets of 10°, mean reaction time were 1.18 ± 0.05 s and 1.39 ± 0.05 s for search on parallel and orthogonal backgrounds, respectively. Reaction times decreased with increasing target offsets and were fastest for pop-out targets (90° offset on parallel background 0.61 ± 0.01 s, orthogonal background 0.60 ± 0.01 s). Across subjects, control reaction times tended to be slower for search on orthogonal as compared to the parallel background (p < 0.001 for each target offset except 90°; Friedman test), indicating a possible asymmetry for search on these two backgrounds (Treisman & Gormican, 1988; Foster & Ward, 1991; see Wolfe, 2001 for additional examples of search asymmetries). 

Adaptation reduced reaction times for search on parallel backgrounds for this subject (Figure 3B, left panel, red). For instance, for target offsets of 20° the subject's mean adapted RT was 0.58 ± 0.01 s as compared to 0.71 ± 0.01 s in the control condition (p < 0.001, Wilcoxon rank sum test). Adaptation also shortened reaction times for search on the orthogonal background (right panel; 0.60 ± 0.02 s vs. 0.69 ± 0.01 for 20° offset targets, p < 0.001), although the effect was smaller. 

For each target offset, we calculated the percent change in reaction time following adaptation for each subject (grey open circles; Figure 3B, n = 6) and then averaged across subjects (solid black circles). The grey filled area represents the 95% confidence interval for the null hypothesis that the pre- and post-adaptation data for each subject came from the same distribution (see Methods). The mean percent change in reaction time was reduced when subjects searched on the parallel background (left panel). For all target offsets smaller than 90°, the mean change in reaction times fell outside the null hypothesis confidence interval, with a maximal effect of −19% for targets with an offset of 10°. A small improvement was also evident for a target offset of 45°. Although search for this target stimulus was not influenced by display set size (Figure 1), subjects did occasionally require additional time to find it. Adaptation reduced the occurrence of these trials. The effect of adaptation was similar in nature but weaker when subjects searched on the orthogonal background (p < 0.001, Friedman test comparing effects on the two backgrounds across all target offsets). 

We found no evidence that shorter response latencies following adaptation reflected a trade-off between speed and accuracy. Subjects generally performed the task with high accuracy both before and after adaptation. For search on the parallel background, subjects made the most errors for 10° targets, responding incorrectly on 12.9 ± 3.0% of trials in the control condition and 8.3 ± 4.4% of trials following adaptation (p = 0.21; Wilcoxon rank sum test). Thus, if adaptation had any effect on error rates, it was to make subjects more accurate. For search on orthogonal backgrounds, the corresponding error rates were 24.1 ± 5.4% and 22.2 ± 4.0% (p = 0.47). Error rates fell rapidly with increasing target orientation offsets and were < 0.5% for offsets of 20° or larger, both before and after adaptation. 

Our search task was performed with the adapter and distracters at cardinal orientations. Performance on many tasks is better at these orientations (the oblique effect; Appelle, 1972; Girshick, Landy, & Simoncelli, 2011). To test whether our results generalize to other distracter orientations, two subjects repeated the main experiment with the adapter and distracters at oblique orientations. The adapter and parallel distracters were oriented at 135° and the orthogonal distracters at 45° as shown in Figure 4 (top row). Example data from one subject are presented in Figure 4A, and the average effect of adaptation across subjects is shown in Figure 4B. The effects of adaptation were similar to those obtained from search among distracters at cardinal orientations. Adaptation significantly decreased reaction times in the parallel background condition (Figure 4B left panel, solid black points). The mean percent change in reaction times following adaptation fell below the null hypothesis confidence intervals for all offsets, except 90°. The effect was weaker for the orthogonal background condition (right panel) and, for most cases, did not reach statistical significance. 

Figure 4

 

Effect of adaptation on search with obliquely-oriented distracters. (A) Example results from a single subject. Mean control (black) and adapted (red) reaction times for search on oblique parallel (135°; left panel) and orthogonal (45°; right panel) backgrounds. (B) Mean percent change in reaction times following adaptation for two individual subjects (grey open circles) and across subjects (black filled circles) for search on oblique parallel (left panel) and orthogonal backgrounds (right panel). Grey fill indicates 95% confidence interval for the null hypothesis.

In summary, adaptation reduced reaction times on a visual search task. Adaptation-induced improvements were strongest for test stimuli with small orientation offsets, which required some degree of search. Improvements were also orientation specific in that they were stronger for search on parallel than on orthogonal backgrounds. 

Testing for additional influences on search performance

In our task, test stimuli were preceded by a noise mask in control sessions and by a counterphase grating in adaptation sessions. Therefore, either forward masking or priming effects could have contributed to the differences in performance we observe. To address this, we tested three subjects on a modified adaptation protocol in which a 200 ms noise mask (identical to that of the control condition) was inserted between the adapter or ‘top-up' stimulus and the test stimuli (Figure 5A). The stimulus immediately preceding the test was thus identical for control and adaptation sessions. 

Figure 5

 

Testing the contribution of immediate temporal context. (A) Control sessions (top) were the same as in the standard protocol. In the adaptation session (bottom), a noise mask (0.2 s duration) was inserted between ‘top-ups' and test trials. (B) Mean control (black) and adapted (red) reaction times for an example subject for search on parallel (left panel) and orthogonal (right panel) backgrounds. (C) Mean percent change in reaction times following adaptation for three individual subjects (grey open circles) and across subjects (black filled circles) for search on parallel (left panel) and orthogonal backgrounds (right panel). Grey fill indicates 95% confidence interval for the null hypothesis. Adaptation reduces search times, even when the test stimuli are preceded by a brief noise mask in the adaptation condition.

Adaptation reduced search times, even with the introduction of a mask between the adapter and test stimulus. Example data from a single subject are presented in Figure 5B, and summarized for all subjects in Figure 5C. For target offsets of 20°, for instance, the mean percent change in reaction times following adaptation was −8.5 ± 0.5% and −5.5 ± 1.4% on the parallel and orthogonal backgrounds, respectively. Effects fell outside the null hypothesis confidence interval for most orientation offsets on the parallel background (Figure 5C). 

To compare these effects with those obtained using our standard protocol, we calculated the mean percent change in reactions times for the three subjects that participated in both protocols. Across offsets of 10°–45°, where effects were strongest, the mean reduction in reaction time in these subjects for the standard protocol with parallel distracters was −16.3%, compared to −9.8% with the mask; on the orthogonal background, the improvement was −11.1% and −2.1% for the standard and mask conditions. We found no significant difference between the effects of adaptation measured with these two protocols (p > 0.1 for all comparisons; Friedman test), although we cannot rule out a weak effect due to limited power. 

We have interpreted the reduction in reaction times as an indication that adaptation facilitates search by altering salience. However, adaptation could also reduce search times by improving subjects' ability to discriminate small orientation differences between the target and the distracters, facilitating target identification. To test this, we measured the effects of adaptation on orientation discrimination in two subjects using the method of constant stimuli. Subjects viewed arrays of 12 oriented Gabor patches, but the position of each stimulus was fixed and the location of the target stimulus was identical on all trials (Figure 6A). On each trial, the target was offset from the distracters by an orientation ranging between −45° and 45°. Subjects were instructed to indicate with a key press whether the target was rotated clockwise or counter-clockwise, relative to the distracters. This allowed us to measure discriminability, in the absence of search, in a display that was nearly identical to that used in our main experiment. The structure of the control and adaptation sessions was also identical to the main experiment. 

Figure 6

 

Effect of adaptation on orientation discrimination. (A) Schematic of discrimination task. Target's location was fixed (indicated by arrows) and had as orientation ranging from −45 to +45° relative to the distracters. The orientation was randomly assigned on each trial. (B) Average psychometric functions for two subjects before (black points) and after (red points) adaptation. Means represent data from 48 trials (S1) or 36 trials (S2). Solid lines indicate fits to the data. Horizontal and vertical dashed lines represent 50% correct and 0° orientation offset respectively. Adaptation had little effect on discrimination performance.

Psychometric functions for each subject in the control (black) and adapted (red) conditions were fit with cumulative Gaussian functions to determine discrimination thresholds (sigma or 84% correct; Figure 6B). Subject 1 (left panel) had a discrimination threshold of 2.03° (95% confidence interval of 1.63°–2.31°; see Methods) in the control condition and 2.08° (1.50°–2.40°) following adaptation. Similarly, subject 2 had a control threshold of 4.28° (2.01°–7.55° ) and an adapted threshold of 2.54° (2.31°–7.95°). Thus, discrimination thresholds were much lower than the smallest target offset (10°) and there was minimal change after adaptation. This indicates that the improved performance on our search task following adaptation is not due to a change in target discriminability. 

The strength of adaptation effects in our experiments might also be affected by target location. Subjects were required to fixate during adaptation but were free to move their eyes to find the target during test trials. The adapter had a radius of 9° and the maximal target eccentricity was 7°. Therefore, when subjects viewed the most peripheral targets, targets on the opposite side of the display could fall outside the adapted visual field. If unadapted neural mechanisms recruited at such times contributed to the search strategy (e.g., planning where to look next), this might lead us to underestimate the effects of adaptation on search. 

To evaluate this possibility, we compared the effects of adaptation on search for inner targets (eccentricity of 2°) and outer targets (eccentricity of 5° to 7°). We found no evidence of an effect of target location on percent change in reaction times following adaptation (p = 0.10 for parallel backgrounds and p = 0.46 for orthogonal backgrounds; Friedman test). Importantly, if there were contribution of this effect to our measurements, it would only lead us to underestimate how strongly adaptation affects search. 

Effects of adaptation on eye movements

The neural machinery underlying perceptual salience is thought to be tightly linked to the oculomotor system (see Treue, 2003; Schutz, Braun, & Gegenfurtner, 2011 for review). Specifically, neural responses underlying perceptual salience are used by the oculomotor system to control subjects' gaze so that salient regions may be foveated for further inspection. To gain greater insight into how adaptation facilitates visual search, we recorded the eye movements of six subjects as they participated in our standard search task. 

Typical scan paths from one subject in two control trials are presented in Figure 7A. On these trials, the subject made four saccades to find a target whose orientation was offset by 10° and only one saccade to locate a target offset by 90°. On average, subjects made 3.07 ± 0.09 saccades to locate targets with an orientation offset of 10° (Figure 7B, solid black point, left panel). For offsets of 45°, subjects required just 1.37 ± 0.04 saccades to find the target. Thus, targets with small orientation offsets required subjects to scan the display, whereas targets with offsets of 45° or larger often “popped-out” and were located in just one or two saccades. 

Figure 7

 

Effects of adaptation on saccades during search. (A) Typical scan paths to locate targets with orientation offsets of 10° (left) and 90° (right). (B) Mean number of saccades made to locate targets for four individual subjects (grey) and across subjects (black). (C) Mean percent change in the number of saccades made by individual subjects (grey) and averaged across subjects (black) following adaptation, for search on parallel (left) and orthogonal (right) backgrounds. Grey fill indicates 95% confidence interval for the null hypothesis. Adaptation tends to reduce the number of saccades made to find the target, on parallel backgrounds.

Adaptation reduced the number of saccades made by subjects (Figure 7C). For search on parallel backgrounds, the maximal effect was seen for target offsets of 20° (decrease of −15.1 ± 3.5%) effects fell below the null hypothesis confidence interval for all offsets except 90°. For search on the orthogonal background, the effect of adaptation was weaker (−7.5 ± 3.2% for target offset of 20°) and the mean percent change was significantly different from zero only for target offsets of 20°–45°. 

Thus, similar to the reduction in search time, adaptation caused a decrease in the number of saccades needed to find the target, presumably by altering the salience of the target. 

Discussion

Our results show that adaptation improves performance in a visual search task, for targets requiring some degree of serial search (i.e., particularly for those with orientation offsets less than 45°). Reduced response latencies were associated with a decrease in the number of saccades needed to find the target. For highly salient ‘pop-out' stimuli (targets with orientation offsets of 45° or larger), adaptation had weak or no effects. The effects of adaptation on search were orientation specific as they were stronger when distracters had the same orientation as the adapter than the orthogonal orientation. 

Mechanisms by which adaptation might influence search performance

Models of salience rely on retinotopic arrays of detectors with center-surround antagonism (Wolfe, 1994; Li, 1999; Itti & Koch, 2000). This spatial contextual suppression is strongest when a stimulus is surrounded by similar stimuli. Thus, detectors responding to targets produce stronger responses than those encoding distracters. The difference in the magnitude of responses to targets and distracters determines target salience. 

Adaptation reduces the responses of detectors in a graded manner, most strongly affecting those whose stimulus preference is similar to the adapter (for review see Kohn, 2007). In the parallel background condition, distracters are identical to the adapter so the responses they evoke will be strongly reduced following adaptation. Responses to targets will also be reduced but less so, since their orientation is offset from the adapter. Thus, adaptation should increase the difference between responses to the target and distracters, increasing target salience and reducing search times. The precise improvement predicted depends on the strength and orientation specificity of adaptation-induced response suppression, relative to that of spatial contextual effects. 

For orthogonal backgrounds, targets are more similar to the adapter than distracters are, so the responses they elicit should be more strongly reduced by adaptation. This would reduce the difference with the responses to the distracter, resulting in longer search latencies, particularly for targets offset by 90° (which have the same orientation as the adapter). We did observe a weak tendency for response latency to increase for 90° offset targets, on orthogonal backgrounds (Figures 3B and 4B). However, we also observed faster reaction times for search on orthogonal backgrounds, for targets with smaller orientation offsets. 

Why did performance improve for some target offsets for search on orthogonal backgrounds? One possibility is that adaptation suppresses target responses far below that of the distracters. If salience is determined not by the strongest response but by the greatest differential response (i.e., the response most different from others; Itti & Koch, 2000), a particularly weak response to the target would make it salient. Consistent with this possibility, we found that search times could be reduced by lowering the contrast of a target stimulus of fixed orientation offset, relative to the contrast of distracters (unpublished observations; see also Boehnke et al., 2011 for a related neural effect). 

An alternative explanation is that adaptation reduced the strength of the spatial contextual signal. We have recently shown that, in primary visual cortex (V1), adaptation can both reduce and facilitate neuronal responsivity (Wissig & Kohn, 2012; see also Tailby et al., 2008, Camp et al., 2009; Patterson, Wissig, & Kohn, 2013). Adaptation can enhance responses to stimuli that recruit strong surround suppression, and this occurs because adaptation reduces the strength of surround suppression (Webb, Dhruv, Solomon, Tailby, & Lennie, 2005). For search on orthogonal backgrounds, this mechanism could enhance the response to targets with intermediate orientation offsets: after adaptation, responses to these targets would no longer be suppressed by the distracters, because surround suppression would be less effective. For distracters, whose orientation is orthogonal to the adapter, there would be no change in surround suppression because adaptation effects on the surround are orientation specific. The net influence of adaptation on salience will thus depend on the relative strength and specificity of adaptation-induced reduction in response strength and surround suppression. 

A less mechanistic explanation for the effects we see on orthogonal backgrounds is that salience utilizes distinct maps to represent the different elemental features of stimuli (orientation, contrast, color, etc). Thus, an orientation difference between targets and distracters is represented in an orientation feature map, whereas a difference in the contrast is represented in a contrast feature map. Adaptation reduces the apparent contrast of stimuli similar to the adapter (Blakemore, Muncey, & Ridley, 1973; Hammett, Snowden, & Smith, 1994), which would induce differential responses to the target and distracters in the contrast map. According to this interpretation, adaptation would not hinder search on orthogonal backgrounds by reducing the difference between responses to distracters and targets; it would facilitate search by creating an additional feature cue—the reduced perceived contrast of the target, as compared to the distracters. Adaptation may also affect the apparent orientation of the target as compared to the distracters (i.e., the tilt aftereffect). However, this effect is typically only a degree or so in magnitude (Clifford, 2002) and thus would likely contribute only weakly to the enhancement of target salience reported here. Finally, it is possible that adaptation induces some global improvement in search, which aids performance on both backgrounds, in addition to an orientation-specific effect involving detectors in early visual cortex (Palmer, 1995). 

It is important to emphasize that the improvement in search performance after adaptation was based on a comparison to a baseline condition in which test stimuli were preceded by a noise mask. In the adaptation experiment, the test stimuli on both backgrounds were preceded by a full-field counterphase grating. Thus, some of the non-specific improvement after adaptation (i.e., that occurring on both parallel and orthogonal backgrounds) may have reflected the similar spatial structure of the test stimuli and adapter, relative to the differently structured mask stimulus: the noise mask may have made both target and distractor Gabors highly salient in the control condition, impairing the ability to find targets. Consistent with this explanation, we found the improvement in search performance on parallel backgrounds was weakened by the introduction of a brief mask between adapter and test (but remained significant), and nearly eliminated for many test stimuli on orthogonal backgrounds. 

Relationship to other work on salience and adaptation

Few studies have explored how adaptation affects visual search. Theeuwes and Lucassen (1993) showed robust effects on search for vertical or horizontal line segments embedded in backgrounds of differently oriented lines. However, the authors used chromatic adaptation, presented at a subset of potential target locations, rather than adapting subjects to the feature (orientation) on which search was based. Thus, the improvement they observed likely involved a spatial highlighting effect and reduced uncertainty about target location. 

McDermott et al. (2010) had subjects search for targets defined by a color and form cue, on backgrounds chosen from different color distributions. Consistent with our results, adaptation facilitated search on backgrounds matching the adapter, with an improvement similar in magnitude to the effects we observed. In contrast to our findings, however, subjects had slower latencies and used more saccades to find targets on orthogonal backgrounds after adaptation. This discrepancy with our findings may reflect differences in design, including the stimulus domain containing the feature contrast (color vs. orientation, which may rely differently on retinal vs. cortical effects; see Tailby et al., 2008), the density of the display (dense vs. sparse), and the adaptation protocol (3 m vs. 40 s). Alternatively, the discrepancy could be due to differences in the definition of baseline performance: in our case, measured with a control condition with masking noise preceding the test stimulus, and in McDermott et al. the use of a gray screen. If our control condition, but not that of McDermott et al., reduced the relative salience of all targets, this could explain why we saw improvement after adaptation, for both parallel and orthogonal backgrounds. The different baseline comparisons in our study and theirs may also differentially recruit low- and high-level mechanisms. In any case, our studies are in agreement that adaptation can improve search performance, and that the effects are largest when the adapter is similar to the background or distracters. 

In contrast to these studies and ours, Ng, Boynton, and Fine (2008) found no effect of adaptation on visual search performance. In their task, subjects were adapted to a face category (e.g., male Asian) for 3 m and then instructed to search for a face from this category in a display with distracter faces. Effects of adaptation with complex stimuli like faces may thus be distinct from those involving low-level stimulus features (e.g., orientation, color). 

Although the interaction between visual adaptation and search has been explored by only a handful of studies, search task performance has long been known to be affected by temporal context (Nakayama & Martini, 2011). Performance on search tasks improves with perceptual learning (Karni & Sagi, 1991; Ahissar & Hochstein, 1993). Maljkovic and Nakayama (1994) also showed that reaction times in a dynamic search task were faster for recently encountered targets, a priming effect that showed little retinotopic specificity suggesting it was distinct from low level adaptation. Finally, Najemnik and Geisler (2005) showed that human scan paths in a search task were consistent with those of ideal observers who have learned the statistical structure of the natural world and the spatial resolution of their own visual system. Thus, the idea that search is influenced by stimulus history, either recently encountered or learned through development, is well established. Our study and that of McDermott et al. (2010) suggest that classic adaptation—exposure to a stimulus or ensemble of stimuli for tens of seconds—can also influence search. 

Adaptation is frequently used for dissecting mechanisms of visual processing (Graham, 1989), but the functional role of many adaptation effects are unclear. Our results suggest one important role may be to make salient those stimuli that differ from those recently encountered. That is, adaptation may serve to highlight novel stimuli or associations, so that changes in the environment are more easily detected (Barlow, 1990). This suggestion is consistent with some previous physiological studies, including adaptation-induced changes in receptive field structure in the retina (Hosoya, Baccus, & Meister, 2005) and the mismatch negativity, a scalp potential evoked by the appearance of an ‘oddball' stimulus (Naatanen, Tervaniemi, Sussman, Paavilainen, & Winkler, 2001). In addition, while visual salience has often been considered to be determined by spatial context, there is perceptual (Sachtler & Zaidi, 1993; Murakami & Shimojo, 1995; Tadin, Lappin, Gilroy, & Blake, 2003) and physiological (Webb et al., 2005, Tailby et al., 2008; Camp et al., 2009; Patterson et al., 2013; Wissig & Kohn, 2012) evidence that spatial and temporal contextual effects interact strongly, further supporting a common function for these influences. 

Acknowledgments

This work was supported by the National Institutes of Health (EY016774), the Army Research Office (58760LS) and Research to Prevent Blindness. CA Patterson and SC Wissig were supported in part by an NIH MSTP training grant (T32-GM007288). We thank Ruben Coen-Cagli, Norma Graham, Odelia Schwartz, and Sam Solomon for helpful comments on an earlier version of this manuscript. 

Commercial relationships: none. 

Corresponding author: Adam Kohn. 

Email: [e-mail] 

Address: Department of Neuroscience, Albert Einstein College of Medicine, Bronx, NY, USA.