Adaptation and visual salience

Abstract

We examined how the salience of color is affected by adaptation to different color distributions. Observers searched for a color target on a dense background of distractors varying along different directions in color space. Prior adaptation to the backgrounds enhanced search on the same background while adaptation to orthogonal background directions slowed detection. Advantages of adaptation were seen for both contrast adaptation (to different color axes) and chromatic adaptation (to different mean chromaticities). Control experiments, including analyses of eye movements during the search, suggest that these aftereffects are unlikely to reflect simple learning or changes in search strategies on familiar backgrounds, and instead result from how adaptation alters the relative salience of the target and background colors. Comparable effects were observed along different axes in the chromatic plane or for axes defined by different combinations of luminance and chromatic contrast, consistent with visual search and adaptation mediated by multiple color mechanisms. Similar effects also occurred for color distributions characteristic of natural environments with strongly selective color gamuts. Our results are consistent with the hypothesis that adaptation may play an important functional role in highlighting the salience of novel stimuli by discounting ambient properties of the visual environment.

Introduction

Adaptation is a pervasive and central process in sensory coding. Even brief exposure to a stimulus can produce profound changes in sensitivity and correspondingly dramatic changes in perception. Yet, why do these changes occur? Many studies have explored the purpose of adaptation, yet its functional consequences have yet to be fully characterized (Clifford et al., 2007; Kohn, 2007; Webster, 2004; Webster, Werner, & Field, 2005). One general class of consequences is changes in appearance and recognition, for example of how things “look” after adapting to visual stimuli. These appearance aftereffects may be intimately linked to the establishment and calibration of norms for representing the perceived properties of the world and may also be fundamentally important for maintaining perceptual constancy when the world or the observer changes. That is, adaptation may be important for allowing objects to be coded consistently and thus recognized despite variations in the viewing context. A second class of consequences involves changes in detection and discrimination, in how well observers can distinguish between stimuli. Light adaptation offers a clear example of the crucial role that adaptation plays in maximizing sensitivity by centering the limited dynamic range of neural responses around the ambient light level (Barlow, 1972; Shapley & Enroth-Cugell, 1984). Yet the benefits of sensitivity regulation beyond light and chromatic adaptation have proven more difficult to demonstrate. In particular, pattern-selective adaptation—to attributes such as movement or shape that reflect response changes at cortical rather than retinal levels of visual processing—can also lead to marked changes in appearance and sensitivity, yet has not been as readily shown to improve detection or discrimination (Clifford et al., 2007; Webster, 2004). In this sense, pattern adaptation has remained an enigma because it is not clear in what ways the adaptation makes vision better. 

In this study, we examined whether adaptation aids the detection of new stimuli—or statistical outliers—in the observer's environment. Most prior studies of discrimination following pattern adaptation have examined how the adaptation affects the perception of stimuli that are similar to the adapting stimulus. For example, many studies have explored whether adaptation to contrast improves contrast discrimination around the mean adapting contrast (Barlow, Macleod, & van Meeteren, 1976; Greenlee & Heitger, 1988; Maattanen & Koenderink, 1991; Ross, Speed, & Morgan, 1993; Wilson & Humanski, 1993), or whether adaptation to a pattern like a face improves discrimination near the adapting face (Ng, Boynton, & Fine, 2008; Rhodes, Maloney, Turner, & Ewing, 2007; Rhodes, Watson, Jeffery, & Clifford, 2010), to test for analogies to the sensitivity gains when the eye is adjusted to the mean luminance or color of the stimulus. As noted however, consistent benefits with contrast or pattern adaptation have proven difficult to establish, and where they have been found the effects are modest compared to the drastic changes that adaptation induces in the appearance of the patterns. 

We asked whether the adaptation can instead facilitate the detection of stimuli that specifically differ from the adapting stimulus. That is, we examined whether an important consequence of adaptation may be on how we perceive stimuli that we are not adapted to. Recognizing new stimuli or events is clearly important to perception and learning, and adaptation may be one of many neural mechanisms designed to promote the detection of novelty (Ranganath & Rainer, 2003). Our work was motivated by a proposal by Barlow that one function of adaptation may be to more efficiently encode the ambient properties of the environment in a way that can highlight and draw attention to new properties (Barlow, 1990b). These novel features or “suspicious coincidences” are the least expected and thus most informative clues to the world, and thus are the stimuli that should capture attention (Barlow, 1990a, 1990b; Itti & Baldi, 2009). However, the detection of novelty requires that observers somehow learn the prior statistics for scenes. In other words, we are more likely to detect an outlier if the environment itself is familiar rather than novel. Barlow proposed that adaptation might be one process through which visual coding can incorporate scene statistics (Barlow, 1990b). For example, if two stimulus dimensions tend to covary in scenes (e.g., color and orientation), then adaptation might adjust the responses of mechanisms tuned to each dimension so that their responses are decorrelated, and in this way accentuate the response to stimuli that embody new relationships. Formal models based on decorrelation can successfully account for the selective effects of contrast adaptation on color appearance (Atick, Li, & Redlich, 1993; Zaidi & Shapiro, 1993), or for contingent aftereffects such as orientation-contingent changes in color in the McCollough Effect (Barlow & Foldiak, 1989). Physiological studies have also shown that neurons can selectively adapt to the contingencies in the stimulus (Carandini, Barlow, O'Keefe, Poirson, & Movshon, 1997). However, whether adaptation can be shown to actually improve the ability to detect novelty remains largely untested. We explored how adaptation alters salience of novel stimuli by testing the effects of adaptation on visual search. 

Contextual effects on visual search have been studied extensively. For example, prior knowledge or expectations about targets and distractors can strongly affect how attention is deployed (Wolfe, 1994), and previewing sets of distractors can allow them to be better disregarded during a subsequent search (Watson & Humphreys, 1998). A large number of studies have also explored priming effects on search (Kristjansson & Campana, 2010). Repeated presentation facilitates target detection, even for high salience targets that pop-out when they have not been shown previously (Maljkovic & Nakayama, 1994). These priming effects occur for repetition of both features and locations and for both targets and distractors (Kristjansson & Driver, 2008) and have been interpreted as a form of implicit memory that biases attention for the “learned” properties of the stimulus (Kristjansson & Campana, 2010). Priming has in part drawn interest because it clearly shows that what we currently attend to depends importantly on what we have seen and searched for previously. In the present work, we asked how the history of stimulation might influence attention and search in a different way, by directly changing the bottom-up salience or perceptual code for stimulus properties as observers are repeatedly exposed and thus adapted to the stimulus. The pronounced and selective changes that adaptation induces in visual sensitivity and appearance could potentially have substantial impact on visual salience (Webster, 2004). Yet to our knowledge there has been very little work on the effects of varying the observer's state of adaptation on search performance. One exception was a study demonstrating that chromatic adaptation at one spot on the retina could alter the perceived color of a target at that location and thus alter its salience (Theeuwes & Lucassen, 1993). We instead examined the consequences of spatially global adaptation to the statistical characteristics of backgrounds, to try to simulate the patterns of adaptation that might arise when observers are immersed in a given environment. Our aim was to ask whether visual search for statistical outliers improves when observers are appropriately adapted to their environment. 

To explore adaptation effects on visual search, we focused on search for color. Color is among the clearest features that affect stimulus salience and drive visual search (Wolfe & Horowitz, 2004), and thus is an important feature for understanding the consequences of adaptation on search. Moreover, many of the properties of color search have been well characterized (Bauer, Jolicoeur, & Cowan, 1996, 1998; D'Zmura, 1991; Nagy, Neriani, & Young, 2005; Nagy & Sanchez, 1990; Nagy & Thomas, 2003; Nagy, Young, & Neriani, 2004), though it is not well established how these properties are manifested in the more complex and dense color distributions typical in natural viewing. Salience increases systematically as the color difference between targets and distractors increases (Bauer et al., 1996; Duncan & Humphreys, 1989; Nagy & Sanchez, 1990). Thus we could probe how adaptation influenced targets that varied widely in salience. When distractors include more than one color, the detectability of the target depends not only on the absolute color differences but on the color relationships between the target and background. D'Zmura (1991) showed that targets that lie between two distractors in color space and thus fall within the background distribution are difficult to detect, whereas targets that fall outside the distribution support rapid search. This effect roughly follows the principle of linear separability: search is rapid (or slow) when targets can (or cannot) be separated from the background elements by a line in color space. Importantly, this effect was found to hold for any direction in the space, including different chromatic axes or different combinations of luminance and color. This suggests that visual search depends on mechanisms that can be tuned to many different directions in the volume of color space (Bauer et al., 1996; D'Zmura, 1991; Nagy & Thomas, 2003; Nagy & Winterbottom, 2000). 

We also focused on color because the effects of adaptation on color appearance are strong and relatively well characterized (Webster, 1996). Adaptation to a distribution of colors that are confined to restricted directions in color space results in large and selective changes in color appearance (Webster & Mollon, 1991, 1994). Specifically, the adaptation reduces the apparent contrast of colors along the adapting axis more than other directions. These selective contrast changes are accompanied by changes in the perceived hue of stimuli, which rotate away from the adapting axis and thus toward the orthogonal axis. The rotations are analogous to the tilt aftereffects that result from adaptation to oriented lines or gratings, but are often substantially stronger, with biases of 30 deg or more possible in perceived hue angle. Like visual search, these aftereffects can also be selective for any arbitrary direction in color space, and thus again imply that they are mediated by mechanisms that can be tuned to different color and luminance directions. This selectivity could plausibly arise if color is encoded by multiple “higher order” mechanisms with fixed tuning functions that vary in their relative sensitivity with adaptation (Krauskopf, Williams, Mandler, & Brown, 1986; Webster & Mollon, 1994). Alternatively, color contrast adaptation has served as the principal example for modeling adaptive changes in channel tuning functions in order to decorrelate their outputs as proposed by Barlow (Atick et al., 1993; Barlow, 1990b; Zaidi & Shapiro, 1993). Either way, we hypothesized that the changes in the neural code for color induced by contrast adaptation could alter the relative contrast and hue of targets and distractors in ways that could either enhance or diminish the conspicuousness of novel colors. 

To test these ideas, in our experiments observers searched for target colors on highly cluttered backgrounds. The backgrounds were composed of a random collage of overlapping ellipses with colors drawn from different distributions in color space, while the targets were single elements with a color that varied from within to far removed from the background distribution. The stimulus was specifically designed to simulate the natural and ecologically significant task of foraging for a fruit among a dense and variegated background of foliage. Note that this display is more complex and natural than the discrete elements that are traditionally used to study visual search, while more constrained than natural images so that we could still carefully control the role of color in the task and the state of adaptation. We asked if observers were faster at finding the fruit if they were first adapted to the forest. Our results suggest that adaptation to the ambient color distribution does enhance visual search for novel colors, and suggest that these improvements can occur for the color distributions characteristic of some natural color environments. Thus our results support the notion that one potential function of visual adaptation may be to enhance sensitivity to novel properties of the world. 

Methods

Apparatus and stimuli

Stimuli were displayed on a SONY 20SE or 500PS color monitor controlled by a Cambridge Research Systems VSG card. Stimulus levels on the monitor were calibrated with a spectroradiometer (Photo Research PR650) to enable the reproduction of arbitrary colors from colorimetric specifications. Luminance levels were defined photometrically rather than empirically for individual observers. Observers viewed the display binocularly from a distance of 260 cm, at which the screen subtended 6 by 8 deg. For the studies measuring eye movements, the viewing distance was decreased to 57 cm and the display subtended 31 by 41 deg. Eye movements were recorded with a Cambridge Research Systems Video Eyetracker. The observer's right eye was tracked during each search. 

The search task involved locating a color singleton on a dense background of distractors whose luminances and chromaticities varied along different directions in color space. An example of the stimuli is shown in Figure 1, in which the background colors vary in luminance and in chromatic contrast along the LvsM axis of the color space (see below). The two images illustrate the case where the target color is very different from the background and thus is easy to find, or where the target color is similar to the background and thus largely camouflaged.

Figure 1

 

Examples of a highly conspicuous (left) or inconspicuous target on color backgrounds defined by chromatic variations along the LvsM axis.

 

The background was formed by drawing up to 40,000 superimposed ellipses in video memory to create a virtual image ∼9 times the displayable area on the screen. Different samples from the background could then be displayed by selecting different locations. The high density was chosen to completely pave the image, but as a result, most ellipses were partially occluded and many were completely occluded. Each ellipse was chosen to have a random position and orientation and a random size that varied between 0.2 and 0.4 deg along the minor axis and between 0.6 and 0.8 deg along the major axis. The color of each ellipse was also chosen at random from a predefined distribution and was reassigned for different trials by randomly regenerating values in the look-up table. 

On each trial, the target was shown at a random location on the left or right side of the displayed background. The target was always superimposed on the background elements and thus never occluded. For most experiments, the target was a uniform circle with a diameter of 0.5 deg, chosen so that shape provided a weak but consistent cue to detection. The salience of the target was varied by varying the target's color. Because the target could be found by shape alone, we could assess search even for colors that fell within the background distribution. However, this also meant that search times were necessarily limited by the time required to scan the display for this cue. In conditions monitoring eye movements, the circular target was instead replaced by an ellipse so that only color cues could be used for detection. 

Target and background colors were defined by their contrasts along the LvsM, SvsLM, and Luminance axes of cone-opponent space and were scaled to nominally equate sensitivity to the cardinal axes based on a prior study (Webster & Mollon, 1994) according to the following equations: 

$\mathrm{L}\mathrm{v}\mathrm{s}\mathrm{M}=(r_{\mathrm{m}\mathrm{b}}-r_{\mathrm{w}})*1955,$

(1)

 

$\mathrm{S}\mathrm{v}\mathrm{s}\mathrm{L}\mathrm{M}=(b_{\mathrm{m}\mathrm{b}}-b_{\mathrm{w}})*5533,$

(2)

 

$\mathrm{L}\mathrm{u}\mathrm{m}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{e}=\left(\right(L-L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}})/L_{\mathrm{m}\mathrm{e}\mathrm{a}\mathrm{n}})*70,$

(3)

where r _mb and b _mb are the chromaticities in the MacLeod–Boynton color space (MacLeod & Boynton, 1979), r _w and b _w are the coordinates of the white point (with values of 0.6568 and 0.01825, equivalent to illuminant C), and L is the photometric luminance with a mean value of 30 cd/m². Background colors were typically restricted to a single line or plane within this space by uniform random sampling of colors over a contrast range of ±80. For experiments in which the chromatic axis of the background was varied, the ellipses were also randomly varied in luminance contrast over a range of ±30 units (±40%). Target colors varied over the same contrast range and sampled different levels within the LvsM and SvsLM chromatic plane at the mean luminance or within the Luminance and LvsM plane, depending on the experiment. For example, Figure 2 plots the full set of target and background color contrasts within the chromatic plane, with backgrounds drawn from one of four continuous lines of contrasts at 0–180, 45–225, 90–270, or 135–315 deg, and targets (spots) from varying contrasts (0, 10, 20, 40, and 80 units) at each 45 deg interval. Each image shows a subset of the target stimuli against one of the backgrounds that vary along the LvsM axis (0–180 deg) or the SvsLM axis (90–270 deg), or along the two intermediate chromatic axes (45–225 or 135–315 deg), corresponding to color variations along the positive or negative diagonal of the LvsM and SvsLM plane (as defined by our scaling of the plane). Note that on each background the effective target contrast—on which target salience should depend—varies with the orthogonal distance of the target color from the line defining the background axis (e.g., salience varies with SvsLM contrast on the LvsM background and with LvsM contrast on the SvsLM background). We analyze the search results in terms of this relative contrast, as opposed to the absolute target contrast (distance from the achromatic origin of the space), which would be related to salience on an achromatic background.

Figure 2

 

Target and background colors used in the search task. Each image shows examples of the target chromaticities against one of the 4 backgrounds that each varied along a single axis in the chromatic plane. Target chromaticities spanned a range of angles and contrasts in the plane (filled circles in the diagram) while backgrounds uniformly sampled contrast along one angle in the plane (lines in the diagram).

 

Procedure

Observers searched for targets on the backgrounds after adapting to the same or different backgrounds. During adaptation, a random segment from the stored background was displayed and updated every 300 ms. This resampling was designed to prevent local chromatic adaptation and instead adapted the observer to the consistent color statistics of the background, and is similar to the pattern of adaptation that might occur from rapidly and randomly scanning the background. In most experiments, observers initially adapted for a period of 3 min. A new sample of the background was then displayed with a target and signaled by a tone. This image remained static until observers indicated the location of the target (left or right) by responding on a keypad. Thus the dependent measure was the reaction time required to locate the target. Subsequent targets were shown interleaved with 6-s intervals of readaptation, with reaction times recorded for each response. Trials with an incorrect response occurred approximately 3% of the time and were excluded. On a very small fraction of trials (<1/1000) the target was obscured because it fell on a cluster of background elements that were very similar in color. A separate response button was provided to also exclude these trials. Otherwise, all correct responses were included for analysis. 

During a single run, observers were adapted to a single background but searched for the targets on two different backgrounds—one that was again a sample from the adapting background and one that varied along the orthogonal color axis. For example, if they adapted to the LvsM axis, then on each trial the target would appear on either a background defined by the LvsM axis or on a background defined by the SvsLM axis. The 2 test backgrounds were randomly interleaved until each of the 33 targets was shown on each background, resulting in 66 trials per run. Note that this randomization, as well as the large array of possible target colors, meant that observers could not anticipate which background or target color they would be searching for on any trial. In a single daily session, observers completed 4 or 5 runs with the same adaptation and target set. Mean reaction times were normally based on 20 repetitions of the same target, background, and adapting condition, with the order of adaptation counterbalanced across multiple sessions. The study was thus designed to compare the reaction times for searching for the same color targets under 4 adapting and background conditions: 

No background: Chromatic targets were detected on a background that had no chromatic contrast (but varied in luminance contrast), after adapting to a uniform gray field. For targets that instead varied in both color and luminance, the background during both testing and adaptation was a uniform gray field. 

Neutral adaptation: Targets were detected on a color-varying background after adapting to a uniform gray field. 

Same-background adaptation: Targets were detected on a color-varying background after adapting to random samples from the same background. 

Orthogonal-background adaptation: Targets were detected on a color-varying background after adapting to random samples from a background that varied along a perpendicular direction in the color space. 

The “no background” condition provided baseline reaction times for detecting the targets based on their absolute color contrast (i.e., on and after adapting to a uniform field), while the three adaptation conditions were chosen to examine how the presence and adaptation to different backgrounds affected target salience, as measured by search times. 

Participants

Observers included 3 of the authors and 9 additional observers who participated for course credit. All observers had normal color vision as assessed by standard screening tests. Different observers were tested in different subsets of conditions, with all conditions confirmed on naïve observers. All procedures followed protocols approved by the university's Institutional Review Board. 

Results

Search in the chromatic plane

An example of the search times on each of the 4 backgrounds is shown in Figure 3 for one observer. The upper two panels show performance on the cardinal LvsM (0–180 deg) or SvsLM (90–270 deg) background, while the lower panels plot the settings when the background varied along the two intermediate chromatic axes (45–225 deg or 135–315 deg). In each case, the search times are plotted as a function of the target contrast as defined by the distance of the target chromaticity from the background axis (e.g., as the SvsLM contrast when searching on the LvsM background or vice versa). The time to locate the targets decreases asymptotically as this distance increased. Search for “zero-contrast” targets, which lay within the background color axis, required several seconds on average and was again limited by detecting the weak shape cue. Alternatively, high target contrasts that were far removed from the background axis were detected rapidly. Notably, these targets did not completely “pop out,” for the search times remained longer than when the targets were presented against a uniform field. Thus the backgrounds weakly interfered with the target detection even when they varied along “independent” directions in color space, and even when these directions were confined to the LvsM or SvsLM cardinal axes (Nagy et al., 2005). Alternatively, there is very clear selectivity for each of the axes. That is, search times always improved as the target deviated from the background, regardless of the direction in the chromatic plane. This would not be predicted for the intermediate axes (i.e., the 45–225 deg or 135–315 deg backgrounds) if detection were based on the independent signals along the LvsM and SvsLM axes (since the component contrasts along the cardinal axes were identical in the two cases), and confirms the selectivity of visual search for multiple color directions (Bauer et al., 1998; D'Zmura, 1991; Monnier & Nagy, 2001; Nagy & Thomas, 2003).

Figure 3

 

Search times for a single observer on each of the 4 backgrounds. Each plot shows the reaction times as a function of the distance of the target from the background axis. Search times on each background are shown after adaptation to (a) the same background (red circles), (b) the orthogonal color background (blue triangles), or (c) a uniform field (black squares). Lines show the search times when the targets were instead shown on backgrounds that varied only in luminance.

 

Again, our primary aim was to ask how performance varied with the observer's state of adaptation. To compare this more directly, the average search times from Figure 3 have been replotted in Figure 4 to show the change in search times across the different adaptation conditions. The panels plot the response times for either the same or orthogonal adaptation minus neutral adaptation (i.e., adaptation to the uniform field) for each target, and now show the settings for 3 observers at each background axis. To test for effects of adaptation, we used a sign test to compare the number of times a target was found more quickly under each condition. Comparisons were tested for 3 cases: (1) same vs. orthogonal adapt: to test whether observers were faster finding a novel color when adapted to the same background they had to search on vs. the “opposite” background; (2) same vs. neutral adapt: to test whether prior adaptation to the search background facilitated performance relative to no prior exposure; and (3) orthogonal vs. neutral adapt: to test whether adaptation to the orthogonal color background impeded performance relative to no prior exposure. The three comparisons thus addressed different questions but are not independent. For each, we excluded tests of zero contrast that were within the background axes, since no effect was predicted for these stimuli, and pooled settings across the observers. The results of this analysis are shown in Table 1 and indicate that there were in fact significant adaptation effects along each axis. Specifically, search for the color targets was faster when observers were first adapted to the background they were searching on and was slower when they were adapted to the orthogonal background. Both outcomes are consistent with an adaptation effect that is selective for the target or background axis. That is, adaptation to the orthogonal axis is likely to reduce sensitivity and thus salience along the very axis that distinguishes the target from the background (e.g., searching for SvsLM tests on an LvsM background should be harder if observers are adapted to the SvsLM contrast). Conversely, adaptation to the background may facilitate search by decreasing sensitivity to the background (e.g., finding SvsLM targets on LvsM backgrounds becomes easier if observers are adapted to the LvsM contrast). However, it is less evident that the changes in salience are a simple consequence of the changes that adaptation induces in the perceived contrast of the targets and distractors, an issue we address below.

Figure 4

 

Relative search times on the 4 chromatic backgrounds. Symbols in each panel plot the difference in response times between neutral adaptation and adaptation to the same background (red circles) or orthogonal background (blue triangles) for 3 observers.

Table 1

 

 

Sign tests comparing search times on different background axes. Fractions give the proportion of targets that were found more quickly on each background after adapting to (1) the same background vs. neutral background; (2) neutral background vs. the orthogonal background; or (3) same background vs. the orthogonal background.

Table 1

 

Sign tests comparing search times on different background axes. Fractions give the proportion of targets that were found more quickly on each background after adapting to (1) the same background vs. neutral background; (2) neutral background vs. the orthogonal background; or (3) same background vs. the orthogonal background.

LvsM/SvsLM plane—manual response
Background	Same < Neutral	Neutral < Orthogonal	Same < Orthogonal
0–180 Deg	56/72: p < 0.0001	50/72: p = 0.0006	66/72: p < 0.0001
45–225 Deg	50/72: p = 0.0006	40/72: NS	53/72: p = 0.0001
90–270 Deg	52/72: p = 0.0001	51/72: p = 0.0003	61/72: p < 0.0001
135–315 Deg	57/72: p < 0.0001	57/72: p < 0.0001	64/72: p < 0.0001
LvsM/Luminance plane—manual response
Background	Same < Neutral	Neutral < Orthogonal	Same < Orthogonal
0–180 Deg	42/72: NS	57/72: p < 0.0001	62/72: p < 0.0001
45–225 Deg	54/72: p < 0.0001	27/72: NS	46/72: p = 0.0122
90–270 Deg	40/72: NS	58/72: p < 0.0001	60/72: p < 0.0001
135–315 Deg	49/72: p = 0.0015	33/72: NS	47/72: p = 0.0064
LvsM/SvsLM plane—eye tracking
	Radial target array	Rectangular target array
Background	Same < Orthogonal	Same < Orthogonal
0–180 Deg	53/56: p < 0.0001	66/72: p < 0.0001
90–270 Deg	53/56: p < 0.0001

 

Interestingly, the settings also reveal a marked asymmetry between the search times on the two intermediate chromatic axes. Specifically, observers were faster finding the targets on the bluish-yellowish (135–315 deg) background than on the opposite magenta-greenish (45–225 deg) background. This is shown in Figure 5 by comparing the search times on either background under neutral adaptation. A sign test comparing search times for the two cases was again highly significant (by < mg, 63/72, p < 0.0001). Note again that these axes have the same component contrasts along the LvsM and SvsLM axes, and thus the difference reflects an interaction between these cardinal axes. Note also that the difference is evident only when searching on the color-varying backgrounds. That is, blue and yellow targets were found as quickly as the complementary axis colors on a uniform gray field. The results suggest that visual search for color cannot be accounted for by separable LvsM and SvsLM dimensions, and in particular suggest that bluish-yellowish backgrounds have a lower effective contrast.

Figure 5

 

Comparison of search times for targets shown on the bluish-yellowish (135–315 deg) background (red circles) or magenta-greenish (45–225) background (blue triangles) following adaptation to the uniform field. Solid and dashed lines plot search times for the corresponding targets on the achromatic backgrounds. The three panels show results for three observers.

 

Search in the luminance vs. chromatic plane

We next extended the search experiments to examine backgrounds that had different combinations of luminance and chromatic contrast. Contrast adaptation is strongly selective for luminance or chromatic variations and can also be selective for how color and luminance covary (Webster & Mollon, 1994). For example, adapting to a bright-red/dark-green modulation reduces perceived contrast along this axis more than to a bright-green/dark-red axis, even though both axes contain the same luminance and chromatic components. We therefore tested whether adaptation to these backgrounds could also selectively enhance visual search for targets defined by novel color–luminance directions. For these conditions, the backgrounds varied along a single axis in the Luminance and LvsM plane and consisted of either a pure luminance or chromatic variation, or covarying luminance and chromatic contrast along axes of 45–225 or 135–315 deg within the plane (again as defined by our choice of scaling). The range of contrasts once more extended over a range of ±80, and observers now searched for targets that spanned different directions and contrasts in the luminance-chromatic plane (Figure 6).

Figure 6

 

Target and background colors used for testing search for different luminance and chromatic axes. Each image shows examples of the target stimuli against one of the 4 backgrounds that each varied along a single axis in the Luminance and LvsM plane.

 

Figure 7 shows an example for one observer of the search times on the 4 backgrounds, each again tested under 4 conditions (same, orthogonal, or neutral adapt, or no background), while Figure 8 plots the search times for the same or orthogonal adaptation relative to neutral adaptation for the 3 observers tested. Search for the targets was clearly selective for the luminance-chromatic axis defining the background. Specifically, as with the backgrounds along different chromatic axes, the salience of targets along each color–luminance axis increased as they deviated from the background axis. This is consistent with studies from adaptation, masking, and visual search suggesting that luminance and chromatic contrast are not processed independently (Gegenfurtner & Kiper, 1992; Hansen & Gegenfurtner, 2006; Nagy & Winterbottom, 2000; Webster & Mollon, 1991, 1993). For the present results, there is a strong asymmetry between the response times for the luminance and chromatic targets, with faster detection for targets with luminance contrast. Unlike the asymmetry noted above for the two intermediate chromatic directions, the luminance vs. chromatic difference is also evident when observers were searching on the uniform gray background. While this difference could in part reflect a difference in how contrasts along the two axes are scaled, it is likely that it also reflects faster processing for luminance than chromatic signals (e.g., Bompas & Sumner, 2008; Braithwaite, Watson, Andrews, & Humphreys, 2010).

Figure 7

 

Search times for a single observer on each of the 4 color–luminance backgrounds. Each plot shows the reaction times as a function of the distance of the target from the background axis. Search times on each background are shown after adaptation to (a) the same background (red circles), (b) the orthogonal color background (blue triangles), or (c) a uniform field (black squares). Lines show the search times when the targets were instead shown on uniform fields.

Figure 8

 

Relative search times on the 4 color–luminance backgrounds. Symbols in each panel plot the difference in response times between neutral adaptation and adaptation to the same background (red circles) or orthogonal background (blue triangles) for 3 observers.

 

Comparisons of the adaptation effects are given in Table 1. These were less robust than the aftereffects for the different chromatic axes but still show a consistent effect for faster search times when observers were adapted to the same background than to the orthogonal background. Thus like the chromatic axis results, adaptation again modulated the relative salience of the background and targets, and these effects were again selective for the color–luminance direction defining the stimuli. 

Search for targets within the background color axis

Recall that for each of the different backgrounds we tested, the targets included 9 stimuli that fell along the background axis and thus had zero effective contrast (Figures 2 and 6). Specifically, these stimuli could not be distinguished as the target by a unique color and thus could be found only by the shape difference (though this does not preclude the possibility that the local color contrast between the target and background may have affected the visibility of the target—e.g., if a reddish target happened to fall on top of all greenish distractors). We also analyzed the effects of the different adaptation conditions on these trials, to again compare whether searches were faster when observers were adapted to the same background they had to search on or the orthogonal background. However, for the same 24 comparisons shown in Table 1 for the targets outside the background (i.e., 8 different adapting axes times the 3 contrasted adaptation conditions), only one of the sign tests reached significance for targets inside the background (20/27 faster responses after adapting to the same background vs. neutral adaptation for the 45–225 deg chromatic axis; p = 0.01). The results remained insignificant when we increased the power by pooling across all 8 backgrounds. Thus there was little evidence that adaptation improved search based on the circular shape of the target. One implication of this is that the faster search times we found following adaptation to the search background—and the impaired performance for the orthogonal background—were not simply a general tendency to respond more quickly or slowly on these backgrounds. Instead, the adaptation specifically impacted performance for the targets that differed in color from the background. A second implication is that adaptation apparently did not increase sensitivity to the different contrast levels defining the background. This is at least consistent with other evidence suggesting that contrast adaptation does not improve contrast discrimination, or sensitivity to differences along the adapting axis (Barlow et al., 1976). Finally, it is notable that adapting to the ellipses did not facilitate search for the circle. This is likely because the circle was chosen to be intermediate in shape to the background ellipses and thus was not linearly separable from them (Wolfe, 1994). Had the circle had an area that was larger or smaller than the distribution of ellipses then adaptation might again have enhanced detecting it. 

Color search and chromatic adaptation

In the preceding cases, the mean luminance and chromaticity were the same for all backgrounds. Any differences between the pre-exposure conditions thus depended on how the visual system adjusts to stimulus contrast. In the next set of conditions, we instead examined the effects of differences in the mean color of the stimuli. Adapting to an average color bias in the background should induce a negative aftereffect in colors seen subsequently. For example, after adapting to a reddish field both the background and target colors should all appear greener. These chromatic adaptation effects arise primarily in the retina and produce changes in color appearance that are largely separable from the changes induced by cortical contrast adaptation (Webster & Wilson, 2000). Adjustments to the mean are well known to be important for coding contrasts, and color salience has been shown to depend strongly on the relationships between targets and distractors and the mean background color (Rosenholtz, Nagy, & Bell, 2004). Here we again focused on how color salience is affected when observers are adapted to the mean color they are searching on or a different color. 

Figure 9 shows an example of the stimulus chromaticities. In this example, observers searched for a range of colors centered on LvsM or SvsLM background that had a mean value of 34 along the +L–M axis so that they had a reddish bias. Adaptation was to a uniform field with the same mean chromaticity as the background or to a chromaticity instead shifted 34 units in the −L+M direction, which (under neutral adaptation) appeared more greenish. Because of chromatic adaptation, the adapting background and thus color contrasts centered on the background should appear more neutral over time. Conversely, when adaptation is to the −L+M background, the switch to the +L–M background should cause the colors to appear reddish. How these mean shifts affect salience should depend on the relationship between the color directions defining the backgrounds and the mean color shift, and specifically, on whether the adaptation is changing sensitivity along the color direction required for discrimination (Krauskopf & Gegenfurtner, 1992). For example, suppose that the background and mean color difference both vary only along the LvsM axis. In that case, chromatic adaptation should only alter the sensitivity of the L and M cones, while the targets are visible only because of signals from the S cones, which remain in a constant adaptation state. Thus chromatic adaptation should not affect the search times. Alternatively, if the mean shift is along the LvsM axis while the backgrounds vary along the SvsLM axis, then the targets can be distinguished from the background only by the LvsM signals, and the shift in adaptation should make observers less sensitive to these signals (since contrast thresholds increase with increasing background contrast; Krauskopf & Gegenfurtner, 1992; Switkes, Bradley, & De Valois, 1988). In general then, for colors along the LvsM or SvsLM axis, adaptation should not affect search when the background and mean changes are along the same axis (Figure 10) but should impede search when they are along orthogonal axes (Figure 11).

Figure 9

 

Coordinates of the targets and background colors used to assess the effects of adaptation to the mean chromaticity on color search.

Figure 10

 

Backgrounds that vary in contrast along the same axis as the mean color shift, and thus orthogonal to the axis that distinguishes the target from the background.

Figure 11

 

Backgrounds that vary in contrast along axes perpendicular to the mean color shift.

 

An example of the search times following chromatic adaptation is shown in Figure 12 for one observer, for each of the 8 conditions illustrated in Figures 10 and 11. The top panels compare search for the targets when the background colors and mean color change were along the same axis. This resulted in comparatively little change in the search times despite a large presumed change in the perceived colors of both the targets and the backgrounds when the mean color of the backgrounds shifted. Sign tests based on 2 observers' settings nevertheless showed that detection rates for the targets improved when observers were adapted to the mean color of the background in 3 of the 4 cases (Table 2, unshaded cells). This improvement is counterintuitive, because as noted above for these adaptation conditions there should be minimal change in the signal defining the target. Moreover, adapting to the complementary background color might have been expected to reduce the effective contrast of the background (e.g., an LvsM background centered on +L would appear gray and span a wide range of reddish to greenish contrasts after adaptation to this background; while the same chromaticities might instead all appear more reddish and thus more similar after adapting to the −L background). Yet if anything search was again faster after adapting to the mean of the background. One possibility is that adaptation induced a common hue into both the targets and the background (e.g., so that they all appeared redder). This hue shift might have increased the perceived color similarity between the target and background, thus reducing salience, even though the target–distractor distance remained unaffected. We explore this possibility further below.

Figure 12

 

Search times for a single observer on each of the 8 backgrounds with a mean color shift. Top 4 panels: Background axes that are parallel to the shift in mean color (Figure 10). Bottom 4 panels: Background axes that are perpendicular to the direction of mean color shift (Figure 11). Each plot shows the reaction times as a function of the distance of the target from the background axis. Search times on each background are shown after adaptation to the mean color of the background (red circles) or to the complementary color (blue triangles).

Table 2

 

Sign tests comparing search times on LvsM- or SvsLM-varying backgrounds after adapting to a mean bias in chromaticity along either axis. Cells compare search times when the background axis is on (unshaded) or orthogonal to (shaded) the mean color difference.

 

The lower panels of Figure 12 show search times when the backgrounds were orthogonal to the mean adaptation shift, so that target salience depended on the signals along the mean adapting axis. Here the effects of adaptation were dramatically larger and again significantly faster when observers were adapted to the mean color they were searching on (Table 2, shaded cells). Again this is consistent with adaptation to the complementary mean color leading to a saturated contrast response along the target axis (Krauskopf & Gegenfurtner, 1992; Switkes et al., 1988). These results thus highlight the importance of adaptation to both the mean and variance in the color distributions for modulating color salience. While the sensitivity adjustments underlying chromatic adaptation are distinct and reflect different stages of visual coding (Webster, 1996), these again produce selective changes in color sensitivity that differentially affect sensitivity to backgrounds and targets in ways that functionally serve to enhance the salience of novel colors. 

Adaptation vs. learning

The preceding results demonstrate that prior exposure to the colors defining a background can facilitate search for novel colors on that background. In the case of mean changes in chromaticity, there is little doubt that the pre-exposure effects reflect an influence of adaptation. Yet for the backgrounds that varied only in color axis it remains uncertain whether the improvements in performance are a consequence of adaptation and how it was changing the effective salience of the targets and backgrounds, or whether observers were instead simply learning which specific colors were in the background and using this to better guide their search. To try to distinguish between these possibilities, we conducted control experiments to assess whether the pre-exposure effects were more consistent with perceptual adaptation or learning. 

Color search and eye movements: For these experiments, we switched from measuring reaction times to monitoring eye movements while observers searched for the targets. Our goal was to assess whether observers were scanning the displays in the same way under the different adaptation conditions. If there were changes in the search strategies between the familiar and novel backgrounds, then these differences might provide an alternative account of why search performance was affected by pre-exposure to the backgrounds. 

As noted in the Methods section, for these experiments viewing distance was decreased so that field size increased to 31 by 41 deg, and the circular target was replaced by an ellipse, in order to allow us to also assess the search performance when color was the only cue available to the target. Because of this, these experiments only used target colors that differed from the adapting background. For different experiments, the set of target chromaticities formed either a radial (Figure 13a) or rectangular (Figure 13b) grid of coordinates. During adaptation, the background was also sampled at a different rate (750 ms), and a black fixation cross was added to the final adapting frame so that observers were fixating the center of the display at the beginning of each test. This frame was displayed for a variable time (1 s ± 0.5 s) to avoid anticipatory responses. The adaptation sequence otherwise remained the same. During the search, we recorded the number, location, and duration of each saccade. For the purposes of the experiment, saccades were defined as a shift in eye position of 1 deg or more during the 20-ms sampling interval of the eye tracker. Target detection was defined as a fixation within 2.5 deg of the target that was maintained for 500 ms, at which point the trial terminated. The trial was also terminated if the target was not located within 10 s, though this occurred only for a very small number of trials.

Figure 13

 

(a) Radial or (b) rectangular grid of coordinates for target colors used in the search tasks in which eye movements were monitored.

 

Figure 14 shows illustrative results for one observer tested with the radial grid of targets. The top panels compare the time to locate the targets on the LvsM or SvsLM background after adapting to either. The results replicate the manual reaction times by again revealing that observers were faster finding the targets when they were searching on the same background axis they were adapted to. In fact, in these conditions, the effects were substantially stronger and highly significant for all of the 4 observers tested (Table 1). We are not sure of the basis for this difference, though as noted there were several differences in the specific stimuli in the two experiments. The second row shows results for the same searches but, this time, plots the average number of saccades as a function of target contrast. This measure again reveals a consistent difference between the backgrounds—observers required fewer fixations to locate the target when searching on the background they were adapted to. Importantly, this rules out a simple “startle” effect as the basis for the difference in search times between the same and orthogonal backgrounds. That is, the sudden switch to a new background could theoretically have delayed the start of the search if attention was momentarily captured by the change. Note that this would be in contrast to priming, which instead facilitates search by reducing the latency of the first saccade (Becker, 2008). Yet this should have only introduced a constant added delay and thus should not have increased the number of saccades. The third row plots fixation durations vs. contrast. These decreased with higher contrast targets, in line with previous reports (Hooge & Erkelens, 1998), but again also showed a significant difference with adapting condition. Finally, the lowest panels show the saccade magnitude as a function of contrast. Unlike the preceding measures, these differed only on 4 of the 8 conditions (2 adapt × 4 observers) tested, and thus showed less effect with adaptation. We also did not observe consistent changes in saccade amplitude though this has also been found previously to vary with target–distractor similarity (Jacobs & O'Regan, 1987).

Figure 14

 

Eye movement patterns for one observer for target search on the (left column) LvsM or (right column) SvsLM background. Top row: Search times vs. target contrast after adaptation to the LvsM background (red circles) or SvsLM background (blue circles). Second row: Number of fixations as a function of target contrast. Third row: Average fixation duration as a function of contrast. Bottom row: Saccade amplitude vs. contrast.

 

Are these differences in the eye movements with adaptation condition because targets were more salient on familiar backgrounds, or because observers scan familiar backgrounds in different ways? For example, familiarity with the background might somehow allow observers to sample the image more efficiently. This could be reflected in shorter fixations if they could encode or recognize the colors more quickly, or in larger saccades if the “attentional spotlight” were broadened so that they could sample the scene more coarsely. Since both fixation duration and saccade magnitude could vary with target salience, to isolate differences in search strategies, we instead compared eye movements on the two backgrounds as a function of search time. However, when compared in this way neither measure was significantly different between the two adapting conditions. We also attempted a further analysis (not shown) where we compared the direction of the saccades as a function of search time. These should generally point toward the target for highly salient colors that pop out, while varying more randomly for camouflaged targets. However, searches on the two backgrounds were again indistinguishable by this metric. The lack of an effect of the adaptation on the scan patterns is perhaps not surprising because observers may already be sampling the images in nearly optimal ways (Najemnik & Geisler, 2005). Alternatively, the analysis did reveal some differences between observers, for example in the scan paths or initial fixation directions typical of different observers. Yet though observers may have used different idiosyncratic strategies to search the images, these strategies appeared similar whether they were searching on a highly consistent or novel background. This is in line with the finding that observers tend to search in similar ways whether target conspicuity is fixed and thus known or varied across trials (Over, Hooge, Vlaskamp, & Erkelens, 2007). Thus the results for the eye movements are consistent with the possibility that targets were easier to find among familiar colors—not because observers searched among these colors in different ways, but because adaptation to them increased the salience of unfamiliar colors. 

Adaptation to nonselective backgrounds: As a second test for discriminating a familiarity effect from an adaptation effect, we repeated the search experiment with a “nonselective” color background. In this case, the background colors were sampled from hue angles at all directions but with a fixed contrast of 28 (Figure 15). We reasoned that adaptation to this background should lead to a general loss in sensitivity to all color directions and thus would not in principle facilitate search. Alternatively, if observers learned the set of background colors, then this might again allow them to more quickly detect a new color. For this condition, target colors also spanned a range of hue angles and had contrasts of 57 or 80. In pilot studies, we also included targets that had a lower contrast than the background, including a zero-contrast gray. However, these proved too difficult to locate, as might be expected because these targets were not linearly separable from the background colors (D'Zmura, 1991).

Figure 15

 

Target and background coordinates for color search on the nonselective color background.

 

Results for 3 observers are shown in Figure 16. A sign test of the search times did not significantly differ between adaptation to the color background and adaptation to a grayscale background that had the same luminance contrast but no chromatic contrast, and instead approached significantly longer searches for the color background (same background < neutral; 8/24, p = 0.076). The lack of an advantage on the nonselective color background is again consistent with the predictions for contrast adaptation while at odds with a simple learning account, since prior exposure to the background colors did not help observers to ignore them. Thus this further suggests that the facilitation we found for selective backgrounds was a consequence of the adaptation rather than learning which colors defined the background.

Figure 16

 

Average search times as a function of the color axis for targets presented on the nonselective background, after adaptation to the background (red circles) or a uniform field (blue triangles). The three panels show results for three observers.

 

Even though this nonselective background was chosen to be unbiased in color, search times for all of the observers systematically varied with the hue angle of the target in the color space. In particular, the longest and shortest response times tended to be along +S and −S poles of the SvsLM axis (90 and 270 deg), respectively. This asymmetry was also reported by Nagy and Sanchez (1990) for color search among uniform-color distractors. Interestingly, this difference was not evident in our measurements reported above when observers were searching on the backgrounds confined to a single axis (e.g., where, as seen in Figure 4, the responses to S-cone increments and decrements are more symmetrical). However, the present conditions differed in that the targets were along the same axes as the background elements but had a higher contrast than any of the background colors. The basis for this asymmetry in the search times is uncertain though it is consistent with a compressive nonlinearity along the SvsLM axis that might have made S-cone decrements more salient than S-cone increments (Boynton & Kambe, 1980). 

Search and target contrast

In all of the conditions thus far, we defined target contrast as the perpendicular distance from the target to the background axis. However, as noted in the context of the chromatic adaptation results, it is possible that the critical color difference was in the relative hue angle of the target and background. For example, on the LvsM background, a set of targets could be chosen such that they varied in LvsM contrast but had the same SvsLM contrast. This would equate the effective contrast difference from the background (since this depends only on the SvsLM contrast) but would not equate the hue difference from the background (since this depends on the ratio of SvsLM to LvsM). If salience depended in part on the target to background hue difference, then adaptation might in part help or hinder search by increasing or decreasing the perceived hue difference rather than changing the relative effective contrasts per se. To assess whether hue differences also played a role in the target salience, we compared search times for targets that were sampled from a rectangular lattice of chromaticities rather than a radial array of targets (Figure 13b). This allowed us to test targets that were the same in terms of their distance from the background axis but which varied widely in hue. The search times were measured with the eye tracker for targets presented on the LvsM background for 2 observers and on the SvsLM for a third observer. 

Results are shown for each observer in Figure 17 as plots of the search times in terms of the component target contrasts along the LvsM or SvsLM axis. As contrast increases from the axis background, search times decreased and were again significantly faster after adapting to the background axis than to the orthogonal axis (Table 1). Searches for contrasts running parallel to the background were instead nearly flat, and if anything tend to dip near the central chromaticities in the array. Note that along these dimensions the lower contrast targets are more nearly perpendicular to the background axis and thus are the furthest in hue angle from the background. Conversely, they are the closest targets to the mean color of the background. The fact that search times did not substantially improve for the targets at the extreme LvsM contrasts of the grid indicates that the distance from the mean of the background is a poorer predictor of color salience. Alternatively, the tendency for more perpendicular hue angles to be found more quickly suggests that hue differences may play some role in determining the visibility of the targets, so that a simple metric like distance to the background axis does not fully capture the salience of the stimuli.

Figure 17

 

Search times for targets sampled from the rectangular stimulus grid, as a function of the (left column) LvsM or (right column) SvsLM contrast of the targets. Search times are shown after adaptation to the background axis (red circles) or the orthogonal axis (blue triangles). Lines plot the fits of Gaussian functions to the search times. Top and middle panels: Results for two observers searching on the LvsM background. Bottom panels: Results for a third observer tested on the SvsLM background.

 

Search and natural color distributions

In the final set of experiments, we extended the measurements to examine the properties of visual search within color gamuts that more closely approximate the color characteristics of actual natural scenes. For this, we used two distributions from the study of Webster and Mollon, one corresponding to a panoramic view of an arid scene in the Sierra Nevada mountains and the second taken from a forest in the Western Ghats in India (Webster & Mollon, 1997). These were chosen because Webster and Mollon (1997) had previously tested the effects of adaptation on color appearance for these distributions, and because they represented roughly the range of variation typical of the outdoor environments they sampled. In particular, the Sierra scene included sky and dry vegetation and thus had a strong blue–yellow bias in the distribution, while the Western Ghats scene was dominated by more lush foliage and had a principal color axis close to the SvsLM axis. For each, we again used the circular target and manual reaction times to sample search for a wide range of colors within and outside the distributions along each of the principal planes of the color space. Figure 18 shows the color distributions from each scene and the set of target contrasts, while Figure 19 shows examples of the backgrounds. Note that the target array was centered on the mean chromaticity of each scene, and observers were always adapted to the mean for each distribution. For both distributions, the mean luminance was reduced to 10 cd/m² for display on the monitor, but the range of chromaticities and luminances was otherwise preserved. The color of each ellipse was selected at random from the measured distribution. As a result, the backgrounds obviously do not include the spatial variations in color in natural scenes (e.g., there was no segregation between earth and sky) but are again typical of the pattern of local stimulation that would arise from randomly scanning the scene.

Figure 18

 

Color distribution (red circles) from an (top) arid or (bottom) lush scene used to define the colors of the backgrounds, and the set of target colors (blue triangles). The distribution colors are plotted as contrasts along the different pairs of cardinal axes.

Figure 19

 

Examples of the backgrounds defined by the (left) arid or (right) lush color environment.

 

Figure 20 shows the average reaction times for targets on both backgrounds for 3 observers. As before, the reaction times predictably varied from several seconds when the target color was within the background gamut to low values for target colors that were very distant from the background, but the metric for distance in this case is less evident. For the measure shown in the figure, we treated the search as a signal detection task (Palmer, Verghese, & Pavel, 2000) and calculated the signal-to-noise ratio (SNR) of each target contrast relative to the RMS contrast of the background (given by the standard deviation of the background colors). (An alternative to RMS contrast would be to choose the maximum background contrast along each axis, though for our distributions these values were very highly correlated (r > 0.99), with peak contrast roughly twice the RMS contrast.) Values of SNR were computed over all angles in the color space assuming that color contrasts are encoded roughly uniformly by linear mechanisms tuned to different directions, and thus that the contrast is weighted by the cosine of the angle between the element's axis and the projected axis. Again this is consistent with the selectivity of the color search for multiple color directions observed in the results above and in previous studies (Bauer et al., 1998; D'Zmura, 1991; Nagy & Thomas, 2003; Nagy & Winterbottom, 2000). We further assumed that each axis was encoded by separate “on” and “off” pathways so that background elements only contributed to the noise when they had the same sign as the target (e.g., so that luminance increments were not affected by luminance decrements). Finally, target salience was taken from the color direction that yielded the highest SNR. Note that this often corresponded to “off-axis” directions that differed from the test direction (D'Zmura & Knoblauch, 1998).

Figure 20

 

Search times as a function of signal-to-noise ratio for targets presented against the arid (red circles) or lush (blue triangles) background. The three panels show results for three observers.

 

As Figure 20 illustrates, this simple measure of salience captures much of the variation in search times with target color and, in particular, produces a rough correspondence between the results for different targets (e.g., luminance vs. color) and on the two very different backgrounds. However, on a finer scale there may remain differences on the two backgrounds. For example, search times tended to fall more steeply on the arid color set, a result that might parallel the weaker effects of blue–yellow backgrounds we noted in Figure 5. Moreover, this simple measure cannot predict failures from strict linear separability such as our finding that target salience is affected even by orthogonal color backgrounds (Bauer et al., 1996; Nagy et al., 2005). Nevertheless, the similarities in performance across the many different backgrounds and targets we tested suggest that the differences in the behavior of color search for different color directions are not pronounced, and that much of this behavior can therefore be approximated by a single metric. The results for the natural color backgrounds further suggest that target colors do not begin to emerge from the background until the signal is roughly 3 times higher than the background and does not asymptote until the ratio reaches a value of 6 or more. Thus by this measure color salience roughly follows the background distribution but is substantially broader. 

Finally, we asked whether the adaptation effects we observed for visual search on highly selective backgrounds could also arise for natural color distributions. Figure 21 compares the search times on either background when observers were first adapted to the background or to a uniform field. Adaptation to the two conditions was interleaved and followed the procedures of the preceding experiments. Sign tests were restricted to color targets with a signal-to-noise ratio greater than 3 (though equivalent effects were also found when all targets were included). For the bluish-yellowish scene, prior adaptation again enhanced search for the targets relative to neutral adaptation (same adapt < neutral; 84/117, p < 0.0001). Alternatively, the differences were not significant for the forest scene. The different outcomes for the two scenes are in fact expected because the color distributions for the scenes varied widely. Specifically, the arid scene was composed of a highly restricted range of hue angles while the forest scene was much less selective, and Webster and Mollon (1997) showed that these differences were also manifested in how selective changes in color appearance were following adaptation to each background. The present results thus again parallel the predictions for adaptation and suggest that natural backgrounds that present a selective set of colors to the observer can again lead to selective changes in color salience through adaptation.

Figure 21

 

Search times on the (left columns) arid or (right column) lush background following adaptation to samples from the background (red circles) or to uniform fields (blue circles). Rows plot the results for 3 observers.

 

Discussion

Visual search is a fundamental and routine behavior central to many of the ways we explore and interact with our environment (Wolfe, 2010). We have shown that one form of search—for unexpected and uncharacteristic colors in the environment—is more efficient if observers are first adapted to their environment. These results have implications for both the processes of color vision and the consequences of sensory adaptation, and we consider these in turn. 

Color in search

As noted, the implications of visual search for color coding have been explored in a number of previous studies (e.g., Bauer et al., 1996, 1998; D'Zmura, 1991; Nagy et al., 2005; Nagy & Sanchez, 1990; Nagy & Thomas, 2003; Nagy, Young et al., 2004). Our results confirm that visual search can be mediated by color differences along multiple directions in color space (Bauer et al., 1998; D'Zmura, 1991; Nagy & Thomas, 2003; Nagy & Winterbottom, 2000). Both the salience of the targets and how salience was affected by adaptation were selective for each of the different axes we probed. This is consistent with a wealth of evidence from both behavioral (e.g., Eskew, 2009) and physiological (e.g., Gegenfurtner, 2003; Lennie & Movshon, 2005) studies for a cortical representation of color in terms of “higher order color mechanisms” (i.e., mechanisms that are selective for color directions that are intermediate to the luminance, LvsM, and SvsLM cardinal axes that characterize color coding in the retina and geniculate; Eskew, 2009). Models like decorrelation can account for selective adaptation to multiple directions with only a small number of retunable channels (Atick et al., 1993; Webster & Mollon, 1994; Zaidi & Shapiro, 1993); and color search could similarly involve a flexible reweighting of signals carried only by the three cardinal mechanisms (D'Zmura, 1991; Nagy, Neriani, & Young, 2004). However, a variety of paradigms point to the encoding of color by more than three mechanisms even under a single state of adaptation (e.g., Eskew, 2009). Thus a parsimonious account of our results is that both the color salience and the adaptation reflect multiple mechanisms tuned to different directions in the volume of color space. 

While such results point to a more or less uniform tiling of color space, the search times also reveal a number of asymmetries and interactions between the visual signals along different color directions. First, orthogonal directions were not independent, for search times were elevated even when background elements varied along axes or planes orthogonal to the target (Nagy et al., 2005). Note that this might again reflect the presence of multiple broadly tuned mechanisms, for channels tuned to intermediate directions would then be sensitive and contribute to the response to both the target and the background, even if the target and background are orthogonal (Webster & Mollon, 1994). Second, reaction times were substantially slower for chromatic targets than luminance targets. This difference is well known and is likely to reflect differences in the pathways carrying luminance and chromatic contrast (Bompas & Sumner, 2008; Braithwaite et al., 2010; McKeefry, Parry, & Murray, 2003). Third, when searching on the nonselective color backgrounds there was a consistent difference between the reaction times for S-cone increments and decrements (Nagy & Sanchez, 1990). Finally, when searching on the selective backgrounds we also observed a clear difference in the search times along the two diagonals of the chromatic plane. Again these diagonals have equivalent component contrasts along the LvsM and SvsLM axes, yet the bluish-yellowish pairing behaved as though it had an effectively lower contrast. These effects are intriguing because the blue–yellow axis is thought to be a fundamental dimension for color appearance but typically fails to be revealed in measures of visual performance (Webster, 1996), including measures of search times for displays with uniform-color distractors (Nagy & Sanchez, 1990). However, reduced sensitivity to the bluish-yellowish diagonal has also been observed in some previous studies of color discrimination (Nagy, Eskew, & Boynton, 1987). It has also recently been found in studies of the neural response to color in different cortical areas (Goddard, Mannion, McDonald, Solomon, & Clifford, 2010) and of the effects of color on visual discomfort (Juricevic, Wilkins, & Webster, 2010). Notably, it is also a difference that is apparent in most uniform color spaces, which typically have elongated contours along the bluish-yellowish diagonal when projected into cone-opponent space (McDermott, Juricevic, & Webster, 2009). It is possible that all of these biases are a manifestation of adaptation to natural color distributions, which typically also have a blue–yellow bias (Webster & Mollon, 1997). That is, contrasts along the blue–yellow axis may be less salient because the world varies more along this axis. 

Adaptation and visual salience

We have shown that prior exposure to a color-varying background enhances the salience of novel colors on that background. Effects of this kind appear to match well with subjective experience. Much of what seems to draw our attention in the world are the unexpected or novel properties, and it is not unexpected that this novelty should become more apparent and thus more noticeable as we become more familiar with our environment. However, our results also suggest that at least part of these effects is probably closely linked to processes of sensory adaptation in the visual system (Barlow, 1990b; Ranganath & Rainer, 2003). Evidence for this included: (1) the improvements in search times were found only for novel colors and not for colors within the background distribution; (2) the pattern of changes was consistent with the expected properties of both chromatic adaptation and contrast adaptation; (3) the improvements were found only for backgrounds that varied selectively in color in ways that should differentially affect sensitivity to the background and target colors; and (4) analyses of observers' eye movements did not reveal a change in search strategies between searches on adapted and unadapted backgrounds. Thus adaptation may play an important role in modulating “bottom-up” visual salience by enhancing the salience of novel stimuli relative to the background. This conclusion is also in accord with intuition. We are often unaware of the changes in visual sensitivity as we adapt to a stimulus (e.g., as you stare at a waterfall there is little hint that the response to movement is changing), yet are struck by the aftereffects when we switch to a different stimulus (e.g., the oozing of the rocks to the side). Note that the aftereffect itself is in essence an exaggeration of how the current stimulus differs from the stimuli we have adapted to. Thus the adaptation highlights and arguably draws attention specifically to how a stimulus differs from the current environment. In this sense, the processes of adaptation and attention may be closely linked, and there may be correspondingly close neural links (Barlow, 1997; Rezec, Krekelberg, & Dobkins, 2004). 

While this supports the notion that adaptation may help highlight novel properties of the world (Barlow, 1990b), we emphasize again that this is one of many consequences of adaptation. The changes in appearance induced by adaptation remain the most salient characteristic of visual aftereffects and as noted these appearance changes probably play a fundamental role in perception (Clifford et al., 2007; Webster et al., 2005). Thus adaptation is likely to be at least as important for calibrating appearance as sensitivity or salience. In this regard, it would be misguided to define the purpose of adaptation operationally in terms of a specific outcome, and the very different perceptual consequences of adaptation, from constancy to discrimination, may ultimately all reflect a process designed to optimize visual coding. However, our results are consistent with the idea that enhancing the salience and detection of novelty is one important functional consequence of this optimization. 

Importantly, the changes we observed are to the relative salience of the backgrounds and targets. Adaptation even to a single color axis reduces the perceived contrast of all color directions (Webster & Mollon, 1994), so any enhancements of the target contrast are not absolute. For example, adaptation did not improve search times above those in the no background condition, nor does adaptation to one color direction improve absolute contrast sensitivity along orthogonal directions (Krauskopf, Williams, & Heeley, 1982). The differences are only made manifest when the target is presented in the context of the background, and only when the target differed from the background. Thus they reflect a form of unmasking. These relative—rather than absolute—changes in visual performance may be an important factor in general for understanding how adaptation improves vision and may be one reason why these improvements have been difficult to demonstrate for contrast or pattern-selective adaptation. In this regard, note also that the largest relative changes we observed were when comparing adaptation to the same or orthogonal backgrounds. This is not surprising but is important because this comparison may be closer to the kinds of adaptation shifts that are more likely to occur in natural viewing. Most studies of adaptation compare performance relative to absolute sensitivity under neutral adaptation, typically to a uniform field. However, this zero-contrast baseline is highly unnatural. To understand the consequences of adaptation, it is more relevant to ask how perception and performance varies from one realistic context to another (Webster, 1996). 

Specifically how the adaptation altered visual salience is uncertain. The simplest account is that the changes in salience follow directly from the changes adaptation induces in color appearance (Webster & Mollon, 1994). For example, selective adaptation to the background might reduce its perceived contrast more than the target contrast, so that the target becomes more visible. To assess whether this account is plausible, we estimated the change in equivalent contrast of the targets by fitting Gaussians to the search times for the rectangular lattice of targets in Figure 17. Standard deviations increased by roughly a factor of two for each observer for the target contrasts orthogonal to the adapt axis. This suggests that the effective target-to-background contrast ratio differed by a factor of two between the two adapt conditions, which is roughly consistent with the magnitude of the appearance changes induced by color contrast adaptation (Webster & Mollon, 1994). However, one problem with this explanation is that contrast adaptation tends to have diminishing effects on higher contrast patterns, so that there may be relatively little change in the contrast of the adapting stimulus itself (Georgeson, 1985; Webster & Mollon, 1994). A second issue is whether the adaptation effect on salience reflects changes only in the effective contrast of the target or might also depend on changes in its effective hue angle. In addition to reducing perceived contrast along the adapting axis, adapting to the background axis also rotates the perceived hue of all off-axis targets toward the orthogonal color direction. As a result, the targets may appear more different from the background in hue angle even though the effective color difference (absolute perceived contrast) has decreased. Some role for hue differences in color salience is hinted at by the search results we observed for the rectangular grid of targets. Again these included sets of targets that had a fixed Euclidean distance from the background axis but varied widely in hue, and the search times for these targets tended to be faster for larger hue differences. As seen in Figure 17, Gaussians fit to these axes showed a weak but consistent minimum for the perpendicular hue angles. However, this factor is small relative to the influence of the distance from the target to the background axis. 

A further possibility is that the signals mediating color salience are distinct from the signals underlying color appearance. The neural basis for visual salience remains unclear, and different proposals have been advanced for the cortical locus of saliency signals (Itti & Koch, 2000; Li, 2002; Treisman & Gelade, 1980). Yet regardless of their basis, the stimulus differences defining salience cannot be simply predicted from the appearance differences between targets and distractors (Wolfe & Horowitz, 2004). For example, a color does not emerge from the background until the color difference is much larger than required for discriminating targets and distractors (Nagy & Sanchez, 1990). As a result, it is possible that both backgrounds and targets can change in perceptual salience even if there is no corresponding change in their appearance, and thus that adaptation may modulate functionally distinct neural signals. This is also in accord with the findings that attention can have profound changes on visual salience without producing correspondingly large changes in visual appearance (Blaser, Sperling, & Lu, 1999; Carrasco, Ling, & Read, 2004; Prinzmetal, Nwachuku, Bodanski, Blumenfeld, & Shimizu, 1997), and that it might change the adaptation gain in neurons separately from the contrast gain (Rezec et al., 2004). This dissociation could allow adaptation to fulfill the potentially conflicting demands of regulating both appearance and salience—so that how “noteworthy” a stimulus appears could vary without a corresponding change in what it “looks” like. 

While we focused on color, comparable effects of adaptation on salience might be expected along many perceptual dimensions. For example, backgrounds that have a bias in orientation or motion direction produce analogous selective aftereffects in these patterns (Clifford, Wenderoth, & Spehar, 2000). In fact, adaptation to vertical gratings may decrease the discrimination thresholds for gratings tilted to either side of vertical, consistent with an increased sensitivity to the orthogonal axis (Regan & Beverley, 1985). One difference between these dimensions is in the tuning of the underlying mechanisms. For example, color contrast or motion adaptation can induce much larger angular changes in perceived hue or motion direction compared to tilt aftereffects, and this is likely because the tuning functions are broader (Clifford et al., 2000; Schrater & Simoncelli, 1998; Webster & Mollon, 1994). It is thus possible that the effects of adaptation on salience might be more pronounced for a dimension like color or motion than orientation or size. Comparisons of the aftereffects on search within these different dimensions might thus provide further clues to the extent to which the performance changes could be tied to expected characteristics of visual coding and pattern adaptation. 

For color, we also showed that adaptation can affect visual salience when observers are adapted to the color characteristics of natural scenes. For the two distributions we tested, enhancements were found for the arid scene that had a strongly biased color gamut but not for the less selective colors from the forest scene. This is again consistent with an actual adaptation effect as the basis for the performance changes and is also again at least qualitatively consistent with the changes in color appearance induced by adaptation to these distributions (Webster & Mollon, 1997). An important implication of these results is that the natural world varies enough and in ways that can substantially alter the states of visual adaptation. Here we have shown that one consequence of this adaptation is to increase the salience of the stimuli we are not adapted to. To the extent that adaptation is routinely engaged in natural viewing, much of what we notice about the world may be a visual aftereffect (Webster, 2004). 

Acknowledgments

This work was supported by NIH EY-10834 and a Nevada NASA EPSCoR Collaborative Pilot Grant. 

Commercial relationships: none. 

Corresponding author: Michael A. Webster. 

Email: [e-mail]. 

Address: Department of Psychology/296, University of Nevada, Reno, Reno, NV 89557, USA.