Attention to locations and features: Different top-down modulation of detector weights

Abstract

It is well known that attention improves the visibility of a target. In this study, we examined the effect of attention on the selectivity profile for a target. We used a masking technique to measure the tuning function for detecting a target while cueing either its orientation or its location. In the presence of an orientation mask, uncued thresholds were maximally elevated with a parallel mask and decreased with increasing mask orientation from the target. The presence of a cue reduced the masking effect but the shape of the function was cue-specific: The orientation cue consistently improved thresholds at the target orientation, whereas the location cue typically improved thresholds at all orientations relative to the function measured in the absence of attention. The selective versus overall increase of sensitivity observed in our study may be due to differences in the weighting of individual detectors that determine the behavioral tuning function in the two cueing conditions.

Introduction

The sensory effect of a stimulus is not simply determined by a cascade of noisy, feed-forward filters. Rather, the response to the stimulus is modulated by different mechanisms that increase or decrease the resulting strength of the neuronal and perceptual response. Among these mechanisms, attention is one that has attracted philosophers and scientists for a long time (James, 1890/1950). It is well known that perceptual judgment is distracted by concurrent information and that cueing some aspect of a stimulus enhances the quality of its perception, with measurable changes in its appearance (Carrasco, Ling, & Read, 2004). 

These behavioral observations have found physiological counterparts in studies where the discharge rate of visual neurons was recorded under different attentional states in awake animals. Attentional modulation seems to occur in neurons in several early visual areas, and the amount of modulation grows with the level of processing (Maunsell & Cook, 2002). Whereas V1 neurons show consistent but weak enhancement of their average activity (about 8%, McAdams & Manusell, 1999), MT or V4 neurons are enhanced by 25% of their discharge rate (Treue & Maunsell, 1996) and VIP and MST neurons by 50% or more (Cook & Maunsell, 2002; Ferrera, Rudolph, & Maunsell, 1994). 

The majority of the single neuron data is consistent with attention increasing the overall activity of individual neurons. The increased response without a change in the selectivity, or tuning width, of the neuron is referred to as a gain change. The analogous effect in psychophysics is called enhancement (Bashinski & Bacharach, 1980; Cameron, Tai, & Carrasco, 2002; Carrasco, Penpeci-Talgar, & Eckstein, 2000; Eriksen & Hoffman, 1974; Lu & Dosher, 1998; Posner, 1980). To facilitate the comparison between psychophysics and physiology, the term detector will refer to the single neuron level and the term filter will refer to the population of neurons that contributes to the perceptual tuning functions. 

A number of psychophysical studies have focused on low-level measures, such as contrast sensitivity, to study the behavioral effects of directing attention to a stimulus. In particular, an exhaustive series of studies by Lu and Dosher (1998, 2000a, 2000b) explored various possible mechanisms of attention. They measured contrast thresholds in external noise and used their perceptual template model to predict a unique signature for each attention mechanism, including stimulus enhancement and external noise exclusion. Noise exclusion implies that the shape of the template becomes more selective to filter out external noise. They measured thresholds for detecting a stimulus in the absence of a cue, and when a precue directed attention to or away from the stimulus. They then compared the shape of the contrast thresholds versus external noise function under different cueing conditions to the predicted signatures for each attention mechanism. External noise exclusion, which can be mapped into a change of selectivity, was found to be the primary mechanism under conditions of central (vs. peripheral) precueing and high external noise. Stimulus enhancement seemed to be a secondary mechanism that plays a role in noiseless conditions with peripheral cues. A study by Lee, Itti, Koch, and Braun (1999) that used a discrimination task suggests that both stimulus enhancement and increased selectivity operate under conditions of full attention compared to the case when attention is divided between the discrimination task and a distracting task. In the spatial frequency and in the orientation domain, their data are better fit by a model that has both a higher gain and a more selective filter in the “fully attended” condition than in the “poorly attended” condition. In a different study, Carrasco and colleagues (2000) measured the contrast sensitivity function under different conditions of cueing, uncertainty, and task (detection vs. discrimination). Most of their results were incompatible with increased filter selectivity and were taken to support enhancement of the signal arising from attended locations. A recent study (Eckstein, Shimozaki, & Abbey, 2002) used a psychophysical reverse correlation technique to visualize the filter used by an observer in a Posner-type cueing paradigm (Posner, 1980). The shapes of the perceptual filters at the attended and unattended locations are consistent with a model that weights the information at the cued location according to the validity of the cue, but requires no other change in the visual filter at each location. Using a critical-band-masking paradigm, Talgar, Pelli, and Carrasco (2004) obtained a similar result. They showed that attentional modulation increased the sensitivity of the spatial frequency filter mediating letter identification without affecting its bandwidth. In summary, it appears that psychophysical studies to date do not provide unequivocal information about the mechanism of attention. 

Following a tradition started with Posner (1980), experimental studies of selective visual attention have used spatial pointers of various kinds to cue the location where the relevant stimuli are displayed. Thus, directing attention toward competing locations is the most widely used method for studying the effects of selection, from reaction time studies with human observers to single neuron recording in animals. 

However, the spatial domain is not the only visual domain that can afford selection. Indeed, in natural contexts, competing stimuli may share the same location but differ in their relative distributions of certain key features. Empirical studies have shown that attention selects dimensions of a visual image other than space, such as the occurrence of a specific feature within a dimension. For example, attending to the direction of motion modulates the responses of neurons in monkey area MT that are selective to that direction. The “feature-similarity gain model” proposes that this modulation depends on the similarity between the attended direction and the cell’s preferred direction (Martinez Trujillo & Treue, 2004; Treue & Martinez Trujillo, 1999). Specifically, these authors show that this modulation multiplicatively increases the response of the neuron when the attended direction matches the preferred direction of the neuron and decreases its response when the attended direction corresponds to the neuron’s anti-preferred direction. Functional imaging studies in humans also show that attending to a particular direction of motion at one location increases the response of early visual areas to the same stimulus in an unattended location (Saenz, Buracas, & Boynton, 2002). Moreover, by summoning attention to features, in particular to one of two possible directions of moving dots, Corbetta and Shulman (2002) suggest that overlapping brain regions are involved in attention to locations and attention to features. Another recent study has shown that observers can track the temporal dynamics of a feature changing over time in one of two superimposed stimuli (Blaser, Pylyshyn, & Holcombe, 2000), suggesting a role for attention in selecting within and/or between features at the same spatial location. In a study on the visibility of a smooth motion trajectory in dynamic noise, Verghese and McKee (2002) showed that the motion trajectory itself acts as an implicit cue that draws attention to similar directions of motion in the vicinity of the trajectory. 

In the present study, we use a masking paradigm to investigate the nature of attention mechanisms. In a fully crossed design, we studied the effects of both spatial and feature cues on the tuning function for both space and orientation. The psychophysical use of masking paradigms (Legge & Foley, 1980) reveals the shape of the psychophysical filter used to detect a given stimulus. Typically, masking is greatest when the mask is at the same location or has the same feature value as the stimulus, and decreases with increasing separation between mask and stimulus. We reasoned that once the shape of the masking tuning function was obtained under neutral conditions, any effect of the cue would be reflected as a change from this baseline function. This experimental design allowed us to assess the effect of attention to both spatial location and to a specific feature value. If indeed attention adds flexibility to our sensory and cognitive systems, then we should be able to direct attention to space, features, or objects. Here we examine what mechanisms are employed when attention is directed to locations and to features. 

Methods

The task in all of the experiments in this study was to detect a test patch, typically in the presence of a mask. The stimulus could occur in one of two locations, at an eccentricity of 7° on the right or on the left of the fixation point and could assume one of two orthogonal orientations, vertical or horizontal. Each test location had two mask patches at orthogonal orientations (except for the symmetric mask experiment; see later) displayed simultaneously with the test. In the orientation-masking condition (Figure 1, left panels), we varied the orientation of the mask pair with respect to the test, while keeping mask location fixed, overlapping the target. In the location-masking condition (Figure 1, right panels), we varied the location of the mask components with respect to the target location, while keeping their orientations fixed (one of the mask orientations matched the test orientation; the other was orthogonal to it). For both conditions, we modulated attention by using two different cues: a location cue (Figure 1, middle row) that signaled the target location and an orientation cue (Figure 1, bottom row) that signaled the target orientation. Both the spatial and orientation cues were 100% valid. A neutral-cue condition (Figure 1, top row) with no cues was added for comparison. All experiments were run in a temporal two-interval forced-choice (2IFC) paradigm. Each interval displayed the cues for 106 ms (79 for observer S3), followed by the stimuli for 26 ms. The central cue control experiment had different temporal durations (see later). The cues stayed on while the signal was displayed to avoid masking due to offset transients. Then both the stimuli and the cues disappeared. 

Figure 1

 

Stimuli and conditions. The left and right panels display the stimulus configuration for the orientation-masking and the location-masking experiments, respectively. Each row represents a different cueing condition: neutral cue (top), location cue (middle), and orientation cue (bottom). For simplicity each panel shows the test stimulus + mask on the right side of fixation and the mask alone on the left side of fixation. In the experiment, the test could occur either on the left or right, and either in the first or second of two temporal intervals. For the orientation-masking experiment, the left panel shows examples of mask angles of 0° (top), 14° (middle), and 45° (bottom). For the location-masking experiment, the right panel shows examples of mask distances of 0.5° (top), 2° (middle), and 8° (bottom). Because the spatial arrangement of the mask differed in the two masking conditions, we modified the spatial configuration of the cue, so that it did not overlap the stimulus.

In the orientation-masking condition, the neutral cue was a square outline that preceded the stimulus at each of the two locations (Figure 1, top left). In the location-cue condition, only one box was displayed (Figure 1, middle left). It appeared at the location of the test in the signal interval and in one of the two locations selected randomly in the noise interval. In the orientation-cue condition, the cue appeared in both locations. The cue was made up of the sides of the square that matched the test orientation in the signal interval and the horizontal or the vertical sides selected randomly in the noise interval (Figure 1, bottom left). The superimposed mask was made up of two orthogonal mask components that were rotated with respect to the test orientation. The two mask components were presented simultaneously with the test. The left panel of Figure 1 shows mask angles of 0°, 14°, and 45°, in the top, middle, and bottom rows, respectively. Detection of the test was measured as a function of the tilt of the closer mask component to the test orientation. The observer’s task was always to report which of the two temporal intervals contained the test patch. 

The location-masking experiment followed the same basic design except for two main differences relating to the placement of the masks and the spatial configuration of the cues. We used two mask components as before, but fixed their angle at 0° and 90° with respect to the test. The position of the two components at each location was varied in λ units (λ is a full period of the carrier grating) along an isoeccentric circle of radius equal to 7° (Figure 1, right panels). The test location was kept fixed along the horizontal meridian passing through the fixation point. As for the cue, we changed its appearance relative to the previous experiment because a square box would overlap with the mask as it moved away from the test. Instead we displayed a rotated T-like cue at an eccentricity closer than the stimuli (5.5°), as depicted in the right panels of Figure 1. Pilot experiments showed that the cueing effect was stronger for cues inside than outside the isoeccentric circle. For the location cue, we displayed the whole T on one side (Figure 1, middle right), and for the orientation cue we displayed the bar that matched the test orientation on both sides of fixation (Figure 1, bottom right). 

The target patches were sinusoidal gratings at 2 cycles/deg windowed by a Gaussian profile with a space constant equal to 0.5°, resulting in odd-symmetric Gabor patches. Each mask patch was made up of black and white stripes of random width (1-D noise or “barcode” noise) whose high-frequency content above 12 c/deg was filtered out, resulting in slightly blurred bars. 

Apparatus

We designed our stimuli using the Psychophysics Toolbox for Macintosh (Brainard, 1997). Stimuli were displayed on a 15″ Sony Trinitron with a mean luminance of 39 cd/m² and a refresh rate of 70 Hz. The contrast resolution was 8 bits and each mask component had a contrast of 19%. We initially used dithering to achieve contrasts of less than 1% by displaying alternate lines of a stimulus while keeping the others at mean luminance. Subsequent versions of the Psychophysics Toolbox allowed an overall contrast depth of 10 bits, 8 of which could be used within a single frame. Through this change we gained contrast resolution (with minimum contrast being about 0.3%) at the expense of contrast range. As the overall usable contrast range was now only 40%, we had to reduce the contrast of each mask component to about 9.8% so that about 20% could be left for the target stimulus. 

Procedure

Detection of the target was measured using a 2IFC procedure where the test was displayed in one of two temporal intervals. The observer’s task was to simply report which interval contained the target. The test could be either on the left or right of fixation and could be either vertical or horizontal. Observers knew that the test could vary in location and in orientation. The cue when present was 100% valid in the signal interval, while it was randomly selected in the noise interval. Signal and noise intervals occurred in random order within each trial. 

Each interval was preceded by a tone, and at the end of a trial, acoustic feedback for errors was provided. Observers responded at the end of each interval by pressing the left or right button of a two-button mouse, and the response to a given trial triggered the following trial. Each cueing condition was measured in consecutive, blocked sessions to ensure that observers used a consistent strategy within a cueing condition. We reasoned that interleaving cueing conditions might lead the observers to use some sort of “average” strategy that would hide the effect of specific cues. The order of cueing conditions was randomized across observers. Before each measurement session, practice sessions were given. In the first two experiments (pure detection and orientation tuning), we used a fixed number of contrast values interleaved within a block of trials. These values were chosen to be in the range of contrast detection data from previous studies. For each of six contrast values, we measured the accuracy (proportion correct) for at least 100 trials to achieve a psychometric function, resulting in at least 600 trials for each condition. We fitted the psychometric functions with a Gaussian function and set the thresholds to a criterion of 75% correct responses. To assess the statistical significance of our data, we used a bootstrap procedure to estimate the standard deviation of the thresholds. 

For the other experiments we used the QUEST procedure (Watson & Pelli, 1983), which adjusted the contrast on each trial to find the maximum likelihood estimate of threshold. This procedure requires fewer trials (about 150 for the whole psychometric function). Threshold estimates obtained with this adaptive method were consistent with those obtained with fixed contrast levels. 

Observers

Five observers participated in our experiments. All had normal or corrected-to-normal vision. Two observers were authors (S1 and S2), and the others (S3, S4, and S5) were unaware of the purpose of the experiments. We collected a partial set of data from another observer that confirmed the trend shown here. 

Results

Detection of the target without masks

In the first set of measurements, we tested the effect of different cues in the absence of a mask. Even though this is not the main focus of our study, unmasked detection thresholds provide the necessary baseline to compare the effect of different cues in the presence of a mask. In addition, these data provide a control for sensory interference of the cues with the test stimulus. We reasoned that any interference due to a relatively high luminance cue could elevate detection threshold compared to other studies in the literature. The test stimulus appeared in one of two locations, was either horizontal or vertical, and was present in one of the intervals in our 2IFC task. Observers were asked to choose the interval with the test patch. Unmasked detection thresholds were measured with a neutral cue, a location cue, and an orientation cue (see Figure 1). The cues were the same as those used in the orientation-masking condition (see Methods). 

Based on previous studies using similar conditions (Foley & Schwarz, 1998; Solomon, Lavie, & Morgan, 1997), we did not expect a substantial effect of the cue when the contrast of our masks was set to zero. Indeed, detection thresholds were substantially unchanged among the different cueing conditions for each observer (Figure 2). Moreover, threshold contrast ranged between 3.5% and 5%, consistent with the previous data at that eccentricity (Virsu & Rovamo, 1979). The lack of a cueing effect could be because attention is not needed or deployed for unmasked detection. However, it is also possible that at detection threshold, observers act as if extremely uncertain of the spatial location and the temporal onset of the stimulus (Pelli 1985). The cue might indeed reduce stimulus uncertainty, but under these conditions of high intrinsic uncertainty, the reduction in uncertainty by a factor of 2 is too small to be reliably measured from the psychometric function slopes. 

Figure 2

 

Unmasked detection thresholds for three observers. We use the same color code for all figures: the neutral-cue condition is plotted in blue, the location-cue condition in red, and the orientation-cue condition in green. In all cases, thresholds did not change significantly across cueing conditions, ranging from about 3.5% to 5% contrast.

Tuning function for orientation

In this experiment, we estimated the tuning function for detecting our test stimuli in the presence of overlapping masks of varying orientation. We reasoned that by systematically tilting the mask from the test in the neutral-cue condition, thresholds would improve as the mask orientation deviates from the test orientation, reducing the detrimental effect of the mask. The shape of this function would reflect the orientation selectivity of the psychophysical filter selective for our test stimulus. Thus, by measuring this function under different cueing conditions and by comparing its shape relative to the basic function obtained in the neutral condition, we should be able to directly observe the effect of attention on a psychophysical filter. Stimuli and task are described in the left panels of Figure 1 and in the Methods section. Our masking experiment was atypical in two ways. First, we fixed the test orientation and moved the mask orientation away from it rather than the converse, which is more standard (Campbell & Robson, 1968). Second, and more importantly, we used two orthogonally oriented mask components rather than only one. We did this because in the pilot study observers reported that when the mask angle deviated from the test they sometimes detected the test based on the “plaid-like” appearance of test and mask together. To avoid artifacts that could arise from the use of a plaid cue, we used a pair of orthogonal masks that always formed a plaid, as shown in Figure 1 (the mask-alone stimuli are displayed on the left of fixation). The orientation difference between the closer mask component and the test was taken to be the mask angle. This angle varied between 0° and 45°, which was sufficient to measure a complete orientation tuning function. 

Thresholds for detecting the test as a function of mask angle in the neutral-cue condition are reported in the left column of Figure 3. Thresholds decrease (sensitivity increases) with increasing tilt of the mask away from the test for all four observers. Sensitivity improves significantly, ranging from 25% for S1 to about 50% for the other three observers. In all cases thresholds decrease quickly with increasing tilt of the mask, reaching their lowest value at mask angles of about 15°. The trend of the function is well captured by a simple Gaussian fit to the data (see 1). This kind of fit, a standard for assessing tuning functions both physiologically (Enroth-Cugell & Robson, 1966; Treue & Maunsell, 1996) and psychophysically (Campbell & Kulikowski, 1966), describes the results with an r² always higher than .95. More importantly, the orientation tuning functions are consistently shaped across observers and provide a baseline for comparison with the data in the other two cueing conditions. There are two main points to note here. First, the width (σ_neutral) of the function is narrow, around 6° instead of the 15–25° range reported for orientation-tuned neurons in areas V1 and V4 (DeValois, Albrecht, & Thorell, 1982; McAdams & Maunsell, 1999). Second, the functions reach a floor at contrasts higher than the detection threshold for unmasked stimuli. We have reasons to think that both effects are due to the presence of a second orthogonal mask component. We will come back to this point in the discussion of the symmetric mask experiment. Observer S4’s data were collected after we switched to the new version of the Toolbox, which allowed greater contrast resolution at the cost of decreased contrast range. Consequently, the mask components had a contrast of 9.8%, about half the contrast used for the other three observers. This explains why her masked thresholds are lower than that of the other observers. 

Figure 3

 

Orientation tuning functions under three cueing conditions. Each horizontal row shows data for each of four observers. Each panel plots contrast thresholds as a function of the mask angle with respect to the test. Panels in the leftmost column report data and fits for the neutral-cue condition (in blue). The error bars in all panels represent the standard error of the threshold estimate calculated using a bootstrap procedure. The blue squares represent thresholds for the mask angles used in this experiment. For all observers thresholds are maximally elevated when test and mask angles coincide, while the effect decreases as the mask angle deviates from the test. The smooth line is the Gaussian fit through the data using Equation 1 (see 1). The middle column shows the effect of a location cue on thresholds (in red). The circles are the thresholds for this condition, and the red smooth line is a difference-of-Gaussian (DoG) fit to the data for all observers except S4, whose data are better fit by a simple Gaussian. The blue line is the Gaussian fit to the neutral-cue data, and the error bars for the smallest mask tilt are shown for comparison. For all subjects except S4, the location cue reduces thresholds by an amount roughly proportional to threshold elevation with the neutral cue. The right column reports the data and fits for the orientation-cue condition (in green). The triangles are the thresholds while the green smooth line is the DoG fit through the data. The blue line is the fit of the neutral-cue tuning function. Here the cue is most effective when the test and mask orientation coincide, with significant differences at the smallest mask angles for S1, S2, and S3, and at the two smallest angles for S4. For larger orientation differences, the benefit of the orientation cue is reduced in all observers except S1, so that thresholds are closer to the neutral-cue condition.

The central and the right columns of Figure 3 show the tuning functions obtained in the location- and the orientation-cue condition, respectively. For comparison, the fit of the neutral- cue condition is replotted for each observer. To visualize the effect of the cues in the two conditions, we fit these functions by subtracting a new “attention” Gaussian function from the baseline Gaussian that fits the neutral-cue data for each observer (1). We need to stress at this point that this difference-of-Gaussian operation serves only to describe the data and does not imply any modeling of the underlying processes. Indeed, both the location- (middle column) and the orientation-cue (right column) functions are different from the baseline function obtained in the neutral-cue condition and show some benefit of cueing the test. Furthermore, the location and orientation cues show very specific effects across all observers. When we cued the location of the test, the benefit of cueing was roughly proportional to threshold elevation in three of four observers. Observer S4 did not show any substantial effect of the location cue. When instead we cued the orientation of the test without providing any information about its location, the cueing effect was non-uniform across the tuning function. There was a large, significant effect for mask angles very close to the test orientation with a much smaller effect for more tilted mask orientations. This resulted in a sharp dip in the tuning functions at mask orientations close to the test orientation. 

What we observe here is a potential dissociation between the effect of a location and an orientation cue on the orientation tuning function. The location-cue condition shows facilitation across a large part of the tuning function, whereas the orientation-cue condition shows a sharp notch of facilitation around the test orientation. However, an alternative description of the tuning function with the orientation cue is that the peak is shifted away from the test orientation. This peak shift might be because observers detected our stimuli using orientation filters (channels) tuned away from the test orientation, known as “off-channel looking” (Blake & Holopigian, 1985). In this case, the presence of the mask causes a shift in the filter monitored, from one tuned to the test to one with slightly different preferred orientation. Even though the dissociation between the two cueing conditions is already a control for this effect, we wanted to test this possibility directly. 

Symmetric mask experiment

The aim of this experiment was to control for possible stimulus configuration effects due to the particular mask used in the previous experiment. Moreover, we wanted to check that the central dip obtained in the tuning function for the orientation-cue condition was not because the observer was monitoring filters tilted away from the test orientation. To reduce the efficacy of such a strategy, we replaced the orthogonal mask components with two mask components that were tilted symmetrically with respect to the test, as sketched in Figure 4a. To avoid the possibility that the suprathreshold masks cued the test orientation, we set half the trials to be “catch trials,” where the average mask angle was rotated by 90° with respect to the test. We compared the neutral cue with the orientation-cue condition in two observers, S1 and S2. If off-channel looking is the cause of the central dip and peak shift in the orientation-cue condition, then the use of two symmetric mask orientations that flank the test orientation would make the strategy of monitoring an oblique detector inefficient, moving the peak of the tuning function back to its unbiased position (0°). Figure 4b shows that orientation cueing is unlikely to be mediated by off-channel looking. When test and mask orientation coincide or are very close (Figure 4b, bottom panel), the orientation cue causes a reduction in threshold relative to the neutral-cue condition. Even though the effect is not as statistically robust as in the previous experiment, mostly due to the subjective difficulty of the task, it is consistent for both observers. Moreover, observer S2 shows a significant benefit of the orientation cue even at a mask angle of 1.9°. In general, thresholds for the two smallest mask tilts are significantly lower than those with masks tilted 6° from the test. When the masks are tilted by ±6°, there is little benefit of the orientation cue. However, when the angle of the mask deviates more than 6° from that of the test, the benefit of cueing is clearly visible. In other words, under this new stimulus configuration, there is both a preferential gain at the cued orientation in addition to a more diffuse gain across orientation (similar to the case for location). 

Figure 4

 

Stimuli and data for the symmetric mask experiment. a. Examples of mask components at three different orientations. The left column shows the mask components alone, while the right column shows the mask plus a vertical test. Rather than being made up of two orthogonal components, the two mask components had symmetric orientations with respect to the test. b. Tuning functions for observer S1 (top panel) and S2 (bottom panel) as a function mask orientation. The blue squares are thresholds for the neutral-cue condition, and the green triangles are thresholds for the orientation-cue condition. The error bars represent the standard error of the threshold estimate calculated using a bootstrap procedure. The benefit of the cue at mask angles close to the test argues against “off-channel” looking mediating the effect of the orientation cue.

This experiment also shows that when the orthogonal component of the mask was removed, the two anomalies in the tuning functions measured in the previous experiment were removed. In particular, the tuning function for the neutral-cue condition shows a width of about 20°, which matches more closely the selectivity profile of orientation-tuned neurons reported previously (Blakemore & Campbell, 1969; DeValois et al., 1982; McAdams & Maunsell, 1999). Moreover, thresholds keep improving down to levels close to the unmasked detection thresholds, well below the asymptote of about 10% contrast that we found before for these two observers. 

So far, we have found a diffuse effect of a spatial cue and a sharp and highly selective effect of a feature cue over the tuning function for orientation measured by masking. Both the dissociation between the two cueing conditions and the symmetric mask experiment show that this is unlikely to be an artifact of off-channel looking, suggesting that the cues have different effects in the presence of a spatially superimposed mask. Before we sketch any interpretation to our data, a key question remains: Is the notch found with an orientation cue-specific to the dimension (i.e., is there a specific interaction between the orientation cue and the orientation modulation of the mask), or is it specific to the cue (an orientation cue relative to a spatial cue)? If the former is true, then switching the dimension over which the masking occurs should reverse the effect: Masking over spatial locations should produce a notch for the location cue and a wide and nonspecific effect for the orientation cue. If instead the effect is cue-specific, then we should observe the same pattern (as in Figure 3) for the tuning function for location. The following experiments investigate this issue. 

Tuning function for location

In the following set of experiments, we repeated the measurements done in the first experiment but instead of changing the orientation of the masks while keeping their location fixed at the test location, we kept their orientation fixed while we varied their location relative to the test. One mask component was always horizontal, while the other was vertical, and the pair of masks was moved symmetrically away from the test along an imaginary circle of radius 7°, centered on fixation. Recall that the test orientation could be either horizontal or vertical. The basic mechanisms of lateral masking are different from those of pattern masking for overlapping stimuli, an issue that is still under debate (Carandini, Heeger, & Senn, 2002). However, we reasoned that in the absence of a cue, varying the location of the two masks with respect to the test should reveal a function where threshold elevation is maximal for locations at or near the test location, and comes down to the level of unmasked detection for mask distances very far from the test. We expect this function to have a shape similar to that obtained in the orientation domain and to be affected by our cues showing a reduction of the masking effect on thresholds. If the narrowing observed in the orientation-masking function with an orientation cue is simply due to dimension-specific interactions between cue and mask, then we should expect to reverse the two cueing effects found previously. Such a result would support an interaction between the nature of the stimulus and the cue used to summon selective attention. If on the other hand the benefit of cueing depends on the specific cues used, then we may need to reconsider previous results in this light. 

The results of this experiment for three observers are summarized in Figure 5 in exactly the same form used for the orientation-masking experiment. First, in the neutral-cue condition we were able to obtain a clear tuning function by moving the location rather than the orientation of the masks. This function was highly consistent across observers, and was well described by the Gaussian fitting procedure used previously. Thresholds were maximally elevated when the masks were at the same location as the test, but improved rapidly with increasing mask distance. This pattern of thresholds resulted in a location tuning function with a width slightly larger than 1λ in all the observers. The functions reached their asymptote at contrast values close to the unmasked detection thresholds. Again, cueing either the location or the orientation of the test was beneficial. Moreover, the cueing effect followed a pattern similar to that of the orientation-masking experiment: Cueing the location caused a slight improvement along the whole tuning function, whereas cueing the orientation showed a benefit only within a very small range of mask locations overlapping or very close to the test location. 

Figure 5

 

Location tuning functions for three cueing conditions. Panel arrangement, color code, and symbols for conditions follow the same pattern of Figure 3. Each panel of the first three columns plots contrast thresholds as a function of the distance of the mask from the test (in λ units). In the neutral-cue condition, in blue, threshold elevation falls rapidly with distance. The location-cue thresholds (in red) are lower than the neutral-cue thresholds for mask distances up to 4 λ. Thresholds for the orientation-cue conditions are lower than the neutral- and the location-cue conditions when test and masks overlap or when they are very close to each other, but any facilitation of the cue disappears completely across observers at distances of about 1 λ. The reliability of the small dip obtained with the orientation cue is also supported by the fact that threshold estimates at the peak (λ = 0) are significantly different in two of the three observers. This pattern confirms the previous experiment and suggests a cue-specific effect independent of the nature of the mask.

In summary, in this experiment we measured the spatial selectivity profile of the psychophysical filter detecting the test and found that the cueing effect shown in the orientation tuning function was reproduced with exactly the same pattern. The location cue improved detection of the test signal at almost all mask distances that showed any masking effect, whereas the orientation cue improved thresholds only for mask locations that overlapped or were very close to the test. 

Central cue experiment

The final control experiment addresses the issue that the orientation cueing effect occurs due to sensory interactions between the orientation cue and the test. The orientation cue might act as a flanker that facilitates detection of a parallel test but has little effect on an orthogonal test. The same rationale does not apply to the location cue because it has two orthogonal orientations. Our results so far argue against such sensory interactions because the orientation cueing effect occurs with the different arrangements of cue and test that we used in the two experiments. Nevertheless, we decided to test directly for the possibility of sensory interactions. We moved the cue away from the peripheral stimuli, bringing it to fixation 7° (14 λ) away from the test. The cue now appeared 500 ms before the stimuli to allow the additional time required for a central cue to be effective (Cheal, Lyon, & Hubbard, 1991). There is no benefit of moving the eyes as the central cue signals only the orientation of the test and not its location. The cue was a cross made up of a horizontal and vertical line. While the whole cross was shown in the neutral-cue condition, only the line matching the test orientation was shown in the orientation-cue condition. 

The data in Figure 6 show clearly that the dip obtained using an orientation cue throughout our study is not an artifact due to low-level sensory interaction between cue and stimuli. A dip in the orientation-cue condition appeared consistently and significantly in all the observers even when the cue appeared 7° away from the test. 

Figure 6

 

Data for the central cue control experiment for three observers. The blue squares are thresholds for the neutral-cue condition, and the green triangles are thresholds for the orientation-cue condition. The error bars represent the standard error of the threshold estimate calculated using a bootstrap procedure. Thresholds for the 0° condition are significantly lower in the presence of an orientation cue for all three observers, a confirmation that the effect of the orientation cue is reliable and that it is not due to sensory interaction between cue and test.

Discussion

In this study, we used the technique of pattern masking to directly assess the effect of different cues on the tuning characteristics of the psychophysical filter used to detect a stimulus. In the absence of a mask, cueing had no effect on the detection of the test. In the presence of a mask, we obtained a basic tuning function for the neutral-cue condition, in which observers were uncertain about the location and orientation. This function, measured for various mask values along both the orientation and the location dimensions, defined the effect of a mask on the detection of a test stimulus. In both space and orientation, the uncued masking effect was greatest when the mask value matched that of the test, and decreased as the mask deviated from the test. The window over which the masking occurs is thought to mirror the sensitivity profile of the filter that detects the stimulus. This is also the baseline condition corresponding to maximum uncertainty. We then directed attention to either the location or the orientation of the test to observe whether the cues affected the basic selectivity of the filter. The resulting tuning functions assumed two characteristic trends for all the conditions tested. Cueing the location of the test produced a wide, nonspecific effect on the tuning function. In comparison, cueing the orientation of the test produced a highly selective effect at angles and locations close to the test. Both these effects occurred across conditions, irrespective of whether the masks were modulated in location or in orientation. Furthermore, we excluded the possibility that the narrow effect of the orientation cue was due to off-channel looking. Although the notch of the tuning function with an orientation cue is surprisingly selective, it is consistently found throughout our conditions. The notch has also been observed in an adaptation experiment by Dao, Lu, and Dosher (2004) that has many features in common with our paradigm. Two other studies also show this very selective benefit in the presence of superimposed masks. Saarinen and Levi (1995) measured the effect of learning on vernier acuity and found that after learning, vernier acuity showed the largest improvement at mask orientations close to the orientation of the vernier stimulus. Zenger and Sagi (1996) obtained a similar pattern of results in an orientation-masking study. They used a pair of superimposed masks whose orientation varied symmetrically about the test (similar to our symmetric mask experiment) and found a large decrease in thresholds when mask orientations coincided with the test. Because there was no uncertainty about test orientation, these conditions are analogous to our experiment where we cued one of two test orientations. Presumably observers in their study were attending to the single vertical test orientation. 

The masking paradigm we used is a simpler and more direct method to evaluate the effect of cueing than the model-driven approach of Lu and Dosher (1998). In some aspects, the study by Lee et al. provides similar results (see Figure 2d in Lee et al., 1999), but under different conditions. Furthermore, because we measure tuning functions, our data may relate more directly to the physiological studies that have measured the effect of attention on the response function of individual neurons. Through the masking technique and the use of a fully factorial design, we have evidence for two cue-specific mechanisms of attention rather than different mechanisms depending on the exact nature of the task or on the external noise level. 

Selection mechanisms

The following discussion will examine whether our results disentangle the current debate about the mechanisms of visual selective attention. First, why was neither cue effective in the pure detection condition, when the mask contrast was zero? Logically, there are two possible alternatives: that selective attention is not recruited or that our measure is not sensitive enough to detect its effects. While selection makes sense when other stimuli are present, this may not have been the case when there was no mask in a competing location or orientation. We think it more likely that the null result for the pure detection task is due to intrinsic uncertainty—observers monitor so many detectors across the location and the orientation dimensions that the reduction in uncertainty by a factor of two provides little benefit. If this is the case, then the small improvement in thresholds due to a two-fold reduction in stimulus uncertainty could well be within the error of measurement. 

But can uncertainty be the cause for the cueing effects observed under suprathreshold conditions in the presence of a mask? There are two possible locations in each interval and two possible orientations. If the location and orientation dimensions are independent, this amounts to an uncertainty of 4 in each interval. Either the location cue or the orientation cue reduces this uncertainty by a factor of 2. The predicted decrease in thresholds caused by this twofold reduction in uncertainty is about 20%, based on the expressions for uncertainty outlined in Verghese and Stone (1995). Therefore, the halving of uncertainty should result in a 20% decrease in thresholds across all mask angles and locations. Consider the data of Figure 3. There is a decrease in thresholds with both the location and the orientation cue, but it is not simply a proportional decrease in thresholds across all mask angles. Such a decrease would have resulted in data that are simply shifted down vertically on the log axis for threshold. However, the observed cueing effect differs in the two masking functions, showing a greater and more specific effect in the orientation domain. Moreover, the differences of thresholds between the neutral cue and the two cueing conditions at the baseline, where the orientation/location of the mask is widely separated from the test, are typically smaller than that predicted by uncertainty reduction. 

If uncertainty cannot explain the cueing effects, then what are the mechanisms responsible for the weak but diffuse facilitation found when cueing the location, and of the stronger, highly selective facilitation when cueing the orientation of the test? At first glance, the best candidate mechanism for the location cueing effect would be the enhancement in sensitivity of the filter detecting the test, also known as the “signal enhancement” hypothesis of attention (Bashinski & Bacharach, 1980; Carrasco et al., 2000; Eriksen & Hoffman, 1974; Lu & Dosher, 1998; McAdams, 1999; Reynolds, Pasternak, & Desimone, 2000; Treue & Maunsell, 1996). This would cause an increased response along the whole tuning curve. Because we measured psychophysical tuning curves, we can compare our data to tuning curves obtained from single neuron recordings. If signal enhancement is indeed the underlying mechanism, then our location cueing results are consistent with a number of previous findings from both physiology and psychophysics. Signal enhancement could be the mechanism triggered by the cues to bias the decision among competing locations (Reynolds, Chelazzi, & Desimone, 1999). In our case this occurs in the presence of high noise and with task demands completely similar to those producing bandwidth narrowing (Lu & Dosher, 1998). The small but consistent enhancement (with the location cue) mirrors the effects of attention on single-cell responses in early striate or extrastriate areas where responses across the entire tuning function of a neuron are increased by a factor ranging from about 8% to 25% when attention is directed within the receptive field (McAdams & Maunsell, 1999). 

However, an overall increase in gain across the entire tuning function does not explain the sharp notch in the tuning function obtained with the orientation cue. An alternative to the signal enhancement hypothesis of attention is the re-tuning hypothesis, which implies sharpening of the selectivity profile of a neuron or of a psychophysical filter. It is indeed an open question whether or not the apparent retuning in psychophysical or imaging data (i.e., Murray & Wojciulik, 2004) is a signature for an actual sharpening of single units selectivity (see Boynton, 2004). However, most of the efforts to demonstrate retuning have not been conclusive (Croner & Albright, 1999; Maunsell & Cook, 2002; Spitzer, Desimone, & Moran, 1988), or needed models that do not directly relate to physiology (Dosher & Lu, 2000a; Lu & Dosher, 1998). By measuring the perceptual-tuning functions directly, we have shown that cueing the test orientation causes a sharp dip in the masking functions measured under all conditions. When the mask was close to the test, knowing the orientation of the test helped our observers “see through the mask” by taking advantage of top-down knowledge about the feature of interest. In summary, while the location-cue data could in principle be modeled with a relatively simple gain increase in the relevant filter consistent with previous studies, it is hard to imagine a single-cell mechanism that segregates mask and test so efficiently in the presence of an orientation cue. (Recall that the highly selective effect of the orientation cue occurs for masks that vary in orientation and for masks that vary in location.) The simplest explanation based on the reduction of uncertainty at the decision stage can be rejected because it predicts that thresholds are reduced by a constant factor across the whole tuning function. 

Instead, it is conceivable that both the highly selective effect of the orientation cue and the diffuse effect of the location cue reflect the network activity of the population of neurons involved in the task. (The difference-of-Gaussian fit to the data in Figure 3 and Figure 5 was for purposes of visualizing the data: We are not proposing that a subtractive mechanism underlies the effect of cueing.) As there is little evidence for sharpening of the detector at the level of single cells, attention might cause the narrowing of the perceptual filter by differentially weighting the response of a subset of the population of neurons underlying the task (Verghese, 2001), possibly through gain modulation mechanisms very similar to those suggested by the evidence in favor of the “feature similarity gain model” (Martinez Trujillo & Treue, 2004; Treue & Maunsell, 1999). The baseline response of whole populations of differently tuned neurons would be modulated differently in the two cueing conditions. 

Indeed, we have devised a model where a set of relevant, biologically plausible detectors in the orientation domain contribute to the tuning curves obtained in our experiments. This is a “second-order” or “cascade-of-filters” model that has been proposed to explain the increased selectivity due to perceptual learning in Vernier tasks (Saarinen & Levi, 1995; Waugh, Levi, & Carney, 1993; Yang & Maunsell, 2004). The basic assumption of the model is that cueing changes the weights of each detector in specific ways. Figure 7a replots the data of observer S1 from Figure 3. For comparison with the model simulation in Figure 7b, we have constructed a full tuning curve by assuming that thresholds are identical for masks tilted clockwise and counterclockwise from the test. Figure 7b shows the result of a simulation in which the behavioral tuning curve results from a psychophysical filter composed of detectors with similar orientation preference. Specifically, we assumed that the filter was made up of 5 detectors (dashed functions in Figure 7b) with preferred orientations around the test orientation and spaced at 5° intervals, with an orientation bandwidth of 25°. A tuning curve for the neutral-cue condition is easily reconstructed from this cascade of filters type of model (blue function on the left of Figure 7b). This curve, generated by assigning the same weight (gain) to each component detector, has a bell shape similar to that obtained empirically. The benefits of attending to the location or to the orientation of the test can be achieved by “reweighting” the gain of individual detectors. In the case of the location cue, this reweighting affect increases the gain of all detectors equally, whereas with an orientation cue, the reweighting significantly increases the gain of only the central detector (the one that matches the cue orientation), leaving the others unaffected. Please note that increased gain implies decreased threshold, which is why increased gain is depicted as a smaller detector profile. This simulation shows the effect of these different weighting schemes on a full tuning curve. Increasing the gain of all detectors by a factor of 1.4 yields a tuning curve similar to the location-cue data. Increasing the gain of only the central detector by a factor of 5 while leaving the others unchanged produces the central dip that is characteristic of our orientation-cue data. This simple model captures the main properties of our data surprisingly well. In fact, the tuning curve for the location cue (in red) shows improvement over the entire tuning function, while the tuning function for the orientation-cue condition (in green), shows the largest improvement for mask orientations close to the test. 

Figure 7

 

Simulation of a “reweighting” mechanism that accounts for the two cue-specific patterns in our study. Panel a replots the data of observer S1 from Figure 3 with all three cuing conditions superimposed. For comparison with the simulations, in b we plot a full tuning curve, obtained by mirror reversing the data from Figure 3 about the vertical axis passing through 0°. The three panels in b show simulation of the behavioral tuning function resulting from a cue-specific modulation of the front-end detectors (dashed lines) in each of the three cueing conditions. For the neutral cue (on the left) is the “channel” generated by the responses of an array of five front-end detectors with peak selectivity spaced 5° apart from each other in the orientation domain. The middle and right panels show the tuning function for the location- and orientation-cue conditions, respectively, obtained by modulating the weights the component detectors. In the neutral-cue condition, in blue, the weight assigned to each detector is 1. The location cue tuning function (in red) is obtained by assigning each detector a weight of 1.4. It clearly reproduces the trend of the data, in which the effect of the location cue is visible throughout the range. (Please note that increased gain implies decreased threshold, so increased gain is depicted as a smaller detector profile.) The orientation-cue function (in green) is obtained by assigning a weight of 5 to only the central detector that has a preferred orientation of 0°, while all the others stay fixed at 1. The simulation for this condition reproduces both the central dip and the overlap with the neutral-cue condition for the peripheral regions of the range. (Compare the right side of Figure 7b with the different cueing patterns obtained in Figure 3 and Figure 5.)

It may not be clear that increasing the gain of a filter will improve sensitivity to the test. Let us assume that the response to the mask is R and the response to mask + test is R + ΔR. In other words, ΔR is the incremental response to the test. If the gain of a detector increases by a factor k, then the response of that detector to all stimuli within its passband increases by k, so the incremental response to the test is now k. ΔR. Even if the variance of the noise increases by k, the signal-to-noise ratio or d′ is now k. ΔR/sqrt(k.NoiseVariance). So d′ increases by sqrt(k) even if the noise increases in proportion (Verghese, 2001). Gain can also increase the sensitivity to the test in the framework of Foley’s contrast normalization model (1994). If the excitation to the test grows more rapidly (has a higher exponent) than the divisive normalizing term, then a higher gain at the test orientation will generate a greater response. If this proposal is correct, we may have made substantial progress on the “retuning” versus “enhancement” dispute that has embroiled attention researchers from single-unit recording to psychophysics. The scope of single-unit recording is too narrow to observe retuning if it is indeed a network property, while the enhancement observed behaviorally cannot distinguish between one and many front-end detectors. We believe that both the location and orientation cue signatures can be explained by selectively reweighting the responses at the level of front-end feature detectors. The two different cues would act as different priors in activating such reweighting mechanisms and in selecting the relevant populations for the task. 

Thus, our study may shed light on the mechanisms of visual attention by linking psychophysical data, presumably based on the activity of a population of neurons, with the physiology of single-cell activity. 

Table 1

 

Weighted χ² values for the Gaussian and DoG fits in the orientation and location-masking experiment. The neutral-cue condition (NC columns) shows χ² values relative to Gaussian fits to the data. The location- and orientation-cue conditions (LC and OC columns, respectively) show χ² values relative to the DoG fit to the data unless a * accompanies the value. In this case, the Gaussian fit provided a better fit to that set of data than the DoG

	Orientation masking	Location masking
			NC	LC	OC	NC	LC	OC
S1			0.04	0.05	0.08	0.095	0.049	0.075
S2	0.026	0.00018	0.026	0.085	0.037	0.018
S3	0.015	0.12	0.04
S5				0.139	0.063	0.052
S4	0.081	0.082*	0.072

Acknowledgments

We thank Barbara Dosher for suggesting the symmetric mask experiment and Matteo Carandini for useful advice on the manuscript. This work was supported by the Smith-Kettlewell Eye Research Institute and National Eye Institute Grant EY12038 to PV. 

Commercial relationships: none. 

Corresponding author: Stefano Baldassi. 

Email: . 

Address: Dipartimento di Psicologia — Università di Firenze. Via di S. Niccolò, 93 — 50125 Firenze, Italy. 

Appendix

Fitting procedures

In the datasets of Figure 3 and Figure 5, we estimated the tuning functions in the neutral-cue condition by iteratively fitting the data with a Gaussian function of the form  

 

(1)

where α is the asymptote parameter describing the contrast level where the function flattens, A is the amplitude expressing the strength of the masking effect, σ_neutral is the standard deviation of the Gaussian, and μ is the mean, or peak, of the function, which is fixed at 0. To estimate the beneficial effect of the cues, we subtracted a second Gaussian from the Gaussian fit to the neutral-cue condition. The parameters from this second Gaussian assessed the effect of selective attention to space or features. 

So the data obtained in the location- and orientation-cue conditions were fit with a difference-of-Gaussian (DoG) function of the form  

 

(2)

where the first part is the tuning function from Equation 1, and α₂, A₂ and σ_{location/orientation} in the second part are the key parameters of the “attentional” tuning function that were free to vary. The subscript of the parameter σ indicates that it can be σ_location or σ_orientation according to the cue used. We estimated the goodness of fit in various ways (χ² and R²). In the location- and orientation-cue conditions, we compared the χ² value generated by fitting both Equation 1 and Equation 2 to the same dataset, and the DoG fit was accepted only if it generated a better χ² value. When the DoG fit did not converge consistently, or when it generated lower χ² values than the simple Gaussian fit, we fit the data with a simple Gaussian (Equation 1)