Perceptual consequences of feature-based attention

Abstract

Attention modulates visual processing along at least two dimensions: a spatial dimension, which enhances the representation of stimuli within the focus of attention, and a feature dimension, which is thought to enhance attended visual features (e.g., upward motion) throughout the visual field. We investigate the consequences of feature-based attention onto visual perception, using dual-task human psychophysics and two distant drifting Gabor stimuli to systematically explore 64 combinations of visual features (orientations and drift speeds) and tasks (discriminating orientation or drift speed). The resulting single, consistent data set suggests a functional model, which predicts a maximum rule by which only the dominant product of feature enhancement and feature benefit by feature relevance may benefit perception.

Introduction

It has recently become clear that early visual processing is not purely bottom-up and dependent on stimulus properties alone, but can be substantially modulated top-down by expectations and tasks. A first, widely observed, top-down modulatory effect of attention onto early vision is to locally enhance the cortical representation of stimuli within the focus of attention (Brefczynski & DeYoe, 1999; Chawla, Rees, & Friston, 1999; Lee, Itti, Koch, & Braun, 1999; Motter, 1994; Treue & Maunsell, 1996). This observation is well in line with the shiftable spotlight metaphor of attention (Posner, Snyder, & Davidson, 1980), not only serving to select a subregion of the visual input for further processing, but also highlighting the neural representation of its contents (see, e.g., Itti & Koch, 2001; Treue, 2001, for reviews). 

Previous reports have demonstrated a critical role of task onto this localized attentional modulation within the focus of attention (Corbetta, Miezin, Dobmeyer, Shulman, & Petersen, 1991; Huk & Heeger, 2000). For example, Watanabe et al. (1998) showed, using one stimulus with superimposed translating and expanding fields of dots, differential attentional modulation of fMRI signal, depending on whether the task was to attend to the translating or the expanding feature of the stimulus. Thus, attentional modulation and some forms of perceptual learning appear to be specific not only for location, but also for a subset of stimulus features that are relevant to the task engaging attention. 

More recently, a nonspatial, feature-based form of attentional enhancement has been reported, wherein the activity of neurons tuned to some visual feature of a stimulus (e.g., a given color or direction of motion) is increased throughout the visual field when attention is engaged onto the stimulus. This so-called feature-based attentional modulation has been demonstrated in human functional neuroimaging (Beauchamp, Cox, & DeYoe, 1997; Saenz, Buracas, & Boynton, 2002), monkey electrophysiology (McAdams & Maunsell, 2000; Treue & Martinez-Trujillo, 1999), and human psychophysics (Duncan & Nimmo-Smith, 1996; Saenz, Buracas, & Boynton, 2003; Shih & Sperling, 1996). For example, in a single-unit macaque physiology study, Treue and Martinez-Trujillo (1999) showed how attending to an upward-moving field of dots on one side of the visual field increased activity in area MT, selectively for neurons tuned to upward motion, although they could have receptive fields in the opposite hemifield from the attended stimulus; however, attending to a downward-moving field of dots did not. In a human psychophysics study, Saenz et al. (2003) showed how performance on a dual-task was significantly better when human observers divided attention across two spatially separate stimuli and attended to a same feature on both sides (same direction of motion or same color) compared with opposing features. Of particular interest here, Vidnyánszky et al. (2003) recently suggested that motion after-effect, induced by an ignored field of dots whose dominant motion was carried by dots of a given color, appeared stronger if subjects attended, in a distant aperture of the display, to dots of that color moving in random directions. Hence, feature-based attention seems to affect even task-irrelevant features, in this case, motion, whereas the task-relevant feature was color. Whether the enhancement of task-irrelevant features was comparable to that of task-relevant features, however, was not elucidated. 

In sum, previous reports suggest a critical interaction between task and stimulus in shaping attentional modulation and perceptual learning for many isolated pairings of tasks and stimuli at same or different visual locations. However, a complete quantitative understanding of how task-relevant and task-irrelevant features are comparatively enhanced by attentional modulation remains elusive. Here, we operate a systematic dissociation between tasks and stimulus features to elucidate their interplay in shaping feature-based attentional modulation. We employ dual-task human psychophysics and two distant drifting Gabor stimuli to systematically explore 64 combinations of visual features (orientation and drift speed) and tasks (discriminating orientation or drift speed). Our systematic study thus aims at casting previous and future findings on isolated task and feature combinations into a single conceptual and computational framework. 

Methods

Subjects and paradigm

Three normal volunteer human subjects simultaneously performed two pattern discriminations (Lee et al., 1999; Sperling & Melchner, 1978) on two drifting Gabor stimuli presented bilaterally to fixation (Figure 1). In each block of 100 trials, subjects divided attention between both stimuli: one engaged their primary task (first response and behavioral priority and dominance), presumably triggering a first set of feature-based attention effects, and the other was their secondary task (second response), presumably triggering a second set of effects. Here we investigate how performance at the primary and secondary tasks might have benefited from each other through possible feature-based attentional enhancement triggered at the other site. 

Figure 1

 

Experimental paradigm. Stimuli were two drifting Gabor patches (parameters are given in the Methods section) presented bilaterally to the central fixation cross. Dashed circle (not in actual stimuli) indicates the primary stimulus (given behavioral priority and first response). Arrows indicate drift direction and speed. Here, the primary task is a two-interval forced-choice orientation discrimination (subjects responded whether the stimulus was vertical then tilted or tilted then vertical). The secondary task is drift speed discrimination (faster then slower or slower then faster). Using the notations of the text, the condition shown is OVS–DHS.

Stimuli and tasks

Stimuli were generated in Matlab (Mathworks, Natick, Massachusetts) using the Psychophysics Toolbox (Brainard, 1997). Subjects were seated at a viewing distance of 80 cm from a 22" color monitor (LaCie, Inc., Hillsboro, OR; 28° × 21° usable field-of-view, 72 Hz) and rested on a chin-rest. Mean screen luminance was 30 cd/m²; room, 4 cd/m². Drifting Gabor stimuli (spatial frequency 3.6 cpd, envelope FWHM 0.55°, contrast 0.4) were horizontal or vertical (H or V orientation feature), drifting slowly (3.33 cps; S drift speed feature) or faster (5.00 cps; F drift speed feature), and were displayed at 4° eccentricity from fixation. The timing of each trial is shown in Figure 1. 

Tasks were orientation (O) or drift speed (D) discriminations, using a dual two-interval forced-choice (2IFC) paradigm. On any given trial, only one parameter differed between the two successive Gabor patch presentations, independently on each side of the display. In the orientation discrimination tasks, stimuli could be oriented vertically (condition V) or horizontally (condition H) versus slightly tilted off that orientation. Subjects then reported whether the slightly tilted patch had appeared first. In the drift speed discrimination task, stimuli drifted at the slower (condition S) or faster (condition F) speed versus slightly increased speed. Subjects then reported whether the patch with slightly increased speed had appeared first. Subjects reported the order of the alternative targets for both the left and the right sides of the display, by pressing two keys, and received auditory feedback (two brief tones were emitted for incorrect answers, and three tones for correct answers). 

Given 128 possible task and stimulus combinations (primary side, left or right; tasks, either O or D on each side; stimuli, each among HS, VS, HF, or VF), we measured dual-task discrimination thresholds for 64 combinations (secondary stimulus speed always S). 

Trials

Subjects trained on every task and stimulus combination until threshold stabilization was reached to factor out perceptual and other learning. After training, between 10 and 30 blocks of 100 trials were acquired for each data point plotted in Figures 2 and 3 (depending on difficulty of task combinations). Thresholds (75% correct performance) were determined with an adaptive dual staircase procedure, adjusting the difference in patch orientation or drift speed between the two successive alternatives for each stimulus in each trial, according to success or failure of previous trials (Lee et al., 1999). Each threshold was computed from a maximum-fit of a Weibull function with two degrees of freedom to the staircase data (Itti, Koch, & Braun, 2000). To provide a baseline for the dual-task thresholds reported in this study, single-task thresholds were also measured by instructing subjects to only perform one task at one site while keeping the stimuli at both sites identical to the dual-task stimuli. 

Figure 2

 

Orientation secondary task performance. An example task and stimuli combination is listed below the figure to show the eight conditions. In this example, the secondary task is always OVS, and the primary tasks are OVS, OVF, OHS, OHF, DVS, DVF, DHS, and DHF corresponding to the eight conditions. Each data point represents threshold elevations (ratio of secondary thresholds to corresponding single-task thresholds) averaged over left-primary, right-primary, and corresponding horizontal-secondary and vertical-secondary conditions. Error bars represent standard error on the mean. Statistical t test shows p < .05 for most comparisons, p < .075 at worst for the significant difference between thresholds described in the results.

Figure 3

 

Drift speed secondary task performance. An example task and stimulus combination is listed below the figure to show the eight conditions. In this example, the secondary task is always DVS and the primary tasks are DVS, DHS, DVF, DHF, OVS, OHS, OVF, and OHF corresponding to the eight conditions. Format is as in Figure 2.

Results

We focus on how different tasks and stimuli at one site modulated the thresholds at the other site, that is, how feature-based attentional modulation triggered at the one site affected perception at the other site. Because primary task thresholds did not exhibit large variations across tasks, here we focus on the secondary thresholds first. (The primary thresholds will be subsequently handled by the modeling.) We define stimulus orientation as relevant to the orientation discrimination task whereas stimulus drift speed is irrelevant, and conversely. Figures 2 and 3 show secondary thresholds under identical or different tasks (T or

$\bar{\mathrm{T}}$

), same or different stimulus feature relevant to the secondary tasks (R or

$\bar{\mathrm{R}}$

), and same or different feature irrelevant to the tasks (I or

$\bar{\mathrm{I}}$

). Table 1 shows all the task combinations and corresponding relevant properties to the secondary task. Note that stimuli always shared some task-irrelevant features, such as luminance contrast; because these were invariant, we assume no differential effect on observations. To focus our analysis on differential threshold modulation by stimuli and tasks at the other site, all thresholds are reported as threshold elevations (i.e., normalized by the corresponding single-task thresholds). In addition, all 64 thresholds were collapsed into 16 compound data points by averaging to reduce bias across left- and right-primary and across vertical and horizontal orientations. 

Table 1

 

All possible task and stimulus combinations. Here, if the left side is primary, then the right side is secondary, and conversely. The upper half of the table shows all the task combinations when the secondary tasks are orientation discrimination tasks; the lower half of the table shows the secondary tasks are drift discrimination tasks. Here, TRI and the other top row labels represent features relevant or irrelevant to the secondary tasks.

TRI	T⁢RI―	TR―I	TR⁢I―	T―R⁢I	T―RI―	T⁢R―I	T⁢R⁢I―
OVS–OVS	OVF–OVS	OHS–OVS	OHF–OVS	DVS–OVS	DVF–OVS	DHS–OVS	DHF–OVS
OHS–OHS	OHF–OHS	OVS–OHS	OVF–OHS	DHS–OHS	DHF–OHS	DVS–OHS	DVF–OHS
DVS–DVS	DHS–DVS	DVF–DVS	DHF–DVS	OVS–DVS	OHS–DVS	OVF–DVS	OHF–DVS
DHS–DHS	DVS–DHS	DHF–DHS	DVF–DHS	OHS–DHS	OVS–DHS	OHF–DHS	OVF–DHS

With identical tasks and stimuli (TRI), perception of the secondary stimulus presumably benefited maximally from feature-based attention triggered at the primary (Saenz et al., 2003). Indeed, secondary thresholds were the lowest (best) of all conditions, yet they were higher than the corresponding single-task thresholds (threshold elevations greater than unity), due to splitting of attentional resources across both tasks (Lee et al., 1999; Sperling & Melchner, 1978). Progressively decreasing the similarity of both stimuli showed how secondary thresholds benefited when stimuli shared task-relevant features (TRI better than

$\mathrm{T}\bar{\mathrm{R}}\mathrm{I}$

) (McAdams & Maunsell, 2000; Saenz et al., 2003; Treue & Martinez-Trujillo, 1999), but did not benefit further from additionally sharing task-irrelevant features (TRI same as

$\mathrm{T}hui\mathrm{R}\bar{\mathrm{I}}$

). However, when no task-relevant feature was shared, sharing a task-irrelevant feature benefited thresholds (

$\mathrm{T}\bar{\mathrm{R}}\mathrm{I}$

better than

$\mathrm{T}\overline{\mathrm{R}hui\mathrm{I}}$

), although often more weakly than sharing a task-relevant feature. Together, our results under same-task conditions hence suggest a maximum rule (at the basis of our model below), by which perception improves according to the maximal product of feature enhancement and feature benefit. 

When tasks differed, sharing any feature improved secondary thresholds (

$\bar{\mathrm{T}}\mathrm{R}hui\mathrm{I}$

,

$\bar{\mathrm{T}}\mathrm{R}\bar{\mathrm{I}}$

and

$\overline{\mathrm{T}hui\mathrm{R}}\mathrm{I}$

better than

$\overline{\mathrm{T}hui\mathrm{R}hui\mathrm{I}}$

), but without distinction between task-relevant and task-irrelevant features as found under same-task conditions (

$\bar{\mathrm{T}}\mathrm{R}hui\mathrm{I}$

,

$\bar{\mathrm{T}}\mathrm{R}\bar{\mathrm{I}}$

and

$\overline{\mathrm{T}hui\mathrm{R}}\mathrm{I}$

not significantly different from each other). Under different-task conditions, our notations here denote as task-relevant a feature that is relevant to the secondary tasks. Different-task thresholds were often higher than the corresponding same-task thresholds (less so for drift secondary), indicating an additional cost of performing our two different tasks (indeed, subjects reported these conditions to be harder than the same-task conditions). Some of this cost may not be considered purely attentional and could be attributed to other processing limitations, including decision processes, short-term memory, overall combined task load (Lee et al., 1999; Sperling & Melchner, 1978), or the ability of switching from primary to secondary in different tasks. 

Our secondary thresholds results suggest a complex pattern of interactions between tasks and stimulus features in determining the perceptual consequences of feature-based attentional modulation. Overall, these results are highly consistent with previous reports for individual pairwise comparisons (Huk & Heeger, 2000; Saenz et al., 2003; Saffell & Matthews, 2003; Watanabe et al., 1998). Most of the primary thresholds also exhibit similar trends with secondary thresholds (primary thresholds are reported together with the model). Our data set of 64 conditions under consistent viewing conditions, stimuli, tasks, and observers, allows us to derive a simple functional model for the observed effects. This simple model will accommodate both the primary thresholds and the secondary thresholds, which assume that primary thresholds are also somewhat modulated by feature-based attention triggered at the secondary tasks. 

Qualitatively, our data suggest that the interaction among attention, stimulus, and task obeys three governing principles: First, engaging attention onto a stimulus through a pattern discrimination task enhances task-relevant features strongly and task-irrelevant features weakly, throughout the visual field. Second, performance at a pattern discrimination task benefits strongly if task-relevant features are enhanced and weakly if task-irrelevant features are enhanced. Third, feature enhancement and feature benefit combine multiplicatively, with the final benefit to perception and task performance dictated by the largest of these products. 

Model

To further formalize these observations and derive a computational model, consider a pattern discrimination task t and a visual feature f represented in the observer's visual system. Let r(f,t)∈[0,1] denote the task relevance of f to t and A(t)∈[0,1] be the fraction of the observer's attentional (including spatial) resources devoted to t. We express the amount of nonspatial enhancement E(f,t) in the cortical representation of feature f resulting from devoting fraction A(t) of resources to task t as:  

$\forall huif,t,E(f,t)=r(f,t)A\left(t\right)$

(1)

 

Now consider another task t′ on a stimulus at the other site. The most parsimonious formulation of how performance of t′ may be influenced by simultaneous performance of t includes at least three terms: First, there is a fraction A(t′) of resources available to t′. This A(t) should be observer-dependent, and A(t′) = 1 − A(t′) in each dual-task condition. Second, t′ may benefit from possibly enhanced feature representations as attention is partly engaged onto t. Third, we assume a simple gain value α applied to the orientation or drift thresholds to represent the subject's ability to perform a certain combination of dual tasks. Certainly, the gains are different when subjects perform two identical tasks compared with two different tasks. Based on our observations, our model tests a maximum (or winner-take-all) rule where total expected benefit is the maximum overall feature of the product between feature enhancement due to t and relevance of that feature to t′. Certainly, the contribution of these factors to task performance depends on complex and nonlinear processes (Itti & Koch, 2001; Lee et al., 1999), not fully elucidated and constrained by our data alone. Thus, our model's formulation only attempts a first-order approximation, with the main merit of clearly identifying key parameters and of providing a simplified understanding of their interactions. Predicted dual-task discrimination threshold Th(t′∣t) for t′ given t then is:  

$\mathrm{T}hui\mathrm{h}(t\prime \mid huit)=\frac{\alpha \times \mathrm{T}{\mathrm{h}}_0(t\prime )}{A\left(t\right)\times [1+\underset{f}{\max }r(f,t\prime \left)E\right(f,t\left)\right]}$

(2)

where Th₀(t′) is the single-task threshold for t′. Here when t is the primary task, t′ is the secondary task, and conversely. 

To test the validity of this first-order formulation, we progressively reduce the number of free parameters. First, we assume that in all eight conditions (TRI,

$\mathrm{T}hui\mathrm{R}\bar{\mathrm{I}}$

,

$\mathrm{T}\bar{\mathrm{R}}\mathrm{I}$

,

$\mathrm{T}\overline{\mathrm{R}hui\mathrm{I}}$

,

$\bar{\mathrm{T}}\mathrm{R}hui\mathrm{I}$

,

$\bar{\mathrm{T}}\mathrm{R}\bar{\mathrm{I}}$

,

$\overline{\mathrm{T}hui\mathrm{R}}\mathrm{I}$

, and

$\overline{\mathrm{T}hui\mathrm{R}hui\mathrm{I}}$

), the attentional allocations A(t) and A(t′) to both sides may differ. The fitting procedure employed a two-dimensional Newton–Gauss optimization, with the root-mean-square difference (in units of measurement standard error) between the 16 model predictions (eight primary thresholds, eight secondary thresholds) and the measured thresholds as the objective function. We assume a relevance of 1 for task-relevant features (e.g., r(V,OVS) = 1), 0.2 for task-irrelevant features (e.g., r(V,OVS) = 0.2), and 0 for features antagonistic to either task-relevant or task-irrelevant features (e.g., r(V,OVS) = r(V,OVS) = 0). These values were invariant across all conditions. Making the value for task-irrelevant features an additional optimization parameter instead of 0.2 yielded the average values equal to 0.2 with small variance, hence the value used here. Thus, the model has 10 free parameters (A(t) or A(t′) for all eight conditions and two gain values for same and different tasks) for each set of 16 data points (8 primary and 8 secondary thresholds averaged from the 64 total task conditions). Overall, this many parameter model suggests a good agreement with our empirical data (R² = .981). The model fitting is done independently for each subject. The fitting parameters are shown Table 2. 

Table 2

 

All parameters of the model fits with 10 free parameters for orientation and drift tasks for all three subjects. A represents the fraction of attentional resources denoted to primary task in that condition. The corresponding attentional resource to the secondary side in that condition is 1A. α_s is gain to orientation or drift thresholds for a certain task combination in the same-task conditions and α_d is gain to orientation or drift thresholds for a certain task combination in the different-task conditions. Note that in each condition, α_s is always less than α_d, indicating that splitting attention across different tasks yields higher gain value and higher additional cost. The parameters suggest that in all the eight conditions, subjects divided attentional resources between both sides similarly (similar A values across conditions), suggesting a model fit with fewer parameters as described in Table 3.

Orientation	A(TRI)	A(T⁢RI―)	A(TR―I)	A(TR⁢I―)	A(T―R⁢I)	A(T―RI―)	A(T⁢R―I)	A(T⁢R⁢I―)	αs	αd
JW	0.63	0.68	0.63	0.66	0.70	0.71	0.71	0.72	1.24	1.30
JJ	0.57	0.48	0.60	0.69	0.74	0.75	0.72	0.74	0.75	0.92
SQ	0.64	0.66	0.61	0.64	0.67	0.69	0.48	0.70	0.82	0.94

Drift	A(TRI)	A(T⁢RI―)	A(TR―I)	A(TR⁢I―)	A(T―R⁢I)	A(T―RI―)	A(T⁢R―I)	A(T⁢R⁢I―)	αs	αd
JW	0.59	0.57	0.60	0.61	0.64	0.66	0.65	0.66	0.78	0.98
JJ	0.50	0.53	0.59	0.60	0.58	0.56	0.59	0.56	0.92	1.01
SQ	0.55	0.54	0.58	0.59	0.57	0.56	0.56	0.58	0.76	0.79

The parameters from the many-parameter fitting suggest that the different A(t) (also A(t′)) obtained from the eight different conditions are similar, which means that subjects divided attentional resources into primary and secondary tasks similarly in each condition. This motivates us to collapse these eight A(t) parameters into one single parameter. Further reduction of the number of free parameters assumes that subjects divided attention into two sides with the same ratio in all conditions. This few-parameter model fitting with only three free parameters still results a good fitness of the model with the data (R² = .956). Table 3 lists the fitting parameters for the three subjects. 

Table 3

 

All parameters of the model fits using three free parameters, for orientation and drift tasks for all three subjects. Notations are as in Table 2.

Subject	Orientation			Drift
	A	αs	αd	A	αs	αd
JW	0.70	1.13	1.33	0.63	0.77	1.01
JJ	0.71	0.69	1.00	0.58	0.93	1.00
SQ	0.68	0.74	0.96	0.57	0.77	0.79

Figure 4 shows model predictions when only these three free parameters were fit to each set of 16 observations from each observer. Overall model prediction accuracy was close to measurement error. The values obtained for A(t)suggest that, as instructed, subjects devoted more attentional resources to the primary task (average A(t) = 64%, range 57–71%, compared with average A(t′)= 36%, range 29–43%). Because the gain parameter in same-task conditions is always smaller than the corresponding different-task conditions, it indicates that there is an additional impairment for performing two different tasks. We assumed the same relevance of 0.2 for task-irrelevant features. Thus, the model has three parameters for each set of 16 data points plus one global parameter for all thresholds. Each model prediction was, on average, 2.01 measurement standard error from the corresponding data point (range 0.008–9.06), and of 96 model predictions (3 subjects and 32 predictions each), the model overestimated the data in 43 and underestimated in 53 cases. 

Figure 4

 

All the primary and secondary performance for both orientation (left) and drift (right) tasks and their model predictions. Cyan wide bars represent the threshold elevation performance and the inside narrow bars represent the corresponding model prediction. The model was fitted with three free parameters, namely, A(t), which is the attentional resource to primary task, so that the corresponding attentional resources to the secondary task in that condition are A(t′) = 1 − A(t); α_s, which is the gain to orientation or drift thresholds for a certain task combinations in same-task conditions; α_d, which is the gain to orientation or drift thresholds for certain task combinations with different-task conditions and different-task thresholds (see Table 3 for each of the 16 data points, with 8 primary thresholds and eight 8 thresholds) independent of subjects. Overall model prediction accuracy was close to measurement error and suggested a good fit (R² = .956). Note that for primary thresholds, the TRI related labels represent features relevant or irrelevant to the primary tasks.

Discussion

Our systematic dissociation between tasks and stimulus features in a single data set unraveled a seemingly complex pattern of interactions between behavioral demands and visual inputs. Our data and model, however, suggest that the interaction among attention, stimulus, and task is well captured by a simple descriptive model. The model predicts a maximum rule by which only the dominant product of feature enhancement and feature benefit by feature relevance may benefit perception. Previous studies, using isolated pairs of tasks and stimuli, illustrated how both task-relevant and task-irrelevant features may be enhanced by attention. Our results not only confirm this fact, but also show that task-relevant and task-irrelevant features are enhanced with different gains and that discrimination performance benefits differently from possibly enhanced task-relevant and task-irrelevant features. 

Our study of same-task conditions is consistent with recent findings from human psychophysics using the 2IFC paradigm that dual-task performance was better when subjects attended to a same feature on both sides of a dual display, compared with when they attended to two different features (Saenz et al., 2003). On the other hand, feature-based attentional effect was also found in an fMRI experiment (Saenz et al., 2002), with the attended stimulus on one side and an ignored stimulus on the other side, which suggested that feature-based attention may not be because of the specific property of dual tasks. Further, under different-task conditions, our results are also consistent with the literature. For example, Rossi and Paradiso (1995) reported how when subjects performed a primary task of discriminating spatial frequency or orientation of a foveal grating, a secondary task of detecting a near-threshold grating in the periphery benefited when the peripheral grating's spatial frequency or orientation matched the attended feature of the primary stimulus. Although with our tasks of orientation and speed discriminations, we have observed an impairment for different compared with same task conditions, it is important to note that this may not necessarily be the case for all task combinations. For example, little or no impairment has been observed in concurrent performance of a visual and an auditory discrimination tasks (Duncan, Martens, & Ward, 1997). Mixing two visual tasks, Morrone, Denti, and Spinelli (2002) found no impairment in concurrent mixed color and luminance contrast discriminations, compared with two-color or two-luminance discriminations. This suggests that the amount of impairment from performing two different tasks compared with twice the same task also is highly task-dependent. In another study, Morrone, Denti, and Spinelli (2004) reported no significant effect on the secondary luminance discrimination task when they varied the primary form or contrast discrimination task relevance and difficulty. These results are partly consistent with our findings (the insignificant difference among thresholds in different-task conditions,

$\bar{\mathrm{T}}\mathrm{R}hui\mathrm{I}$

,

$\bar{\mathrm{T}}\mathrm{R}\bar{\mathrm{I}}$

, and

$\overline{\mathrm{T}hui\mathrm{R}}\mathrm{I}$

in our study, and partly inconsistent,

$\overline{\mathrm{T}hui\mathrm{R}hui\mathrm{I}}$

, has higher thresholds than the previous three conditions). This is possibly because they used a central versus peripheral dual task paradigm whereas we employed two symmetrical peripheral tasks. Another reason could be the nature of the stimuli. In their experiment, the central task is to discriminate form of stimuli or contrast of form stimuli while the peripheral task uses sinusoidal gratings. It is possible that engaging attention onto a form or contrast discrimination yields a different pattern of enhancement than the orientation and speed discrimination tasks used in our study. 

Several studies have shown that human observers performed better when concurrently discriminating two features of the same object compared with two features of different objects (Blaser, Pylyshyn, & Holcombe, 2000; Duncan, 1984). In our experiment, because two moving Gabor patch stimuli could be perceived as part of a common object viewed through two apertures, our findings could also be attributed to an object-based attention effect. However, in many visual objects, the object often is defined by sharing of common features. However, we have no enough evidence that in our study subjects perceived left and right stimuli as part of a common object or not. 

Our results contrast with those of Saenz et al. (2003) in that effects were observed although, we did not require that competing stimuli be present and overlapping with the attended stimuli. It is possible that this difference is in part due to a different choice of tasks used to engage attention onto specific features of stimuli. Saenz et al. (2003) use a speed discrimination task to engage attention. However, it could be argued that this task may not necessarily enhance the cortical representation of stimulus features such as direction of motion or color, which seem irrelevant to the task. This could explain why no feature-based enhancement was observed in this simple situation when both stimuli shared the same direction of motion or color. Adding competing stimuli (moving in the opposite direction or having a different color), then, modified the task by adding a requirement on the observers to separate between the component of the stimuli onto which they performed the task (e.g., upward moving dots or red dots), and the competing components that they should ignore (e.g., downward moving dots or green dots). This separation task required active discrimination of direction of motion and color, which were the features tested for when evaluating possible feature-based attention effects. 

Task-relevant and task-irrelevant features in our study are defined by whether the shared feature in the two stimuli is relevant or irrelevant to one of the dual tasks. As subjects performed two tasks at the same time, feature-based attention would predict strongest effects when a feature at one site that is relevant to the specific feature attended at the other site. In our study, features always were enhanced (attended) at one site, resulting to feature-based attention at the other site. However, in another fMRI study, the fMRI response to the unchanging ignored stimulus in the opposite visual hemifield was increased when observers attended the same features (direction of motion, color, etc.) compared with the opposing features (Saenz et al., 2002). These results suggested that the relevant or irrelevant features could also be defined based on a single task. Another concern is the long exposure of the stimuli. To prevent eye movements across the two tasks, we asked the subjects to focus on the central cross while they performed the tasks. Because we do dual tasks, with both tasks being difficult, the resulting thresholds with shorter presentation of the stimuli were very poor both in primary and secondary tasks. Indeed, subjects reported they had difficulties to perform the two tasks simultaneously when the presentation of the stimuli was shorter. In principle, we want to keep the primary performance as good as possible, which required that we both elongate the exposure of the stimuli and of the blank period between the two presentation periods. Despite this relatively long exposure and interperiod blank of the stimuli, subjects still reported that the task was very demanding, which limited the possibility for eye movement. It should, however, be noted that our rather long stimulus presentation times may slightly blur the distinction between feature-based attention effects during stimulus encoding and comparison with possibly other effects occur during stimulus maintenance. These issues will be addressed in future experiments. 

Computationally, our observations and model are compatible and extend the recent study of Huk and Heeger (2000), which suggested that increasing the gain of neurons encoding task-relevant features might improve their signal-to-noise ratio and hence help improve task performance. Our results are also consistent with a recent study of perceptual learning (Saffell & Matthews, 2003) in which subjects trained to discriminate stimulus direction showed significant improvement in direction discrimination, but their speed or luminance discrimination performance did not improve; conversely, subjects trained to discriminate speed improved in speed but not in direction or luminance discriminations (also see Shiu & Pashler, 1992). In addition, our systematic dissociation between task and features exacerbates the double role of task relevance, not only in determining the strength at which a feature may be enhanced by engaging attention onto a stimulus and task, but also in determining how much a task may benefit and performance may improve from an enhanced feature. 

Thus, our results suggest a neuronal mechanism by which attention enhances the activity of cortical neurons that encode behaviorally relevant stimulus properties. This is highly reminiscent of the biased competition model (Desimone & Duncan, 1995; Reynolds, Chelazzi, & Desimone, 1999), which proposed that multiple stimuli activate competing populations of neurons (at one visual location) and attention biases the competition in favor of neurons that prefer task-relevant features. This was further demonstrated in that engaging attention onto a neuron's preferred stimulus increased the neuron's firing rate, whereas attending to a nonpreferred stimulus decreased firing rate (Luck, Chelazzi, Hillyard, & Desimone, 1997; Treue & Maunsell, 1996). Therefore, the effect of attention on a neuron's response (enhancement or suppression) depends on how the features of the attended stimulus match the feature selectivity and preference of the neuron. Our results extend these findings beyond localized competing stimuli within one neuron's receptive field to distant stimuli across the visual field. 

In that sense, our model is consistent with and extends the “feature similarity gain model” for feature-based attentional modulation (Treue & Martinez-Trujillo, 1999), and exacerbates the critical role of task-relevance, not only in determining feature-based attentional modulation strength [through r(f,t)], but also in determining possible benefits of a modulation onto visual perception [through r(f,t′)]. Although task may have previously often been chosen arbitrarily for the sole purpose of engaging attention onto or away from a stimulus, our study suggests that it has very strong consequences onto both triggering and benefiting from attentional modulation. For example, our model predicts that performing a luminance increment detection task on a stimulus of given color only yields weak feature-based attentional enhancement of the cortical representation of that color, because color is irrelevant to the task; a stronger modulation would be expected if attention was engaged through a color (hue) discrimination task; conversely, our model predicts that enhancing the representation of a given color only weakly improves performance at a luminance increment detection task on a stimulus of that color, but would yield greater improvement on a hue discrimination task. 

We have previously (Lee et al., 1999) proposed that spatial (not feature-based) attention activates a winner-take-all competition among neurons with overlapping receptive fields. The maximum rule found here for feature-based attention is compatible with this mechanism. The neuronal mechanism of the maximum rule proposed here for feature-based attention may be that the multiplicative effect of feature enhancement and feature benefit across features also activates some winner-take-all competition among features. This competition could take place either at the level of early sensory coding or at a higher level, by which all the relevant and irrelevant feature enhancement and feature benefit may take place but only the largest multiplicative effect wins the competition and eventually affects behavior. 

In sum, our systematic dissociation between tasks and stimulus features in a single data set unraveled a seemingly complex pattern of interactions between behavioral demands and visual inputs. Our data and model, however, suggest that the interaction among attention, stimulus, and task follows three simple functional principles: engaging attention onto a stimulus by a task enhances task-relevant features of the stimulus strongly but its task-irrelevant features weakly throughout the visual field; performance at a (possibly different, distant, and concurrent) pattern discrimination task benefits strongly if visual features relevant to that task are enhanced, but only weakly if task-irrelevant features are enhanced; finally, feature enhancement and feature benefit combine multiplicatively, with the final benefit to perception and task performance dictated by the largest of these products. This model allows us to recast a significant body of recent studies of attentional modulation within a simple unifying computational framework. This framework may be used to guide future electrophysiology, imaging, and behavioral studies of the perceptual consequences of attention. 

Acknowledgments

This research was supported by NIH/NEI (grant EY13791-03), NSF (grant IIS-0112991), NIMA (grant 1042502), the USC Zumberge Fund, and the Powell Foundation. 

Commercial relationships: none. 

Corresponding author: Laurent Itti. 

Email: [e-mail]. 

Address: Hedco Neuroscience Building, Room 30A, 3641 Watt Way, Los Angeles, CA 90089.