Covert attention enhances letter identification without affecting channel tuning

Abstract

Directing covert attention to the target location enhances sensitivity, but it is not clear how this enhancement comes about. Knowing that a single spatial frequency channel mediates letter identification, we use the critical-band-masking paradigm to investigate whether covert attention affects the spatial frequency tuning of that channel. We find that directing attention to the target location halves threshold energy without affecting the channel’s spatial frequency tuning.

Introduction

Covert attention is enhanced processing of visual information at a cued location without eye movements. It affects psychophysical performance of many early visual processing tasks, some considered to be “preattentive,” including contrast detection (Carrasco, Penpeci-Talgar, & Eckstein, 2000; Foley & Schwarz, 1998; Lee, Koch, & Braun, 1997; Lu & Dosher, 1998), orientation discrimination, detection, and localization (Baldassi & Burr, 2000; Carrasco et al., 2000; Lee, Itti, Koch, & Braun, 1999; Morgan, Ward, & Castet, 1998), texture segmentation (Yeshurun & Carrasco, 1998, 2000), spatial acuity (Morgan et al., 1998; Yeshurun & Carrasco, 1999), visual search (Carrasco & McElree, 2001; Nakayama & MacKeben, 1989), and letter identification (MacKeben, 1999). These studies, in conjunction with neurophysiological studies using single-unit recordings and neuroimaging techniques, indicate that covertly attending to a location modulates low-level visual processes (Brefczynski & DeYoe, 1999; Ito & Gilbert, 1999; Martinez et al., 1999; McAdams & Maunsell, 1999a, 1999b; Motter, 1993; Reynolds & Desimone, 1999; Ress, Backus, & Heeger, 2000; Roelfsema, Lamme, & Spekreijse, 1998; Treue & Martinez Trujillo, 1999). 

In this study, we assess whether covert attention can change the tuning of a spatial frequency channel (DeValois & DeValois, 1988; Graham, 1989). Specifically, we investigate two hypotheses: 

•  
  Covert attention shifts the peak frequency of the channel. This possibility is raised by the “resolution hypothesis”: attention increases spatial resolution at the attended location. Attention improves performance in both acuity and hyperacuity tasks in the absence of distracters (Yeshurun & Carrasco, 1999), with or without a local mask (Carrasco, Williams, & Yeshurun, 2002). Furthermore, in a texture segmentation task, attention enhances or impairs performance depending on whether spatial resolution is too low or too high for the scale of the texture. The authors concluded that attending to a location is similar to reducing the size of the visual filter used by the observer (Talgar & Carrasco, 2003; Yeshurun & Carrasco, 1998, 2000). This resolution hypothesis is consistent with findings showing that the receptive field constricts around the attended stimulus, causing the unattended stimulus to be left outside of the neuron’s receptive field (Moran & Desimone, 1985; Reynolds & Desimone, 1999; Treue & Maunsell, 1996). Such a compression of the receptive field might increase the peak frequency of the channel.
•  
  Covert attention alters the channel bandwidth, making it better matched to the signal. There is no consensus as to whether attention increases the selectivity of the neuronal response. Some have reported that attention narrows the tuning (for orientation and color) of neurons in V4 (Reynolds & Desimone, 1999; Reynolds, Pasternak, & Desimone, 2000; Spitzer, Desimone, & Moran, 1988), whereas others have found an increased gain but unchanged tuning (for orientation) in V4 (McAdams & Maunsell, 1999a) and (for direction of motion) in MT/MST areas (Treue & Martinez Trujillo, 1999). Increased contrast sensitivity for a grating could arise through a narrowing of the channel tuning, but increased sensitivity for a broadband stimulus such as a letter would arise from a widening of the bandwidth. In general, sensitivity would be increased by better matching the channel to the noise-normalized signal.

To investigate these two hypotheses, we chose a task that isolates the spatial frequency channel that mediates the identification of broadband stimuli, letters. A broadband stimulus could be seen through channels with various tunings, allowing us to test for shifts of peak frequency of the channel as a result of directing covert attention. Given that observers have multiple independent channels with various peak frequencies, one would expect a broadband stimulus such as a letter to activate many channels. However, using a critical-band-masking paradigm with unfiltered letters, the same filter tuning is found for detection of narrowband gratings and identification of broadband letters (Solomon & Pelli, 1994). Critical-band masking of letters allows us to measure the effects of covert attention on the tuning of a spatial frequency channel. 

Covert attention can be directed to a certain part of the visual scene either in a sustained (observers voluntarily direct their attention) or transient (exogenously controlled by the sudden onset of a peripheral cue) fashion (Cheal & Lyon, 1991; Jonides & Irwin, 1981; Nakayama & Mackeben, 1989). We directed transient covert attention to the target location by using a peripheral cue (e.g., Carrasco et al., 2000, 2001, 2002; Talgar & Carrasco, 2003; Yeshurun & Carrasco, 1998, 1999, 2000). To quantify the attentional benefit, we used two control conditions. The “central-neutral” cue appeared at the center of the display. To test for the possibility that this cue reduces the extent of the attentional spread by attracting attention to its location, away from the peripheral target locations (Pashler, 1998), we also employed a “distributed-neutral” peripheral cue presented at all possible target locations. By simultaneously stimulating the detectors at all candidate locations, the distributed-neutral cue should reduce uncertainty as well as any differences in the onset time of activation in response to the central-neutral and the peripheral cues. 

Methods

Observers

Three individuals (two graduate students and one research assistant) from New York University were observers in this experiment. All had normal or corrected-to-normal vision. Two were naive to the purposes of this study. The third was an author. 

Apparatus

Stimuli were displayed on a gamma-corrected computer monitor in a dark room. A video attenuator drove the green gun of the IBM 21″ Multiscan color monitor (Pelli & Zhang, 1991), whose frame rate was 75 Hz. The background luminance of the monitor was 16 cd/m², corresponding to the middle of the monitor’s range. 

Stimuli

The stimuli were created by an Apple Macintosh G4 computer using MATLAB 5.2.1 and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997; http://psychtoolbox.org). A letter was added to a background of visual noise. The noise covered an area twice as high and twice as wide as the letter, centered on the letter. The noise square was outlined in black to clearly demarcate the location. The letter subtended 1.2° and the noise 2.4°. 

The contrast of the letters is specified by the Weber contrast  

 

, where L is luminance of the letter and L_background is the mean luminance of the background. The contrast energy of the letter is the product of the squared contrast and ink area. More generally, energy is defined as the contrast power of the signal, integrated over space , where c(x,y) is the contrast function  

 

and L(x,y) is the luminance at location x,y. Threshold contrast was measured over 60 trials of the modified Quest staircase procedure (King-Smith, Grigsby, Vingrys, Benes, & Supowit, 1994; Watson & Pelli, 1983) using an 82% correct criterion and a β of 3.5. 

For noise, the root mean square (RMS) contrast is defined as  

 

, where the angle brackets denote expected value over all x,y. The RMS contrast of the noise was 0.35. The noise was static and made up of square checks: 2 × 2 pixels. The 2 × 2 check size is advantageous in quadrupling the power spectral density of the noise, and in minimizing the potential for luminance artifacts due to the finite slew rate of the CRT’s video amplifier (Brainard, Pelli, & Robson, 2002; Pelli & Farell, 1999). Each check was an increment or decrement sampled from a zero mean Gaussian distribution truncated at two SDs (Pelli & Farell, 1999). The power spectral density of a random checkerboard (with stochastically independent check luminances) equals the product of contrast power c²_rms and the area of a noise check. The white noise was high- or low-pass filtered at one of 26 cut-off frequencies: 0, 0.125, 0.25, …, 2.875, 3, ∞ c/deg. MATLAB functions were used to fast-fourier transform the noise matrix, zero all the unwanted frequencies, and invert the fast-fourier transform. The signal, like the noise, was uniform within each check. This was achieved by computing the stimulus image at 1/2 of the final size and then expanding to the final size by pixel replication. 

Design

On each trial, the target (N, Z, or X appeared with equal probability at 1 of 8 random locations at 9° eccentricity. Distracters (Vs) occupied the remaining 7 locations. All letters had equal contrast, and were presented at non-cardinal locations, for which contrast sensitivity is similar (30°, 60°, 120°, 150°, 210°, 240°, 300°, and 330°; Cameron, Tai, & Carrasco, 2002; Carrasco et al., 2001). 

There were three cue conditions: (1) Central-neutral cue condition: On one third of the blocks, the cue was a black dot, subtending 0.8° of visual angle, that appeared at fixation. (2) Distributed-neutral cue condition: On another third of the blocks, 8 dots appeared 12.5° away from fixation, one dot adjacent to each possible target location. (3) Peripheral cue condition: In the remaining blocks, a single dot appeared, 12.5° away from fixation, adjacent to the actual target location. All 3 cues were informative regarding the time of display onset. Each observer completed 10 blocks (5 in low-pass and 5 in high-pass noise) at each cut-off frequency for each cueing condition. The order was counterbalanced. Each observer contributed 104 practice and 780 experimental data points. 

Procedure

Observers viewed the display binocularly at a distance of 57 cm. A small fixation cross (0.2° × 0.2°) was present at the center of the screen throughout the block, except when obscured by the central-neutral cue dot (Figure 1). Observers were instructed to fixate on this cross throughout the trial and to report the target identity by pressing the corresponding key (N, Z, or X) on the computer keyboard. Feedback for an incorrect response was given by a low-frequency tone. On each trial, the cue (40 ms) was followed by an interstimulus interval (60 ms) (i.e., a stimulus onset asynchrony of 100 ms), after which the target and distracters appeared for 40 ms.. Given that about 250 ms are needed for saccades to occur (Mayfrank, Mobashery, Kimmig, & Fischer, 1986), eye movements in response to the cue could not occur before stimulus offset, because it was only 140 ms after cue onset. 

Figure 1

 

A schematic representation of a trial sequence. In one third of the blocks, the target was preceded by a central-neutral cue (a dot in the center of the display), in another third by a distributed-neutral cue (a dot adjacent to each of the eight possible target locations), and in the remaining blocks by a peripheral cue (a single dot adjacent to the actual target location). Note that the eight noise patches were outlined in black to demarcate the locations.

Measuring channel tuning

Using the critical-band-masking procedure, we measured the tuning of the channel mediating letter identification. The sigmoidal functions in Figure 2A are the energy threshold for identifying a letter in low- or high-pass noise for one observer in the three cueing conditions. We assume that the energy threshold E is linearly related to the total power passed by the channel filter G(f) mediating letter identification (Majaj, Pelli, Kurshan, & Palomares, 2002; Solomon & Pelli, 1994):  

 

(1)

where E₀ is the energy threshold measured in the absence of noise (zero cut-off frequency for low-pass noise and infinity for high-pass noise), and f is the radial frequency. The filter gain is proportional to the derivative of the measured threshold  

 

(2)

 

 

(3)

The filter is assumed to have a parametric form; log power gain is a parabolic function of log frequency  

 

(4)

and its parameters are estimated by a maximum likelihood fit (Majaj et al., 2002). 

Figure 2

 

A. Energy thresholds in low-pass (x; solid line) and high-pass (o; dashed line) noise for one observer (AMG) for the three cueing conditions. Each data point represents the energy threshold of the observer when low- or high-pass noise with a given cut-off frequency was superimposed on the letter. In this sigmoidal function, the steepest rise or fall in the threshold energy occurs at the noise frequency that interferes most. Because energy is proportional to squared contrast, an energy ratio of 1/2 implies a contrast ratio of 1/√2. B. Graph of the channel derived from the results plotted in A. The solid and dashed curves in each graph represent the channel used by this observer to identify letters in the low- and high-pass noise conditions, respectively.

Channel switching and noise additivity

Patterson and Nimmo-Smith (1980) reported that in a listening task, subjects could switch channels to avoid noise and perform with a lower threshold, a process that they termed off-frequency listening. Correspondingly, observers might look off-frequency in order to use the noise-free part of the spectrum to reduce their thresholds (Pelli, 1981). When low-pass noise is superimposed on a broadband stimulus (e.g., a letter), an ideal observer would use the noise-free high-frequency region of the signal spectrum to perform perfectly. Correspondingly, when high-pass noise is superimposed on this stimulus, the ideal observer would utilize the noise-free low-frequency end of the signal spectrum to maintain perfect performance. This is called off-frequency looking or channel switching (Majaj et al., 2002; Pelli, 1981). 

To assert that the observers utilized a single channel, we must make sure that there was no channel switching (i.e., the low- and high-pass noises are additive). Two noises are said to be additive if their sum leads to a threshold energy elevation that is equivalent to the sum of the threshold energy elevations yielded by each noise alone. If observers switch channels to utilize the noise-free part of the signal spectrum to perform the task, noise additivity would be violated. To detect any such violations, Majaj et al. (2002) used the “noise additivity ratio”  

 

(5)

where E⁺ = E − E₀ is the threshold elevation produced by the display noise, and the subscript indicates the type of noise. For any given cut-off frequency, the high- and low-pass noises are complementary in that their sum is white noise. Therefore, a noise-additivity ratio of 1 indicates that the observer uses the same channel to perform the task, whereas a ratio considerably below one would be indicative of channel switching (Majaj et al., 2002). The noise additivity ratio for people detecting narrowband stimuli such as sine-wave gratings is approximately 0.7 (Pelli, 1981; Solomon & Pelli, 1994). With broadband signals, whose spectra extend beyond the cut-off frequencies of the low- and high-pass noise, an observer free to choose any channel should use the noise-free part of the signal spectrum to identify the signal, attaining a very low threshold in either high- or low-pass noise. However, this strategy cannot avoid white noise, so the noise additivity ratio would be practically zero. Contrary to this prediction, a noise additivity ratio of 0.7 is found for human observers, which is too near 1 to be taken as evidence for channel switching (Majaj et al., 2002). 

Results

Covert attention and threshold energy

Figure 3A plots the ratio of the energy threshold with a peripheral cue to that with a central-(orange) or distributed- (yellow) neutral cue. A ratio of 1 would indicate that the peripheral cue had no effect on threshold. The results show that the thresholds attained in the peripheral cue condition were 0.5 to 0.63 times those with central- and distributed-neutral cue conditions. Thus, the peripheral cue halved the observers’ energy thresholds. 

Figure 3

 

A. The benefit of informative cueing. The ratio of the threshold energy with a peripheral cue to that with a central- (orange) or distributed- (yellow) neutral cue. A ratio of 1 would indicate no effect of a peripheral cue to the location of the target. The error bars represent ±1 SD across all noise cut-off frequencies. B. Average threshold energy in each cue condition for each observer.

Figure 3B plots the average energy threshold for each observer under the three different cueing conditions. Observer thresholds in the peripheral cue condition were significantly lower than those in the central- and distributed-cue conditions (p < .05). 

Covert attention and channel peak frequency

Figure 2A plots the energy thresholds for one observer in the three different cueing conditions. Each data point represents the energy threshold of the observer for a letter in low- or high-pass noise with a given cut-off frequency. The steepest part of this sigmoidal function occurs at the noise frequency that interferes most with the task. The derived channel mediating this task is plotted in Figure 2B. The two curves in each graph represent the filter gain of the channel used by this observer in the low- and high-pass noise conditions. The low- and high-pass estimates of the channel are independent, so their agreement demonstrates reliability of the measurement. 

Figure 4 shows that peak frequency was unaffected by cue condition for all three observers. The average ratio of the channel peak frequency with a peripheral cue to that with a central- or distributed-neutral cue is not significantly different from 1 (p > .1), indicating that the observers used the same channel to perform the task under the three different cueing conditions. 

Figure 4

 

Peak channel frequency in each condition for each observer.

Covert attention and channel bandwidth

We report bandwidth in octaves as log base 2 of the high-frequency limit divided by the low-frequency limit. These two frequencies correspond to those at which the power gain of the channel is half maximum (Majaj et al., 2002). Figure 2B shows the channel mediating the letter identification task. Figure 5 shows that none of the observers show an effect of cue condition on channel bandwidth. Moreover, the average ratio of the channel bandwidth (in octaves) with a peripheral cue to that with a central- or distributed-neutral cue did not differ significantly from 1 (p > .1), indicating that the channel’s bandwidth is independent of the cueing condition. 

Figure 5

 

Channel bandwidth in each condition for each observer.

There are no systematic differences between the high- and low-pass conditions with regard to the bandwidth of the spatial frequency channel mediating the task. The average bandwidth of 1.6 octaves ± 0.3 (mean ± SD) for letters is consistent with previous estimates for letters and gratings (Majaj et al., 2002; Solomon & Pelli,1994). 

Covert attention and channel switching

Figure 6 depicts the peak channel frequencies for each of the three observers in the cueing conditions for high- and low-pass noise conditions. As in prior work (Majaj et al., 2002), for two of the three observers, the estimated channel frequency was slightly higher in the presence of low- than high-pass noise, but, as in that work, the small change in frequency was insufficient to improve threshold much, and thus we do not take it as evidence of channel switching. The third observer, AMG, showed no difference in the channel frequency revealed by low- and high-pass noise. 

Figure 6

 

Peak channel frequency in each condition for both low- and high-pass noise for each observer.

Figure 7 illustrates the average noise additivity ratios for the three observers in the three conditions. None of the ratios differed significantly from 1, confirming that there was no channel switching. 

Figure 7

 

Noise additivity ratios for each cue condition for each observer. The error bars represent ±1 SD across all noise cut-off frequencies.

Our experiments were designed to investigate whether the peripheral cue increases sensitivity by changing the channel tuning. Changing channel tuning (e.g., by switching channel) to avoid the noise would improve sensitivity, but Figures 6 and 7 show no effect of cueing condition on channel tuning. There is no channel switching under any condition. 

Discussion

Using a critical-band-masking paradigm, we examined whether covert attention affects the spatial frequency channel mediating letter identification. Directing covert attention to the target’s location halves the observer’s energy threshold (1/√6 contrast). Contrary to both of our hypotheses, there is no change in the tuning of the channel that mediates the task: Neither center frequency nor bandwidth is affected.1 

Increased sensitivity for letter identification is consistent with previous psychophysical studies showing enhanced sensitivity for sinusoidal gratings (Cameron, Tai, & Carrasco, 2002; Carrasco et al., 2000; Carrasco, Talgar, & Cameron, 2001; Lu & Dosher, 1998) and with neurophysiological findings indicating that attention increases the effective contrast of the attended stimulus by a factor of 1.5 (Reynolds et al., 2000; Treue & Maunsell, 1996). We find factors of and in contrast for the neutral- and distributed-neutral cue, respectively. This finding of no effect of attention at the first stage of filtering is consistent with a texture segmentation study in which covert attention was shown to operate at the second stage of filtering (Yeshurun & Carrasco, 2000) and with a motion study in which attention was shown to affect the third stage of filtering (Sperling & Lu, 1998). 

There are four ways (a-d) that one might attempt to explain the attentional enhancement of contrast sensitivity by the peripheral cue, but only the last two (c and d) are consistent with our results:

•  
  A head start in processing at the target location in the peripheral cue condition. It is likely that any visual stimulus presented close to the target location will start activating detectors in that region in advance of the actual target presentation. Were this the source of the peripheral cue advantage over the central cue, then the distributed cue — in which a cue dot appeared next to each possible target location — should have a threshold much closer to that of the peripheral cue than that of the central cue. Instead, the distributed-cue threshold is much closer to that of the central cue, rejecting this explanation.
•  
  A narrowing of the spread of attention in the central-neutral cue condition. The benefit of the peripheral cue relative to the central cue may be simply due to the central cue drawing attention to the center of the display, away from the target (Pashler, 1998). However, because the distributed cue condition presents no cue in the center, this explanation cannot account for the similar effect of both neutral cues. A similar result has been found for acuity (Carrasco et al., 2002).
•  
  External noise reduction. This hypothesis maintains that attention attenuates stimuli outside of the focus of attention (e.g., Dosher & Lu, 2000; Eckstein, 1998; Morgan et al., 1998; Palmer, 1994; Prinzmetal, Amiri, Allen, & Edwards, 1998; Shaw, 1980; Shiu & Pashler, 1994). Noise-limited models consider the effects of external noise arising from distracters and masks that may be confused with the signal because of spatial and temporal uncertainty (intrinsic and extrinsic) of target and distracter location. These models predict that performance will decrease with increasing set size (i.e., uncertainty) (e.g., Eckstein, 1998; Foley & Schwarz, 1998; Palmer, 1994; Pelli, 1985; Shaw, 1980).
   
  Each of the three cueing conditions had its own degree of uncertainty. For the peripheral cue condition, there was one relevant location. For the distributed cue condition there were eight. For the central cue condition, there is a question regarding the degree of uncertainty. The eight patches of noise were always highly visible. Assuming vast uncertainty would be inconsistent with the high visibility of the noise patches and the small difference in the effectiveness of the two cues. It seems more reasonable to assume that attention is restricted to just the eight patches, which are saliently outlined in black. This predicts little or no difference between the two different neutral cues, as we found.
•  
  Signal enhancement. Other studies support the finding that the attentional effect goes beyond the reduction of uncertainty and maintain that attention improves the quality of the stimulus representation (Cameron et al., 2002; Carrasco et al., 2000, 2002; Lu & Dosher, 1998; Muller et al., 1998; Posner, 1980; Yeshurun & Carrasco, 1999). The attentional increase in contrast sensitivity is independent of spatial uncertainty (Cameron et al., 2002), even when localization performance indicates that there is no uncertainty regarding target location (Carrasco et al., 2000). Likewise, with brief displays, cueing the target location improves performance more than predicted by a signal-detection model of spatial uncertainty (Carrasco et al., 2002; Morgan et al., 1998).

Eckstein, Shimozaki, and Abbey (2002) used the classification image paradigm to compare the template that observers use when their attention is directed to a possible target’s location to that when attention is directed elsewhere. The target was presented on a background of random visual noise. The average noise pattern present when the observer made false alarms is an estimate of the template used to detect the target. Finding that observers use the same templates under valid and invalid cueing conditions, the authors concluded that cueing the target location does not alter early visual processing at that location, which is consistent with our finding of unchanged tuning. Alas, it is hard to extrapolate to more complex tasks from their classification image result because that paradigm seems to be limited to extremely simple visual tasks, typically binary classification. Fortunately, critical-band masking can be applied to any task, allowing us to show here that attention does not affect frequency tuning in letter identification. 

Conclusion

Covert attention reduces the observer’s contrast threshold by a factor of √2 without altering the spatial frequency tuning of the channel mediating the letter identification task. 

Acknowledgments

This work was supported by NSF Grant BCS-9910734/HCP to MC, NIH grant EY04432 to DGP, and a Beatrice and Samuel A. Seaver Foundation grant to New York University for “Neuroimaging studies of emotional and attentional influences on cognition and perception.” We thank Marialuisa Martelli for her insightful comments, AnnaMarie Giordano and Sam Ling for their careful observations and contributions, and Miguel Eckstein for his helpful comments on the manuscript. 

Corresponding author: Cigdem P. Talgar 

Email: . 

Commercial relationships: none. 

Footnotes

Footnotes

¹ We cannot assess the peak gain of the channel directly. Increased gain would amplify both the signal and the noise, without affecting the signal-to-noise ratio, so threshold would be unchanged.