The effects of visuospatial attention measured across visual cortex using source-imaged, steady-state EEG

Abstract

How does attention alter neural responses? Decades of electrophysiological measurements in non-human primates as well as human EEG and fMRI studies have shown that spatial attention modulates firing rates across the visual cortex, but the computations that drive this process are still unclear. Further, while it is well known that attention affects perception, we have only a limited understanding of the link between attentionally driven changes in neural firing rates and subject performance. Here we used a novel human neuroimaging method to measure the effect of spatial attention on neural responses in V1, hMT+, hV4, and the intraparietal sulcus (IPS). Attention altered signals in different ways across the visual cortex: areas V1, hMT+, and IPS exhibited primarily response gain changes while hV4 showed contrast gain modulation. Signals in V1, hMT+, and IPS correlated with contrast detection performance suggesting that behavior can be predicted by population-level signals as early as striate cortex.

Introduction

Our visual environment is complex, and attending to some locations while ignoring others is crucial for reducing the amount of visual information to a manageable level. In this paper, we ask how attention changes neural responses in both early and later visual areas. It is commonly accepted that attention influences responses in higher level visual areas. However, while the functional magnetic resonance imaging (fMRI) literature in humans shows robust attentional modulation in V1 (Brefczynski & De Yoe, 1999; Buracas & Boynton, 2007; Gandhi, Heeger, & Boynton, 1999; Kastner, Pinsk, De Weerd, Desimone, & Ungerleider, 1999; Li, Lu, Tjan, Dosher, & Chu, 2008; Murray, 2008; Silver, Ress, & Heeger, 2007; Somers, Dale, Seiffert, & Tootell, 1999; Tootell et al., 1998), corresponding changes in electrophysiological activity have historically proved elusive. Some groups report that single-unit firing rate changes are absent (Luck, Chelazzi, Hillyard, & Desimone, 1997; Marcus & van Essen, 2002; McAdams & Maunsell, 1999), but other groups report attentional modulation of V1 cells (Chen et al., 2008; Herrero et al., 2008; Ito & Gilbert, 1999; McAdams & Reid, 2005; Motter, 1993; Roelfsema, Lamme, & Spekreijse, 1998; Thiele, Pooresmaeili, Delicato, Herrero, & Roelfsema, 2009). Mehta, Ulbert, and Schroeder (2000) indexed multi-unit activity using multi-electrode array recordings and defined a modulation index (MI) of attention. They reported a mean MI of 0.170 in V4, 0.101 in V2, and 0.0278 in V1 indicating little or no modulation in V1, although some individual recording sites in V1 show moderate to large moderations with MIs > 0.05. Both Chen et al. (2008) and Motter (1993) find that increased attention demand increases the attentional modulation of single units in V1. 

Similarly, many human studies, isolating V1 responses based on the timing and polarity of Event Related Potential (ERP) signals (Jeffreys & Axford, 1972), report robust attentional modulation in extrastriate cortex but no, or weak, effects in striate cortex (Clark & Hillyard, 1996; Di Russo, Martinez, & Hillyard, 2003; Luck et al., 1994; Mangun, Hillyard, & Luck, 1993; Martinez et al., 1999). EEG-based estimates of the timing of the earliest attentional signals vary. Some report mainly delayed modulation (Noesselt et al., 2002) attributable to modulation in a later feedback signal, but there is also evidence for attentional modulation in the earliest cortical responses attributed to V1 (Kelly, Gomez-Ramirez, & Foxe, 2008). Similarly, Poghosyan and Ioannides (2008) report attention modulation in the initial (presumably feedforward V1) responses using magnetoencephalography (MEG). 

One explanation for the discrepancies between human neuroimaging data and electrophysiology could be physiological differences in the source of the measured signal. fMRI measures blood oxygen level-dependent (BOLD) signals that may reflect average changes in presynaptic activity rather than the spike rate across an entire neural population (Devor et al., 2007; Logothetis, 2008). There is even some evidence that fMRI can measure hemodynamic changes unrelated to neural activity (Maier et al., 2008; Sirotin & Das, 2009—although see Kleinschmidt & Müller, 2010). Attentional effects on the BOLD signal can be modeled by a purely additive mechanism (Buracas & Boynton, 2007; Murray, 2008) or by an additive baseline increase combined with multiplicative contrast gain mechanisms (Li et al., 2008). Buracas and Boynton (2007) hypothesized that their data could be explained by non-spike-related activity or by a small baseline increase in the firing rate of whole populations of neurons. Either of these mechanisms might go undetected in electrophysiological measurements from single units but could, nevertheless, generate significant changes in metabolic activity. Psychophysics (Morrone, Denti, & Spinelli, 2004; Pestelli, Ling, & Carrasco, 2009) and visually evoked potentials measured with SSVEPs using standard EEG with no source estimation (Di Russo, Spinelli, & Morrone, 2001; Im, Gururajan, Zhang, Chen, & He, 2007) report multiplicative contrast gain of attention. Meanwhile, electrophysiological experiments indicate that attention causes additive gain changes in V1 (Thiele et al., 2009) and either multiplicative gain changes (Reynolds, Pasternak, & Desimone, 2000) or a mixture of additive and multiplicative gain changes in V4 (Williford & Maunsell, 2006). 

Convergent evidence from a number of sources points to the existence of attentional modulation in both low- and high-level visual areas, but the computational nature of this modulation is unclear. In this study, we measured neural signals in a steady-state visually evoked potential (SSVEP) paradigm using high-density EEG. We combined these measurements with fMRI retinotopic mapping data and electrical head models of individual subjects. This enabled us to estimate the average cortical current density in multiple separate cortical areas (V1, hV4, hMT+, and IPS) and to characterize the effect of attentional modulation of neural activity at the stimulus frequencies at different stimulus contrast levels within these areas. 

We found that neural responses were modulated by attention in all the visual areas we examined, including V1. This effect was contrast-dependent, but the precise nature of the modulation was different in different areas. Some areas exhibited primarily response gain type modulations while others exhibited contrast gain modulation. Since EEG measures electrical currents, rather than blood flow, this attentional modulation must be due to differences in neural activity. 

Finally, our EEG measurements can shed some light on the question of which areas underlie subjects' ability to detect small, transient contrast increments. We found that signals from areas V1, hMT+, and IPS were correlated with subjects' performance in this increment detection task, while signals from area hV4 did not predict behavior. 

Methods

Subjects

Fifteen healthy subjects (mean age 40.7 ± 12.7 years; range 23–66 years; 6 females) with normal or corrected-to-normal vision participated in the study. All subjects had extensive experience as subjects in psychophysical and EEG experiments. All subjects provided written consent, and the experimental protocol was approved by the human subjects Institutional Review Board of the Smith-Kettlewell Eye Research Institute. 

Visual detection task

SSVEP stimuli consisted of two 1.5 cycles per degree (cpd) Gabor gratings with a diameter of 8 degrees at 0, 5, 20, and 50% contrast and centered 7 degrees to the left and right of a central fixation point, which was ∼0.5 degree wide ( Figure 1A). The two gratings were modulated on and off at different temporal frequencies (10 and 12 Hz). This allowed us to separate neural responses to the individual gratings using a temporal frequency-domain analysis (Morgan, Hansen, & Hillyard, 1996; Müller, Malinowski, & Hillyard, 2003; Müller, Teder-Sälejärvi, & Hillyard, 1998; Regan, 1989). Both of the chosen frequencies have been found to be affected in a similar manner by the occipital–frontal network and show response increases due to visual attention (Ding, Sperling, & Srinivasan, 2006). 

Figure 1

 

Experimental task and regions of interest. (A) Stimulus layout. Two gratings (diameter of 8° of visual angle) are displayed with centers 7° to the left and right of a fixation point in the middle of the screen. During a 15-s trial, both gratings are on/off flickering with a temporal frequency of 10 and 12 Hz, to allow for frequency tagging of the resulting EEG signals. The fixation point contains a small arrow indicating the side to attend to (right in this figure). The experiment contains two different trial types: attend left or right while maintaining fixation. Black text in the figure is not shown to the subject. (B) Regions of interest (ROIs). The four left ROIs, V1 (red), hV4 (magenta), hMT+ (yellow), and IPS (blue) on an inflated cortical surface of the left hemisphere of a single subject. Light gray areas are gyri, and dark gray areas are sulci.

The experimental task required subjects to attend covertly to the left or right grating (cued by an arrow at the fixation point) while maintaining central fixation. Subjects were instructed to detect multiple small, brief (500 ms) increments in contrast (“target events”) at the attended location during each 15-s trial. We chose this task because it should be mediated by neurons tuned to the orientation of the stimulus Gabor (Itti, Koch, & Braun, 2000). In comparison, the task of detecting orientation changes, as employed, for example, by Yoshor, Ghose, Bosking, Sun, and Maunsell (2007) is best performed by neurons tuned away from the Gabor orientation. Attention to this off-axis-tuned neural population is not expected to cause significant modulation of the population-averaged SSVEP signal and this may explain the weak attentional effects reported in that paper. 

Each 15-s trial contained a random number of target events (three on average) to force the subjects to pay attention for the entire period. Target detection was indicated by a button press within a 1-s time window after target presentation. This allowed for target detection analysis. Psychophysical testing prior to the experiment was used to determine threshold contrast detection levels for each subject and Gabor “pedestal” contrast. The average target contrast increments were 1, 2, 8, and 17% contrast on their respective contrast pedestals of 0, 5, 20, and 50% contrast, resulting in a performance that was ∼50% correct on all conditions, i.e., the subjects detected about half of all the targets. Each subject took part in one experimental EEG session consisting of five blocks of 24 trials, and the eight possible trial types (four contrast pedestals each with two potential task locations) were randomly interleaved in each block. Subjectively, the subjects did not report any difference in attentional demand between grating contrasts during the experiments. 

Stimulus display

Stimuli were presented using a custom EEG stimulus presentation system (“PowerDiva Video”) running on a PowerPC MacG4 (Apple Computers, Cupertino, CA) driving a 19″ LaCie Electron Blue II CRT monitor (LaCie USA, Hillsboro, OR) running at 120 Hz. This system achieves sub video frame temporal precision during stimulus presentation and is photometrically and temporally calibrated using a photometer and photocell. Additional radiometric calibration was performed using a photospectrometer (OceanOptics USB 2000, Oceanoptics, Dumolin, FL). Contrasts here are reported as Michelson luminance contrast about a mean luminance of 42 cd/m ². 

EEG data collection

The electroencephalogram (EEG) was collected with 128-channel HydroCell “Sensor Nets” (Electrical Geodesics, Eugene, OR) that use silver chloride electrodes embedded in electrolyte-soaked sponges. The EEG was amplified at a gain of 1,000 and recorded with a vertex physical reference. Signals were 0.1 Hz high-pass and 50 Hz (Bessel) low-pass filtered and digitized at 600 Hz with a precision of 4 bits/ μV at the input. Following each experimental session, the 3D locations of all electrodes and three major fiducials (nasion, left and right pre-auricular points) were digitized using a 3Space Fastrack 3D digitizer (Polhemus, Colchester, VT). For all observers, the 3D digitized locations were used to co-register the electrodes to their T1-weighted anatomical magnetic resonance imaging (MRI) scans. 

Artifact rejection and statistical analysis of the EEG data were performed offline. First, onset transients were contained within additional stimulus-driven “prelude” periods that were automatically excluded from the final data set and were not considered to be part of the 15-s steady-state “trial” period. Next, the raw data were evaluated using a sample-by-sample thresholding procedure to remove noisy sensors. These data points were replaced by the average of the six nearest spatial neighbors. Additionally, EEG bins that contained a large percentage of data samples exceeding an absolute amplitude threshold (∼25–50 μV) were marked for exclusion on a sensor-by-sensor basis. These noisy epochs were not considered in subsequent analysis steps. We note that both blinks and saccades generate strong, transient EEG artifacts and our noise-rejection procedure eliminates bins where subjects made gross fixation errors. Once noisy sensors were substituted and artifactual epochs excluded, FFT parameters for each stimulus condition were computed over each of the 30 × 0.5 s non-overlapping temporal bins within each 15-s experiment. Khayat, Sekreijse, and Roelfsema (2006) have shown that spatial attentional modulation begins around 150 ms after the attentional switch but can be sustained at a relatively constant level thereafter up to at least 800 ms. Müller et al. (1998) report an even slower onset, 450–650 ms, of attentional modulation after an attentional shift. We note that Yoshor et al. (2007) chose an integration period of 40–250 ms after stimulus presentation, a window that may have included a significant period of relatively unmodulated neural activity. 

Head conductivity and geometry models

As part of the source-estimation procedure, head tissue conductivity models were derived for each individual from T1- and T2-weighted MR scans at a down-sampled resolution of 1 × 1 × 1 mm (originally 0.82 × 0.82 × 0.9 mm). Boundary element models were computed based on compartmentalized tissue segmentations that defined contiguous regions for the scalp, outer skull, inner skull, and the cortex. To begin, approximate cortical tissue volumes for gray and white matters were defined by voxel intensity thresholding and anisotropic smoothing using the Fast routine from the FSL package ( http://www.fmrib.ox.ac.uk/fsl). The resulting white matter tissue boundaries were used to extract the contiguous cortical gray matter surface. These approximate segmentations were then used as a starting point for the anatomical segmentation procedure in Freesurfer V4.0 ( http://surfer.nmr.mgh.harvard.edu). This package used an iterative mesh-fitting procedure to generate topologically correct estimates of the white matter surface, the pial surface, as well as the inner and outer skull boundaries and the scalp. We used a surface midway between the white matter and the pial surface as our boundary element model cortex. 

Finally, all tissue surface tessellations were visually checked for accuracy to assure that no intersection had occurred between concentric meshes. Co-registration of the electrode positions to the MRI head surface was done by alignment of the three digitized fiducial points with their visible locations on the anatomical MR head surface using a least-squares algorithm in Matlab and electrode deviations normal to the scalp surface were removed. 

Cortically constrained minimum norm source estimates

Cortical current density (CCD) maps were estimated for each time point using an inverse of the electrical “forward model” produced by the MNE software package ( http://www.nmr.mgh.harvard.edu/martinos/userInfo/data/sofMNE.php). The forward model computes the contribution to the scalp voltage distribution of each member of an array of approximately 20,000 dipoles distributed across gray matter and oriented perpendicular to the cortical surface. The L2 or “minimum norm” inverse inverts this mapping to generate smooth maps of cortical activation that are consistent with the recorded scalp data while having the minimum total power (Hämäläinen & Ilmoniemi, 1994). Because the CCD time series are generated from a linear transform of the electrode time series, they contain all the frequency and time information present in the original recordings between 0.1 and 50 Hz. The spatial accuracy of this CCD estimation method is on the order of 10 to 20 mm (Im et al., 2007; Liu Dale, & Belliveau, 2002). We discuss the implications of the accuracy of the CCD on the results in detail in the Supplementary material. 

Definition of regions of interest (ROIs)

For all observers, functional magnetic resonance imaging (fMRI) scans were collected on a 3T Siemens TIM Trio scanner located at the UCSF Neuroscience Imaging Center using a 12-channel whole-head coil and standard Siemens EPI functional imaging sequences with a resolution of 1.7 × 1.7 × 2 mm. The general procedures for these scans (head stabilization, visual display system, etc.) are standard and have been described in detail elsewhere (Appelbaum, Wade, Pettet, Vildavski, & Norcia, 2008; Wade & Rowland, 2010). Retinotopic field mapping produced regions of interest (ROIs) defined for each participant's cortical area V1 and hV4 in each hemisphere (Biswal, Deyoe, & Hyde, 1996; Engel, Glover, & Wandell, 1997; Tootell & Hadjikhani, 2001; Wade, Brewer, Rieger, & Wandell, 2002). ROIs corresponding to each participant's MT homologue “hMT+” were identified using low-contrast motion stimuli similar to those described by Huk, Dougherty, and Heeger (2002). An ROI for the intraparietal sulcus (IPS) was determined anatomically as a circular ROI with a diameter of 10 mm on the cortical surface centered on the gray matter closest to the Talairach coordinates for retinotopically defined IPS2 (±19, −75, 48) from Silver, Ress, and Heeger (2005). The four ROIs are separated by at least 20 mm on the cortical surface (Figure 1B). 

ROI-based signal extraction

Our fMRI-defined regions of interest were mapped to the same cortical meshes that we used in our source imaging procedure. We were therefore able to average cortical current density (CCD) time courses from all mesh points falling within areas V1, hV4, hMT+, and IPS in each individual subject. The time-domain averages were then converted to complex-valued frequency spectra via a discrete Fourier transform in Matlab (The MathWorks, Natick, MA). 

Data were analyzed in discrete, non-overlapping 500-ms bins. Each 15-s EEG trial contained 30 bins resulting in ∼400 bins for each condition and contrast per subject (450 bins minus around 50 bins that contained target presentation or EEG artifacts, e.g., eye blinks or saccades). The frequency spectra of the CCD time courses were evaluated at frequencies uniquely attributable to the first harmonic of the grating modulation frequencies (10 and 12 Hz, corresponding to 5 and 6 cycles per 500-ms bin). 

Frequency-domain data from the same subject were averaged coherently—that is to say in the complex domain. The scalar amplitude of this complex mean was then averaged incoherently across subjects to derive the group-level mean responses in each region of interest. 

Coherent (complex or phase-sensitive) signal averaging within subjects is conceptually similar to computing the average of the evoked time series. It is a powerful technique for detecting weak signals in noise because the response latencies of stimulus-locked signals are relatively constant from trial to trial. True evoked responses add in-phase while signals that are not phase-locked to the stimulus tend to sum to zero in this procedure. Coherent averaging therefore removes both noise and endogenous neural activity such as gamma-band modulations very effectively. 

Incoherent averaging was used for cross-subject averaging as response latencies differed between subjects so significant signal cancellation would have occurred if we averaged the complex response components. Instead, we computed the magnitudes of the components of interest in the per-subject responses and then averaged these scalar values. This technique is conceptually similar to computing a cross-subject statistic on the amplitude of a single event-related potential peak or trough. 

Our processing strategy was consistent across all subjects and conditions. In particular, for the zero-contrast bins where no gratings were visible, the reference frequency was still that of the nominal (invisible) grating. In other words, in the case where the subject performed contrast increment detection tasks at the location of a 0% contrast, 10-Hz grating, we analyzed the corresponding frequency-domain signal assuming a 10-Hz response. 

Neurons in visual cortex are driven most strongly by stimuli in the contralateral visual field. We therefore expected our laterally separated stimuli to project to different hemispheres, so we analyzed responses from each hemisphere separately. The complex amplitudes of the attended frequency components in the hemisphere contralateral to the attended gratings were averaged to determine the amplitude of the attended response. Similarly, the unattended frequency components from the hemisphere contralateral to the ignored gratings were averaged to determine the amplitude of the ignored response. For example, if the subject attended to the left, 10 Hz, grating, the 10-Hz component of the SSVEP in the right side of the cortex was the “attended” response, and the 12-Hz component of the SSVEP in the left side of the cortex was the “ignored” response. 

Trials were randomized, so that in some trials the 12-Hz component was attended and in others the 10-Hz component was attended. Responses at different frequencies were separated in the initial complex averaging procedure, but their scalar amplitudes were combined in later analysis steps. In this way, each temporal frequency (10 or 12 Hz) contributed to both attend and ignore conditions, and any small difference in cortical frequency tuning preference and 1/ f power of signal transmission were averaged out. Importantly, bins containing a target event (a small contrast modulation) were left out of the attention versus ignore analysis to avoid the risk of contamination of the contrast response function. 

Finally, a separate analysis was performed only on the target bins. These were divided into two groups corresponding to target detection “hits” and “misses,” and the same type of frequency-domain amplitude analysis as described above was performed on each group. 

Statistical analysis

In order to avoid the assumption of normally distributed data that may not have been appropriate for our source-imaged EEG response amplitudes, we used a bootstrap procedure (Efron & Tibshirani, 1993) to generate confidence intervals and statistical significance of the differences of the parameters. One thousand bootstrap samples of the population mean were made over the 15 subjects to determine the standard error of the mean (SEM) of the amplitudes at each contrast for “attend conditions”, “ignore conditions”, and the difference between the two. (A single bootstrap sample was made by sampling data values with replacement from all 15 subjects in the study.) Using these 1,000 samples, 1,000 fits to the contrast response function (see Results section) were made to determine the SEMs of R_max, c₅₀, and a, with a fixed n = 1.4 (Busse, Wade, & Carandini, 2009), for attend and ignore (fitted individually, “attend” was not bound by, in any way, the parameters of “ignore”) using a Nelder–Mead simplex method (Nelder & Mead, 1965). We corrected for multiple comparisons over the four cortical areas using the false discovery rate (FDR) method (Genovese, Lazar, & Nichols, 2002). All analysis code was written in Matlab. Analysis software and sample data are available upon request. 

Results

Experimental overview

We characterized the effect of spatial attention by measuring its effect on the contrast response functions (CRFs) measured using source-imaged EEG in four visual areas. The regions of interest (ROIs) were chosen to address a range of questions. As described above, V1 was of particular interest as there are discrepancies in the attentional responses reported in this area in different studies. In addition, macaque areas V4 and MT have been studied extensively with respect to the role of attention (V4: Desimone & Duncan, 1995; Fries, Reynolds, Rorie, & Desimone, 2001; Luck et al., 1997; McAdams & Maunsell, 1999, 2000; Mitchell, Sundberg, & Reynolds, 2007; Moran & Desimone, 1985; Motter, 1993; Reynolds & Chelazzi, 2004; Williford & Maunsell, 2006; MT: Martínez-Trujillo & Treue, 2002; Maunsell & Cook, 2002; Seidemann & Newsome, 1999; Treue & Martínez-Trujillo, 1999; Womelsdorf, Anton-Erxleben, Pieper, & Treue, 2006). Here we examine the responses in their human homologues hV4 (Wade, Augath, Logothetis, & Wandell, 2008; Wade et al., 2002) and hMT+ (Huk et al., 2002; Tootell et al., 1995). Finally, the intraparietal sulcus (IPS) is of interest as it is widely regarded as an integral part of the visual attention control pathway (Bressler, Tang, Sylvester, Shulman, & Corbetta, 2008; Colby & Goldberg, 1999; Corbetta & Shulman, 2002; Lauritzen, D'Esposito, Heeger, & Silver, 2009; Mesulam, 1999). Spatial separation was also a consideration in our choice of ROIs. Cortical current density estimates have limited spatial resolution and so the estimate for a given ROI may be contaminated by activity occurring in neighboring locations. For this reason, the ROIs that we consider here are well separated from each other and the crosstalk between them is very small (see Supplementary material for a complete discussion). 

We asked subjects to attend covertly to one of two sine-wave gratings presented simultaneously 7° to the left and right of a central fixation point ( Figure 1A). The gratings were presented within circular windows 8° in diameter and varied in contrast (0, 5, 20, 50% contrast) from trial to trial. Each grating was modulated in an on/off sequence at either 10 Hz (left) or 12 Hz (right) to generate unique “frequency tagged” responses in the EEG signal (Regan, 1989; Supplementary Figure 1). This guaranteed that we could isolate and analyze response components corresponding to each stimulus grating separately. 

Each trial lasted 15 s, and during this period, subjects were instructed to detect short (500 ms), low amplitude contrast increments in one of the two gratings. Subjects indicated a target event by pressing a button up to 1 s after a target had appeared. Target event contrast increments were determined for each subject before the experiment in order to ensure a correct response rate of ∼50% for each of the four contrasts ( Figure 2). This ensured that the task was engaging enough to maintain a high and constant level of spatial attention across different stimulus contrasts. To ensure subjects could perform the task while maintaining fixation, we repeated the experiment in an eye tracker setup ( Supplementary material). 

Figure 2

 

Fraction of detected targets. The population detection rate is similar for all four contrasts. Error bars are ±1 SD over the population.

Using the source imaging procedure described in the experimental procedures, we were able to extract data from well-defined cortical regions of interest (ROIs) in each of 15 subjects. 

Finally, we fit hyperbolic ratio functions to responses from each ROI in both the “attend” and “ignore” conditions separately to determine how the different parameters of the underlying contrast response function were affected by attention. We also compared the responses in each area with performance on the contrast detection task to identify areas whose activity predicted behavior and perception. 

Contrast and response gain theory

Our data were interpreted within a contrast normalization framework. Neurons in the early visual system typically fire more as the contrast of their driving stimulus rises but saturate at high contrasts because their responses are subject to normalization (Heeger, 1992). For both single cells and populations of neurons, the neural response as a function of contrast is well described by a hyperbolic ratio function of the form (Albrecht & Hamilton, 1982) 

$\mathrm{C}\mathrm{R}{\mathrm{F}}_c=R_{\max }\frac{c^n}{c_{50}^n+c^n}+a,$

(1)

where c is the input contrast, R_max is the maximum amplitude, c₅₀ is the contrast for which the response is half of the maximum response amplitude, a is the baseline response, and n is the overall responsiveness. In this study, n was set to 1.4 based on results from a previous SSVEP study from this laboratory using very similar methods (Busse et al., 2009). At the level of a single neuron, visual attention can modify this function in three distinct ways. Additive gain is defined as an increase in the baseline activity a (Figure 3A). “Response gain” corresponds to an increase in R_max. The effect of response gain is to scale all response values of the contrast response function linearly and its effect is most noticeable at high-contrast levels where the output (and therefore the response modulation) is largest (Figure 3B). Finally, contrast gain is defined as a multiplicative change of the apparent input contrast that can be expressed as a modulation of the c₅₀ parameter (Heeger, 1992). Contrast gain control has its maximum effect at intermediate contrast levels where the response function is changing most steeply (Figure 3C). The aim of this study was to identify which gain control mechanisms best described the attentional modulations in each of the visual areas we examined. We analyze our population data in two ways: To look for multiplicative gain changes, we examine the R_max and c₅₀ parameters in model fits to coherently averaged responses. This fitting procedure is very sensitive to changes in these parameters but is theoretically blind to additive changes in baseline firing rate. Detecting additive changes in baseline firing rates or membrane potential is not possible using our paradigm, but we are able to detect changes in endogenous activity. We examine this issue in a second analysis where we perform phase-incoherent averaging at our stimulus input frequencies. 

Figure 3

 

Cartoon illustrating additive, response and contrast gains. The left column is the contrast response function for ignored (gray) and attended (black) stimuli. The right column is the difference of attended and ignored stimuli. (A) With additive gain, the response difference is constant across different stimulus contrasts. (B) Response gain results in a response difference that increases with contrast. (C) For contrast gain mechanisms, the difference is maximal at medium levels of contrasts before decreasing as responses saturate.

Attentional modulation of stimulus-driven responses

Allocating spatial attention to a target location increased the amplitude of the signal generated by the flickering grating at that location. This effect was present in the average scalp-level EEG data (see Supplementary Figure 2) and was present in all the visual areas studied using our ROI-based source imaging procedure ( Figure 4). To identify the mechanism underlying this increase, we fitted contrast response functions to the responses for both attend and ignore conditions in all four cortical areas. 

Figure 4

 

Fits of the hyperbolic ratio model of the contrast response function to the response amplitudes for attend and ignore. The left column is the fit of the attend (black) and ignore (gray) amplitudes of the current density, and the right column is their difference. (A) V1, (B) hV4, (C) hMT+, (D) IPS. V1, hMT+, and IPS display response gain, while hV4 displays evidence of contrast gain. Error bars are ±1 SEM.

To determine the exact contributions of response gain and contrast gain modulations in each cortical area, we examined the differences in the R _max and c ₅₀ parameters in the contrast response fit. As expected, there was no significant baseline difference ( a) for any of the cortical regions ( Figure 5A)—confirmation that our analysis procedure was free from systematic bias. This is also evident from the fact that the response at 0% contrast is essentially the same for attend and ignore in all four brain areas ( Figure 4, left column). However, this does not rule out the possibility that attention amplifies non-stimulus-locked endogenous activity. In the absence of a driving visual stimulus, the neural response is not phase-locked in time as it is when the responses are driven by visible gratings. When the response modulation for each subject is calculated by averaging individual trials coherently, the lack of response modulation change could be because the phases are simply canceling out each other. To look for changes in the endogenous, non-stimulus-locked activity at the stimulus frequencies, we averaged the individual trials incoherently as opposed to coherently. Specifically, we averaged the scalar amplitude magnitudes of the Fourier-transformed response peaks rather than their complex-valued amplitudes. For all four cortical areas, there is no significant change in the 0% contrast response at the tagged frequencies between ignored and attended stimuli ( Figure 6). Finally, it is possible that attention modulates the endogenous, non-stimulus-locked activity at frequency bands remote from those we tagged. For example, during attention to high-contrast visual stimuli, the energy in the gamma frequency bands increases, and the energy in low temporal frequency bands decrease in macaque V4 (Fries et al., 2001; Mitchell, Sundberg, & Reynolds, 2009). 

Figure 5

 

Contrast response function parameter fit differences between attend and ignore. (A) None of the differences in the baseline parameter, a, are significantly different from zero, indicating the absence of additive gain. (B) The differences in the maximum response, R _max, is significantly higher than zero for V1, hMT+, and IPS, indicating that these areas display response gain with attention. (C) The c ₅₀ is significantly smaller than zero for hV4, indicating that hV4 displays contrast gain with attention. Error bars are ±1 SEM. * p < 0.05, ** p < 0.01, corrected.

Figure 6

 

Responses at 0% contrast for ignored (white) and attended (gray) stimuli for incoherently averaged stimuli. There is no significant difference in response for any of the five areas suggesting that attention has little effect on endogenous EEG responses. Error bars are ±1 SEM.

As indicated by inspection of the difference of fit curves in Figure 4, R _max is significantly higher for attended than ignored stimuli in areas V1, hMT+, and IPS ( Figure 5B). This suggests that attention modulates neural activity in these three areas by way of a response gain. Surface potentials of SSVEP attention responses measured with 10 ipsi- and contralateral electrodes reveal response gain changes with attention (Kim, Grabowecky, Paller, Muthu, & Suzuki, 2007) similar to our findings in these areas. Finally, hV4 has a significant decrease in c₅₀ with attention, indicating a contrast gain change (Figure 5C). 

To conclude, attention to a single spatial location results in a response gain change for V1, hMT+, and IPS, while it results in a contrast gain change for hV4. None of the areas display additive changes in the amplitudes of the endogenous non-stimulus-locked responses around the stimulus frequencies. 

Target detection

In recent years, there has been significant interest in imaging the neural substrates underlying target detection (Donner et al., 2007; Monto, Palva, Voipio, & Palva, 2008; Müller et al., 1998; Ress, Backus, & Heeger, 2000; Ress & Heeger, 2003; Sapir, d'Avossa, McAvoy, Shulman, & Corbetta, 2005; Shulman et al., 2003). To identify areas that are involved in the detection of attended targets, we sorted neural responses from each area for the half-second bins during which targets were presented. Bins were sorted according to whether the subject correctly identified the brief contrast modulation. Figure 7 shows the average neural response for correctly detected targets minus the neural response for missed targets. We found that V1, hMT+, and IPS, the same cortical areas that display response gain, show an increased modulation for hits compared to misses. Part of this could simply be that the areas inherit a signal, which is larger for target hits than misses. Area hV4 activity does not appear to be predictive of target detection despite presumably inheriting a signal from earlier predictive areas, such as V1. This task did not generate enough false positives to determine their relationship to cortical current amplitudes. 

Figure 7

 

Differences in average cortical current density amplitude for 0.5-s time bins with detected targets versus missed targets. Responses for V1, hMT+, and IPS are significantly larger than zero. Error bars are ±1 SEM, * p < 0.005 corrected.

Discussion

Using source-imaged EEG, we have shown that dynamic neural responses in cortical areas V1, hV4, hMT+, and IPS exhibit multiplicative gain changes as a result of covert spatial attention. Moreover, while V1, hMT+, and IPS display response gain changes, hV4 displays contrast gain changes. We note that while the three areas displaying response gain reach a response plateau, hV4 does not although its “attend” response is starting to saturate before its “ignore” response ( Figure 4B). One likely reason for this is the relative insensitivity of ventral surface population responses to higher temporal frequencies (Liu & Wandell, 2005). It is possible that stimuli that drive hV4 responses to saturate more completely could unmask a response gain mechanism as well as contrast gain mechanism. In addition, signal detection analysis reveals a higher response for detected than for missed targets in the cortical areas displaying response gain only. While the SSVEP cannot infer the causal direction of neural signals among different cortical areas, the findings agree with the common assumption that attention signals are mediated through feedback signals from the intraparietal sulcus (IPS) to lower visual areas as indicated by electrophysiology (e.g., Buffalo, Fries, Landman, Liang, & Desimone, 2010), ERP (e.g., Di Russo et al., 2003), and fMRI connectivity analysis (Bressler et al., 2008; Lauritzen et al., 2009). 

Attention modulation measured in early visual areas with fMRI is either described as additive (Buracas & Boynton, 2007; Murray, 2008) or as a combination of additive and multiplicative gain control mechanisms (Li et al., 2008). Buracas and Boynton (2007) suggested that spatial attention acts to marginally increase the baseline firing rates or presynaptic activity of all neurons in the attended region of the cortex irrespective of their tuning. Such change might be undetectable at the level of single units, yet could generate a large mean increase in metabolic demand that would affect the fMRI signal. Since SSVEP reflects EEG responses from neural populations whose responses are phase-locked to the stimulus (e.g., Baillet, Mosher, & Leahy, 2001), it is blind to purely metabolic or hemodynamic changes. Thus, the robust, attentionally driven signal modulations we report here cannot be the result of baseline changes of this type. 

The multiplicative signal increases we find with our SSVEP paradigm are a result of a modulation of the stimulus-driven responses at the grating flicker rates (10 Hz and 12 Hz). The nature of our measurement technique and the high noise levels present at low temporal frequencies in our EEG data sets preclude us from identifying small shifts in the baseline neural activity. We are therefore unable to rule out these changes as a possible explanation for the additive signal changes seen by Buracas and Boynton (2007), Li et al. (2008), and Murray (2008). We can, however, rule out a contrast-independent, additive increase in the amplitude of endogenous neural responses in the frequency range covered by our stimuli. 

Attention modulation measured with fMRI tends to increase with the visual hierarchical level (Kastner, De Weerd, Desimone, & Ungerleider, 1998; Silver et al., 2005; Tootell et al., 1998). Although we note that the estimated attention modulation is statistically significant for all four cortical areas, in this study, the attention modulation in hV4 and IPS is smaller than that observed in V1. This may be because our source imaging methods preclude us from identifying additive increases as well as other increases not phase-locked to the tagging frequencies. Such modulations have been shown to be a significant factor in fMRI measurements of attentionally driven BOLD responses and it appears that relative and absolute magnitudes of these additive effects increase in the extrastriate cortex (Buracas & Boynton, 2007; Li et al., 2008). 

Attention modulation in human compared to primate electrophysiology

Robust attention modulation in primates has been measured in extrastriate visual areas (Desimone & Duncan, 1995; Fries et al., 2001; Luck et al., 1997; Marcus & van Essen, 2002; McAdams & Maunsell, 1999; Mehta et al., 2000; Moran & Desimone, 1985) as well as in V1 cells, or at least in subpopulations. For example, McAdams and Reid (2005) reported increased response in 12, and decreased response in 1, out of 45 studied simple cells in macaque V1. Motter (1993) similarly found that 35% (34/96 cells) was significantly modulated by attention, 24/34 with increased activity, in V1. Further, increasing the attention demand by adding more stimuli increased this number to 43% of the V1 neurons he studied. In addition, silicon multi-electrode studies reveal that the laminar current source density (CSD) is modulated by attention at some cortical layers (Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; Mehta et al., 2000; Monto et al., 2008; Schroeder & Lakatos, 2008). In this study, we find robust multiplicative gain modulations in all four areas. This is in agreement with single-unit findings in V4 (Reynolds et al., 2000), while Thiele et al. (2009) report mainly additive gain changes in V1. 

The discrepancies between humans and non-human primates could be due to fundamental neural differences in attentional mechanisms between the different species. A more likely explanation is that both EEG and fMRI measure average responses from large numbers of cells while electrode recordings target specific subgroups of cell populations or laminar structures. This is supported by the fact that McAdams and Reid (2005) only report a subpopulation of simple cells displaying attention modulation. In general, single-unit recordings are usually biased toward larger neurons (Olshausen & Field, 2005), which potentially show additive gain modulation by attention in V1, as found by Thiele et al. (2009). 

While fMRI activity is clearly correlated with neural activity, the exact neurophysiological correlates of the BOLD signal have not yet been determined (reviewed in Logothetis, 2008). For example, an increased shunting inhibition caused by increased inhibitory synaptic activity would increase intracellular currents. These currents would result in an increased EEG signal and would increase the metabolic demand of the cortical tissue, presumably increasing the BOLD signal. However, in this scenario, the electrophysiological signal, as measured with single electrodes, might not be changed significantly. 

Attention modulation in ERP signals isolated based on waveform dynamics

Our goal in this paper was to study the mechanisms, not the existence of early attentional processes, but it is clear from our data that signals in V1 exhibit robust changes in responses to spatial attention. 

Previous ERP studies in humans tended to identify the cortical location of responses based on two factors: peak response timing (early responses are likely to be generated by early visual areas) and waveform dependence on visual field position, the so-called “cruciform” hypothesis (Jeffreys & Axford, 1972). Most studies report attention modulation in extrastriate cortex but not in striate cortex (Clark & Hillyard, 1996; Di Russo et al., 2003; Luck et al., 1994; Mangun et al., 1993; Martinez et al., 1999) as observed in many single-unit experiments (Luck et al., 1997; Marcus & van Essen, 2002; McAdams & Maunsell, 1999). A few studies report attention modulation in striate cortex: Noesselt et al. (2002) report delayed attention modulation attributed to V1. Kelly et al. (2008) report attention modulation as early as the initial component C1, 57 ms after onset of the stimulus, indicating that it occurs at the earliest possible timing of cortical processing, and attributes that to V1. 

What might be the reason for the discrepancies between the V1 results in this study and previous results? Examining anatomical MRI scans, the ventral and dorsal parts of V1 are rarely symmetrical around the calcarine sulcus, so the actual scalp potential originating from V1 does not necessarily show the polarity reversal hypothesized by the cruciform model. Simulations using realistic forward models confirm this (Ales, Yates, & Norcia, 2010). It is therefore likely that polarity reversal alone is not a reliable method of identifying sources from V1. Relying on this criterion alone may exclude true V1 signals. For this reason, it may be that components with some V1 responses in the studies above do, in fact, exhibit the same robust attention-driven modulation that we find with our cortical current density estimates. It is intriguing that studies that combine the cruciform hypothesis with response timings in individual subjects to identify V1 responses do report V1 modulation with attention (Kelly et al., 2008; Poghosyan & Ioannides, 2008). 

Comparison with attentional gain control models

Recent models of attentional modulation have argued that discrepancies in a range of attention studies in macaque V4 and MT and in human V1 and V2 measured with fMRI can be accounted for with a single underlying gain control mechanism whose effect depends on a few parameters, particularly the size of the attended stimulus relative to the diameter of the attended area and the diameter of the receptive fields (RFs) in the cortical region (Boynton, 2009; Lee & Maunsell, 2009; Reynolds & Heeger, 2009). Specifically, if the stimulus is larger than the RFs and attended area, attention will predominantly show a response gain, but if the stimulus is smaller, attention will show a contrast gain. The visual stimuli in our study are 8° in diameter at an average eccentricity of 7° (3–11°). The population RFs at those eccentricities are <1° in V1 (Dumoulin & Wandell, 2008; Smith, Singh, Williams, & Greenlee, 2001) and the models of Boynton (2009) and Reynolds and Heeger (2009) predict that this combination of target and RF size should result in an attentionally driven change in response gain. This is what we observe. 

Population RFs have, to our knowledge, not been directly estimated at 3–11° eccentricities in hV4, hMT+, and IPS of human. Attention experiments estimate the RF sizes at 5.5° eccentricity to be 4–6° in hV4 (Bles, Schwarzbach, de Weerd, Goebel, & Jansma, 2006; Kastner et al., 2001). We can extrapolate the population RF sizes to be larger than 8° in hV4, at least at the high end of the eccentricities studied here. For these hV4 RF sizes, our data agree with the contrast gain modulation predicted by the normalization models. Neurons in higher cortical areas tend to have larger RFs, at least up to inferior temporal cortex (Bruce, Desimone, & Gross, 1981; Desimone, Albright, Gross, & Bruce, 1984). So we can expect that the RFs in hMT+ and IPS are larger than in hV4. Our findings that these two areas show attentionally driven response gain do not match the predictions of the normalization model of attention. One possible explanation of this discrepancy could be that hMT+ and IPS do not undergo additional attentionally driven normalization but, instead, simply inherit their attentional modulation from neural responses in areas with smaller RFs, for example, V1. 

Target detection

We found that signals in areas V1, hMT+, and IPS were significantly higher during epochs in which subjects detected the presence of a contrast increment target ( Figure 7) compared to epochs in which contrast increments of the same amplitude were missed. While previous EEG experiments show that cortical activity is correlated with target detection performance (Monto et al., 2008; Müller et al., 1998), our data are, to the best of our knowledge, the first to show this in a steady-state VEP paradigm that is directly relatable to the fMRI literature (Ress & Heeger, 2003). 

Ress and Heeger noted that response amplitudes for correct target detection and false alarms (incorrect target detections) were (a) roughly equal to each other and (b) both greater than the response amplitudes for misses and correct rejects. Thus, the response amplitude followed the subject's percept rather than the physical input contrast. In the visual areas noted above, we find a similar result (hits > misses or correct rejects) although our levels of false alarms were too small to complete a full signal detection analysis. We believe that the most likely explanation for variability in target detection performance is small, spontaneous changes in attentional state. As we show in this paper, spatial attention increases neural responses in all cortical areas studied and numerous studies have shown that spatial attention improves performance at the attended location (Braun, 1998; Carrasco, 2006; Cavanagh & Alvarez, 2005; Lu & Dosher, 2008; Sperling & Melchner, 1978; Verghese, 2001). If subjects' ability to deploy and maintain spatial attention varies over time, as it surely does, then this variability will be reflected both in the target detection performance and in the neural response amplitude. The fact that we see very few false alarm events (incorrect positive identifications of a target) suggests that the effects we see are not driven by bottom-up processes: Given an input with small, constant noise, a function that converts input to a firing rate via a threshold produces a firing rate with a large variance that grows with the mean (Carandini, 2004). If attention mainly served to increase the mean feedforward input, these stochastic modulations would be large enough to generate the signal changes measured here. They should also, logically, have driven equal numbers of “false alarms.” While these types of events were seen using more subtle contrast modulations in Ress and Heeger's (2003) study, they are essentially absent in our data. The absence of increased stochastic modulations, which would have increased the false alarm rates, is also supported by the recent findings that attention reduces neural noise (Mitchell et al., 2007). 

Despite showing statistically significant attentional modulation, signals in area hV4 do not reflect differences in target detection performance. However, hV4 appears to undergo primarily contrast rather than response gain control. Modulation amplitudes are therefore small at both low and high stimulus contrast levels (both ends of the sigmoidal contrast response function) and the combination of these effects may explain the diminished effect of brief attentional lapses in this area. 

Conclusion

Using source-imaged EEG, we find that all studied areas display significant, multiplicative gain changes in response to spatial attention. This general type of gain change agrees with multiplicative gain control models proposed by several groups including Boynton (2009) and Reynolds and Heeger (2009). 

We find no evidence for additive shifts in the level of endogenous neural activity close to our stimulus frequencies. Robust baseline shifts of this type are measured in fMRI and they must therefore result either from unmodulated mean shifts in baseline neural response amplitudes, changes in endogenous activity at higher frequencies, or purely hemodynamic changes that may be uncoupled from spiking activity (Devor et al., 2008, 2007; Sirotin & Das, 2009). 

Finally, response amplitudes predict target detection performance in some areas, including V1, hMT+, and IPS but not hV4. We hypothesize that the correlation observed between performance and signal modulation is due to a common underlying, top-down process: slow fluctuations in the magnitude of spatial attention. 

Supplementary Materials

Supplementary PDF - Supplementary PDF 

Acknowledgments

This work was supported by the Lundbeck Foundation (TZL), NIH Grants R01-EY017071 and R01-EY018157, and NSF Grant 820101001 (ARW). The authors would like to thank D. Taylor, L. Dang, L. Renninger, J. Rowland, P. Verghese, P. Ivanov, A. Norcia, and our anonymous reviewers for technical assistance, discussions, and comments on the manuscript. 

Commercial relationships: none. 

Corresponding author: Thomas Z. Lauritzen. 

Email: [e-mail]. 

Address: Second Sight Medical Products, Inc., 12744 San Fernando Rd., Bldg. 3, Sylmar, CA 91342, USA.