Attention selectively enhances stimulus information for surround over foveal stimulus representations in occipital cortex

Abstract

By attending to part of a visual scene, we can prioritize processing of the most relevant visual information and so use our limited resources effectively. Previous functional magnetic resonance imaging (fMRI) work has shown that attention can increase overall blood-oxygen-level-dependent (BOLD) signal responsiveness but also enhances the stimulus information in terms of classifier performance. Here, we investigate how these effects vary across the visual field. We compare attention-enhanced fMRI–BOLD amplitude responses and classifier accuracy in fovea and surrounding stimulus regions using a set of four simple stimuli subdivided into a foveal region (1.4° diameter) and a surround region (15° diameter). We found dissociations between the effects of attention on average response and in enhancing stimulus information. In early visual cortex, we found that attention increased the amplitude of responses to both foveal and surround parts of the stimuli and increased classifier performance only for the surround stimulus. Conversely, ventral visual areas showed less change in average response but greater changes in decoding. Unlike for early visual cortex, in the ventral visual cortex attention produced similar changes in decoding for center and surround stimuli.

Introduction

Early work using positron emission tomography and functional magnetic resonance imaging (fMRI) to investigate visual responses showed that voluntary spatial attention enhances the amplitude of blood-oxygen-level-dependent (BOLD) responses in the attended regions of the visual field and suppresses responses in unattended regions (Brefczynski & DeYoe, 1999; Gandhi, Heeger, & Boynton, 1999; Smith, Singh, & Greenlee, 2000; Somers, Dale, Seiffert, & Tootell, 1999; Tootell, Hadjikhani, Hall, Marrett, Vanduffel, Vaughan, & Dale, 1998). These effects occur not only in both dorsal and ventral extrastriate areas (Chawla, Rees, & Friston, 1999; Corbetta, Miezin, Dobmeyer, Shulman, & Petersen, 1990; McMains & Somers, 2005; Müller, Bartelt, Donner, Villringer, & Brandt, 2003; Tootell et al., 1998) but also in V1 (Brefczynski & DeYoe, 1999; Gandhi et al., 1999; Martínez et al., 1999; Somers et al., 1999), and they extend down to the thalamus (e.g., lateral geniculate nucleus [LGN]) (O'Connor, Fukui, Pinsk, & Kastner, 2002; Schneider & Kastner, 2009). These early observations demonstrated the widespread attentional modulation of stimulus-driven responses across the entire visual system and highlighted the need to carefully control for the effects of attention when measuring the influence of other variables. 

Changes induced by spatial attention include both additive and multiplicative shifts in response. Additive increases in response, or baseline shifts, are most clearly demonstrated when an attention-driven increase in BOLD response occurs even in the absence of visual stimulation (Kastner, Pinsk, Weerd, Desimone, & Ungerleider, 1999; Ress, Backus, & Heeger, 2000; Serences and Boynton, 2007; Silver, Ress, & Heeger, 2007), and they have also been shown to account well for changes in contrast response to simple stimuli (Buracas & Boynton, 2007; Murray, 2008). Multiplicative scaling of neuronal responses has also been demonstrated to result from spatial attention (Saproo & Serences, 2010; Serences, Saproo, Scolari, Ho, & Muftuler, 2009), as well as an increase in the stimulus selectivity of responses (Murray & Wojciulik, 2004). Multiplicative scaling of responses and increased stimulus selectivity should both lead to an increase in the signal-to-noise ratio of responses, and so increase the stimulus information carried by the response, whereas a baseline shift will not necessarily lead to an increase in stimulus-related information. Although increased BOLD amplitude does not necessarily mean there is an enhanced stimulus representation, such increases have often been interpreted as attention boosting the stimulus representation (e.g., McMains & Somers, 2005; Müller et al., 2003). 

There are surprisingly few studies that have used multivariate classification approaches to test whether attention-induced increases in BOLD responses are accompanied by increases in decodable stimulus-related information. Guggenmos, Thoma, Haynes, Richardson-Klavehn, Cichy, and Sterzer (2015) used classification analyses to demonstrate that responses in the lateral occipital complex area contained more information about the identity of object stimuli when the objects were attended. Jehee, Brady, and Tong (2011) found that spatial attention increased responses across the early visual cortex but only increased decodable information for task-relevant features. Conversely, attention-induced increases in information are not always accompanied by increased responses, as demonstrated by Cohen and Tong (2015) for object-based attention directed to one of two spatially overlapping stimuli. Here, we measure changes in BOLD amplitude and changes in decodable stimulus-related information, and we compare the effects of spatial attention on the central versus peripheral parts of the visual field. 

Another untested question is whether spatial attention produces similar increases in BOLD response and/or decodable stimulus information when directed to the foveal part of the visual field. Under natural viewing, attention is usually directed toward the location that the observer is foveating, but research into spatial attention has traditionally used designs where observers covertly attend to a peripheral location (Luck, 2009), including most fMRI studies (but see Smith et al., 2000). Behaviorally, attention varies with eccentricity, such that a range of tasks requiring attentional selection show an eccentricity effect (Carrasco, Evert, Chang, & Katz, 1995) in which performance is better for central compared to peripheral stimuli. Although a reduction in visual acuity and increased crowding with eccentricity may account for much of these effects (Carrasco et al., 1995; Carrasco & Frieder, 1997), they are not the full explanation, suggesting that, in addition to acuity and crowding variations, attention is biased toward the central visual field (Bao, Lei, Fang, Tong, Schill, Pöppel, & Strasburger, 2013; Staugaard, Petersen, & Vangkilde, 2016; Valsecchi, Toscani, & Gegenfurtner, 2013; Wolfe, O'Neill, & Bennett, 1998). 

Here, we address three related questions regarding how spatial attention changes the BOLD response and decodable stimulus information across visual cortex. First, we compare the effects of attention in the foveal and surround stimulus regions. Second, we ask whether these effects depend on the region of visual cortex, comparing attentional effects in early, ventral, and dorsal visual areas. Third, we compare these effects for different stimulus types, including a range of spatiotemporal frequencies and achromatic versus chromatic contrasts selected to be relevant for different cortical areas and regions. We previously used the same stimulus types to show distinct gradients of low-level stimulus preferences across the visual cortex (Goddard & Mullen, 2020). We found attentional effects on the average BOLD response across both the foveal and peripheral visual field. We also found that attention boosted decoding of stimulus type, and that these enhancements were greater for peripheral than central stimuli. 

Materials and methods

Participants

We collected fMRI data on 12 participants (eight female, four male; ages, 21–44 years). All participants were healthy with no history of neurological or psychiatric disorders, and they provided informed consent. Each participant had normal or corrected-to-normal visual acuity and normal color vision as assessed with Ishihara plates (Ishihara, 1990) and the Farnsworth–Munsell 100-hue test (Farnsworth, 1957). Both experiments were approved by the Ethics Review Board of the McGill University Health Centre and were conducted in accordance with the tenets of the Declaration of Helsinki. Data from fMRI experiments are freely available online from the Open Science Framework (doi: 10.17605/OSF.IO/HN537). This online repository includes deidentified raw data from the fMRI experiments, details of the stimulus timing for each participant, and the Analysis of Functional NeuroImages (AFNI) code used to perform the analyses reported here. 

Visual stimuli

All stimuli were radial sine-wave gratings, as used in previous work (Goddard & Mullen, 2020; Mullen, Chang, & Hess, 2015; Mullen, Dumoulin, McMahon, de Zubicaray, & Hess, 2007), but they were divided into center and surround subregions that were presented simultaneously. Each unique stimulus was one of four stimulus types in the center and another in the annular surround, as shown in Figure 1A. The total stimulus (center and surround) was presented in a circular region of 177 degs² visual angle, and the central region was 1.5 degs² visual angle. The four stimulus types used for the stimulus centers and surrounds match those used in our recent work demonstrating gradients of stimulus preference across visual cortex (Goddard & Mullen, 2020), except that in this previous work each stimulus was presented alone, rather than in a center/surround pair. 

Figure 1.

 

Visual stimuli. (A) Spatial layout: in all experiments, the visual stimuli included a small center with an annular surround presented simultaneously. (B) Each stimulus included a different stimulus type in the center and surround. Stimulus types were radial sine-wave gratings isolating the L-/M-cone opponent (red–green, RG), luminance (Achromatic, Ach), or S-cone (blue–yellow, BY) mechanisms, shown at higher contrast in these illustrations. The achromatic stimulus types were either low spatial frequency (0.5 cpd) and high temporal frequency (10 Hz), termed M-type, or high spatial frequency (5 cpd) and low temporal frequency (2 Hz), termed P-type. (C) Example time courses of a 2-Hz achromatic (upper) or 10-Hz achromatic (lower) stimulus block. Each 12-second stimulus block consisted of four trials of a contrast discrimination task. After each pair of stimuli, participants indicated whether the first or second stimulus in the pair was of higher contrast.

For each of the four stimulus types, contrast was achromatic (Ach), isoluminant red–green (RG) or blue–yellow (BY), modulated about a mean gray, calibrated to isolate the luminance, L-/M-cone opponent and S-cone opponent mechanisms, respectively. There were two stimulus types defined by achromatic contrast, with spatiotemporal parameters selected to produce greater stimulation in the magnocellular (Ach M-type) or parvocellular (Ach P-type) pathways. These spatiotemporal parameters were selected on the basis of LGN lesion studies in the macaque (see review by Merigan & Maunsell, 1993), which have shown that parvocellular lesions dramatically reduce behavioral contrast sensitivity to static stimuli of high spatial frequencies (∼98% reduction at 5 cycles per degree [cpd]), whereas magnocellular lesions produce marked loss of sensitivity to high temporal frequencies (∼90% reduction at 10 Hz). The RG, BY, and Ach M-type stimuli each had a spatial frequency of 0.5 cpd, whereas the Ach P-type stimulus had a spatial frequency of 5 cpd. The RG, BY, and Ach P-type stimuli had a 2-Hz sinusoidal contrast phase alternation, whereas the Ach M-type stimulus was modulated at 10 Hz. The low spatial frequency of the two chromatic stimuli reduces luminance artifacts generated by chromatic aberration for the chromatic stimuli (Bradley, Zhang, & Thibos, 1992; Cottaris, 2003; Mullen, 1985). 

Central stimuli included a whole number of spatial cycles, plus a half cycle around the fixation marker, and annular stimuli included a whole number of cycles. This meant that each unique stimulus had a smooth transition between center and surround and another smooth transition at its outer edge. Stimulus types with a spatial frequency of 0.5 cpd included a half cycle when presented in the center and five cycles when in the annular surround, whereas stimuli of 5 cpd included five cycles in the center and 50 cycles in the surround. A small fixation marker was displayed in the center of all stimuli (a black dot). Outside the stimulus area, the screen was at its mean luminance. 

Stimulus chromaticity was defined in a three-dimensional cone contrast space, with each axis representing the quantal catch of the L, M, and S cone types normalized with respect to the gray background (i.e., cone contrast). The vector direction and length within this space define chromaticity and cone contrast, respectively. We determined the isoluminance of the RG stimuli for each subject individually based on perceptual minimum motion settings, as previously described (Mullen et al., 2007; Mullen, Thompson, & Hess, 2010). We also verified the S-cone isolating direction within each participant's isoluminant plane by varying vector angle within a cone contrast space and selecting the direction of minimum visibility (Michna, Yoshizawa, & Mullen, 2007). 

We chose stimulus cone contrasts close to the maximum of the monitor gamut for the chromatic directions (4% for RG, 30% for BY, and 50% for Ach). Stimulus contrast is defined as the vector length in cone contrast units, using the Michelson contrast definition for the three cone axes. This metric differs by a factor of \(\sqrt{3}\) from Michelson contrast as conventionally applied to luminance stimuli. We chose high-contrast values to yield stimuli that were highly visible even at the smallest stimulus size and across all stimulus regions, with the aim of driving robust responses in the visual cortex. We have previously found that stimuli like these evoked similar amplitudes of BOLD response (Mullen et al., 2010). 

Display apparatus and calibrations

We displayed stimuli on a 32-inch BOLD screen liquid-crystal display monitor (resolution 1920 × 1080; Cambridge Research Systems, Rochester, UK). Participants viewed the BOLD screen, which was located at the rear of the MRI bore, through a mirror mounted on the head coil. The total viewing distance was 125 cm. We used a MacBook Pro (Apple Inc., Cupertino, CA) running MATLAB R2017a (MathWorks, Natick, MA) in conjunction with routines from Psychtoolbox 3.0 (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997) to generate the stimuli and draw them to the BOLD screen (refresh rate, 60 Hz; mean luminance, 52.4 cd/m²). The BOLD screen was linearized and color calibrated as described previously (Michna et al., 2007; Mullen, Dumoulin, & Hess, 2008). 

Experimental design

We acquired functional measurements of the BOLD response in a 1-hour session for each participant. Each session was comprised of six runs of 7 minutes, 18 seconds, and each run included 29 blocks. Each block commenced with 3 seconds of the fixation screen (blank gray screen of mean luminance, with fixation dot) followed by 12 seconds of a contrast discrimination task (described below) or 12 seconds of a reference stimulus (either a constant black screen or a constant gray screen). At the end of 29 blocks, there was a final 3-second fixation screen. The first block was always a gray reference stimulus, and a pair of gray and black reference stimuli occurred in the middle (blocks 14 and 15) and at the end (blocks 28 and 29) of each run, with the order of gray and black blocks within these pairs counterbalanced across runs. Each block of the contrast discrimination task included a single stimulus from the 12 unique stimuli shown in Figure 1. The 12 unique stimuli were presented in one block each across blocks 2 to 13 and blocks 16 to 27 of each run. The order of these stimuli was counterbalanced across runs such that every stimulus was preceded and followed by every other stimulus once across the session. The order of these counterbalanced runs varied across participants. 

During the 12-second periods of the contrast discrimination task, participants performed a version of a task used in previous work (Goddard, Chang, Hess, & Mullen, 2019; Goddard & Mullen, 2020; Mullen et al., 2007; Mullen et al., 2010; Mullen et al., 2015). We varied the participants’ attentional state by asking them to perform the contrast discrimination task on either the center or the surround stimulus. The task remained constant within each run, alternating between runs. The timing of the task within each block is indicated by the example blocks in Figure 1B. For each 3-second trial of the task, the attended part of the stimulus (center or surround) was presented twice with a near-threshold contrast difference between them (a 20% contrast increment added to one stimulus and a 20% decrement to the other, yielding a 40% contrast difference about the mean contrast). That is, the maximum contrasts of the low- and high-contrast RG stimuli were 3.2% and 4.8%, respectively; for the BY stimuli, they were 24% and 36%; and for the Ach stimuli, they were 40% and 60%. The unattended (remaining) part of the stimulus was presented twice at the same contrast. Each stimulus was presented within a Gaussian temporal envelope (sigma, 125 ms; total duration, 500 ms), with a 500-ms interstimulus interval, as depicted in Figure 1C. In the remaining trial time (1.5 seconds), the participants indicated with a button press the interval where the attended part of the stimulus contained the higher contrast stimulus. Although the attend-surround condition required covert, rather than overt, attention, the central stimulus was very small, and participants tended to perform better on the attend-surround conditions with mean accuracy (±95% CI) of 91% (±5.3%) compared with 83% (±7.0%) for the attend-center condition, although this difference did not reach significance in a paired t-test, t(11) = 1.85, p = 0.08. 

fMRI methods

Retinotopic and functional localizers

We identified the visual cortical regions V1, V2, V3, V3A/B, LO1/LO2, and hV4 for each participant using rotating wedge stimuli and expanding and contracting concentric rings (Engel, Rumelhart, Wandell, Lee, Glover, Chichilnisky, & Shadlen, 1994; Sereno et al., 1995), standard definitions of these areas (Brewer, Liu, Wade, & Wandell, 2005; Goddard, Mannion, McDonald, Solomon, & Clifford, 2011; Larsson & Heeger, 2006), and the foveal confluence (Schira, Tyler, Breakspear, & Spehar, 2009). To localize areas VO1, VO2, and hMT+ we used data from the retinotopic mapping scans in conjunction with functional localizers for VO (Mullen et al., 2007) and hMT+ (Huk, Dougherty, & Heeger, 2002). Full details of our retinotopic mapping procedures, including scanning protocols, data preprocessing, and region of interest (ROI) definitions have been described previously (Goddard et al., 2019). 

Scanning protocols

All magnetic resonance imaging took place at the McConnell Brain Imaging Centre, McGill University, Montréal, Canada. Functional T2* MR images were acquired on a 3T Siemens MAGNETOM Prisma system with 32-channel head coil (Siemens Healthineers, Erlangen, Germany). Gradient–echo pulse sequences were used to measure the BOLD signal as a function of time. We used a scanning protocol with partial head coverage (including the occipital cortex and the LGN, with slices oriented approximately parallel to the calcarine sulcus), no acceleration, and fine spatial resolution (repetition time = 3000 ms; echo time = 38 ms; 28 axial slices; 1.5-mm³ resolution). Head movement was limited by foam padding within the head coil. Scanning protocols used for retinotopic, localizer, and anatomical scans have been described previously (Goddard & Mullen, 2020). 

fMRI analysis

Surface definition and preprocessing of functional data

For each participant's anatomical template, we used the automatic segmentation processes from FreeSurfer 6.0 (Dale, Fischl, & Sereno, 1999; Fischl, Sereno, & Dale, 1999) to define the gray/white matter and pial/gray matter boundaries. For all other MRI data processing, we used AFNI/SUMA packages (AFNI 17.2.12) (Cox, 1996; Saad, Reynolds, Argall, Japee, & Cox, 2004). All functional data were preprocessed using slice-time correction and rigid-body motion correction before being aligned to the participant's anatomical template. Functional data from the experimental runs were preprocessed in parallel in two different ways. Data used in the group-level surface analysis described below were projected onto the cortical surface by averaging between the white and pial boundaries, spatially smoothed (Gaussian filter, full width at half maximum of 4 mm). Data for all ROI-based analyses (which included the subcortical LGN as well as cortical ROIs) were not projected into a surface space, but the volume data were smoothed (Gaussian filter, full width at half maximum of 3 mm). In both cases, data from each node or voxel were scaled by their mean response for the run and multiplied by 100, yielding data expressed as percent BOLD modulation (about a mean of 100%). 

Within-subject generalized linear modeling

We modeled BOLD responses during functional scans using a generalized linear model (GLM) with the AFNI script 3dDeconvolve. Our model included estimates for each of the unique stimuli and for the gray and black reference stimuli, each using BLOCK(12,1), with the 3-second fixation periods in between each stimulus used as the implicit baseline. The model also included regressors for linear and polynomial trends and six motion correction parameters. We obtained fits for two versions of these GLMs: one in which the model fit a single beta weight for each stimulus type, and the other where it returned individual beta weights for each block, resulting in 12 beta estimates for each stimulus. The first version was used to estimate the average response of each ROI to each stimulus, and the second version was used in the classification analyses described below. 

Between-subject linear mixed-effects modeling

We analyzed the GLMs of each participant at a group level using linear mixed-effects models, implemented with the AFNI script 3dLME (Chen, Saad, Britton, Pine, & Cox, 2013). In these models, we included the first type of GLMs (with a single beta weight for each stimulus type) based on the data that were either warped to match one of the AFNI anatomical template spaces (TT_N27, used for LGN only) or projected into the surface space of each participant (all cortical ROIs). For each participant, we used a standard-mesh surface model (mesh density ld141) to facilitate inter-subject alignment of the cortical data (Saad et al., 2004). Our linear mixed-effects models of data from all subjects (N = 12) included main effects of attention condition (two levels) and unique stimulus (12 levels). 

Classification analyses

We used pairwise classification analyses to measure the extent to which the pattern of BOLD responses across each ROI could be used to predict the content of the stimulus center and stimulus surround within each of the two attention conditions (attend center and attend surround). For each classification, we trained classifiers (using Linear Support Vector Machines) to discriminate between two categories of trial and tested on held-out data. 

For each classification, we created pseudo-trials by averaging across a pair of trials with the same value for the dimension of interest but with differing values along the other dimension, as used previously (Goddard & Mullen, 2020). We selected trials for each pseudo-trial such that they were always balanced across the irrelevant dimension of interest. For example, when training a classifier to discriminate center RG from center BY, we created center RG pseudo-trials from stimuli where the RG center was paired with each of the two achromatic surrounds, as well as center BY pseudo-trials from BY center/Ach surround trials. In this example, we did not include trials with a BY/RG pairing of center and surround, as this would yield pseudo-trials that were no longer balanced along the irrelevant dimension of interest (surround stimulus). In an analogous manner, we used this pseudo-trial method to train classifiers to discriminate each pair of center stimuli (six pairwise comparisons) and each pair of surround stimuli (six pairwise comparisons). For every classification, we used data from across the entire ROI, without attempting to separate responses to the foveal and peripheral parts of the visual field. Our balanced design, along with the use of pseudo-trials, ensured that responses to the irrelevant part of the stimulus could not be used by the classifier to correctly predict the stimulus. To ensure that our results did not depend on a particular assignment on trials to pseudo-trials, for every pairwise classification we generated 100 sets of pseudo-trials, updating the random assignment of trials for each set, and averaged classification performance across these. 

For every classification, the data within each attention condition included 12 pseudo-trials (six pseudo-trials from each of two categories). In each case, we repeated the classification analysis six times, always leaving out a pair of pseudo-trials (one from each category) and testing the accuracy of the classification rule on the held-out data. For all analyses, we expressed average classifier accuracy in d′ (a unit-free measure of sensitivity). Chance classification performance yields d′ = 0. 

Statistical analyses

To explore the effects of attention on average responses (beta values) and classifier accuracy, we conducted a range of repeated-measures analyses of variance (ANOVAs) on our ROI-based analyses (detailed below) using R 3.6.3 (R Foundation for Statistical Computing, Vienna, Austria). We used a Shapiro–Wilk test to test the assumption of normality and Levene's test for homogeneity of variance in order to check for violations of these assumptions of the standard ANOVA. In all cases, data for one or main effects and interactions violated either or both of these assumptions, so for all ROI-based analyses we report instead the relevant statistics from a bootstrap resampling procedure with 10,000 resamples, using the permuco R 1.1.0 package and implementing the permutation-based repeated-measures ANOVA described by Kherad-Pajouh and Renaud (2015). 

Results

Effect of attention on BOLD response amplitude

We measured the fMRI (BOLD) signals induced by 12 stimuli combining pairs of four stimulus types arranged in a center/surround layout, at suprathreshold contrasts. We manipulated participants’ spatial attention by requiring them to maintain fixation while covertly attending to the center or surround stimulus and to perform a contrast discrimination task. 

We used between-subjects linear mixed-effects models to evaluate how response amplitude across the cortical surface and in the LGN varied with stimulus type and participant's task. No node in the cortical surface had a significant main effect of stimulus type, F(11, 11) < 3.31 across all nodes, p > 0.01, uncorrected, but Figure 2 shows the large areas of the cortical surface where there was a main effect of participant's task (attentional condition). Because we did not have a baseline condition of no attention to either the center or surround, all node main effects of attention could be driven by response enhancement or suppression with one or both of the attention conditions. To enable better evaluation of these results, we also show the average eccentricity preference of each surface node (Figure 2B) and the approximate locations of our ROIs. These approximate ROI boundaries are defined by the most common (modal) label of each node across participants (N = 12). Note that ROIs were retinotopically defined in each participant's native space, and all ROI-based analyses below were performed in these native spaces, so these approximate boundaries are included for reference only. 

Figure 2.

 

Changes in response amplitude (beta value) with attention across the cortical surface, averaged across participants (N = 12) in a standardized space (MNI). (A–C) Cortical results are shown on the same inflated spherical representations of the left and right hemispheres of the standard space, centered approximately on the occipital poles. Partially inflated surfaces are shown in the inset, for reference, along with the approximate boundaries of the regions of interest (see text for details), and the location of the calcarine sulci (red dashed lines). A linear mixed-effects model revealed a significant main effect of attention (participant's task) at the cortical locations highlighted in A, F(1, 11) < 0.01, uncorrected. The average preferred eccentricity at each node for these participants is given in B. In C, a series of post hoc contrasts shows the average change in beta value for each attended stimulus type and location. In each case, the attended stimulus is shown in the inset to the left (with unattended regions masked white, not to scale). (D) The effect of attention on the LGN for the average across participants (left) and for an example participant (right), using the same color scale as in C. In both cases, responses are shown only for voxels within a LGN mask that had a significantly different response to the attention conditions (p < 0.05, uncorrected). For individual participants, we used the anatomically defined LGN as a mask and for the group average we used spheres of 20-mm diameter centered on the location of the LGN in the anatomical template space.

Across the cortical surface, there were many nodes where there was a significant effect of attention; see Figure 2A for nodes where F(1, 11) > 9.65, p < 0.01, uncorrected. Around the occipital pole (approximately areas V1, V2, V3, hV4, and LO), there are voxels with significant effects in cortical regions responding to both the center and surround parts of the stimulus (as confirmed by comparison with Figure 2B, showing average eccentricity preference). Between these is a band of nodes for which there was no significant main effect of attention, presumably corresponding to those nodes that respond near the boundary of the stimulus center and surround. This is confirmed in Figure 2C, where the direction of the attention effects are plotted for each stimulus when attended in either the center (left-hand plots) or surround (right-hand plots). These plots show the direction of the attention effect for all nodes and illustrate a broadly consistent effect on response amplitude at the occipital pole across stimulus types. For each stimulus type, attention produced complementary response changes in regions representing the foveal and peripheral parts of the visual field, which had greater response when attention was directed to center and surround stimuli respectively. 

Outside the main foveal confluence, the effects of attention were patchier, and the maps of eccentricity were generally less consistent. Areas V3A/B, hMT, and VO1/VO2 did not share the main foveal confluence but each had a separate foveal representation (Brewer et al., 2005; Huk et al., 2002; Press, Brewer, Dougherty, Wade, & Wandell, 2001). Of these areas, only area V3A/B showed some consistent effects with attention around the border with dorsal V3 (see Figure 2A), which tended to be in the direction of greater response during attend to the surround (Figure 2C). There were few nodes around areas hMT and VO1/VO2 that showed a significant effect of attention, and there was no clear organization in the spatial distribution of non-significant effects (Figure 2C). 

Within the LGN response, amplitudes tended to be higher for the attend-surround condition, as seen in the results of the between-subject linear mixed-effects model (Figure 2D, left). In case the inter-subject alignment did not adequately align this small subcortical structure across participants, we also inspected the individual participant's results, particularly to see whether there was a typical spatial pattern of attentional effects within the LGN, based on its retinotopic map. For each participant, we considered voxels within the LGN with a significant (p < 0.05, uncorrected) difference between attention conditions, averaged across stimulus types. We did not find a spatial pattern that was reliable across participants. Of the 24 hemispheres across 12 participants, only eight LGNs included voxels with a greater response for the attend-center condition and also voxels with a greater response for the attend-surround condition (such as those in Figure 2D, right), despite the liberal significance criterion. Within these eight LGNs there was no consistent pattern in the spatial arrangement of these two types of response. Of the remaining LGNs, eight included only voxels with greater attend-surround responses, six included only voxels with greater attend-center responses, and two did not include any voxels with a significant effect. 

To further characterize the effects of attention on BOLD response amplitude, we also performed ROI-based analyses. We averaged BOLD responses across all voxels within each ROI that exceeded a liberal criterion for visual responsiveness (combined response to visual stimuli greater than the combined response to the reference black and reference gray blocks; p < 0.10, uncorrected), and plotted the data for each unique stimulus in the two task conditions separately (Figure 3). Unlike the between-subject linear mixed-effects models (Figure 2), here responses were averaged across voxels responding to different parts of the visual field, so that in many cases the changes to voxels responding to center and surround tended to cancel out, giving similar overall responsiveness in the two attention conditions. A permutation-based, three-way, repeated-measures ANOVA of the effect of ROI, attention (task) condition, and unique stimulus on response amplitude (beta value) revealed significant main effects of ROI, F(9, 99) = 11.69, p = 0.0002, and unique stimulus, F(11, 121) = 4.70, p = 0.0002, as well as a significant interaction between these effects, F(99, 1089) = 5.81, p = 0.0002. There was no significant main effect of attention condition, F(1, 11) = 2.40, p = 0.15, but there was a significant interaction between ROI and attention condition, F(9, 99) = 6.90, p = 0.0002. 

Figure 3.

 

Average response (beta value with units approximating percent signal change). Average beta values are shown for each of the 12 unique center/surround stimuli, along with the two reference stimuli (uniform black and gray screens), within each region of interest. Averages include the subset of voxels from each ROI which exceeded a liberal threshold for stimulus-related visual activity (combined stimulus response greater than combined response to reference, p < 0.10, uncorrected). Along the two axes of each plot, data are plotted for the two attention conditions, where the participant performed a task based on the center or surround stimulus. Error bars are 95% confidence intervals of the between-subject (N = 12) mean.

To test which ROIs were driving the interaction between ROI and attention condition, we used permutation tests for differences between the attentional tasks within each ROI (with false discovery rate [FDR] correction for multiple comparisons). These contrasts revealed two ROIs with a significant effect of attention: areas V1 (estimated difference = 0.22, p < 0.0001), and V3A/B (estimated difference = 0.24, p = 0.003). In both cases, these ROIs showed a greater overall response when participants were attending to the surround (Figure 3). For the remaining ROIs, there was no significant difference in overall response between the two attention conditions for stimuli of the spatial arrangement used here. 

Effect of attention on classification performance

One of the goals of our study was to better understand how attention enhances the representation of the stimulus information. Hence, we used a series of classification analyses to measure how information about the content of the stimulus center and surround varied with attention in each ROI. We trained and tested classifiers on their ability to discriminate the content of the stimulus center or the stimulus surround using the pattern of responses across voxels within each ROI, within each attention (task) condition. For all classification analyses, we used the beta values for every voxel within the ROI, without selecting voxels on their stimulus response or on their eccentricity preference. For the center stimulus, there were six pairwise classifications of the four stimulus centers, and similarly there were six pairwise classifications for the surround stimulus. 

Classifier performance (averaged across the six pairwise classifications) was above chance for decoding the content of the stimulus center and surround in each ROI (Figure 4). A permutation-based, three-way, repeated-measures ANOVA of the effect of ROI, attention (task) condition, and stimulus location (center or surround) on classifier accuracy revealed significant main effects of ROI, F(9, 99) = 26.52, p = 0.0002; attention condition, F(1, 11) = 7.38, p = 0.017; and stimulus location, F(1, 11) = 11.13, p = 0.013, as well as a number of significant interactions, including significant interactions between stimulus location and attention condition, F(1, 11) = 28.59, p = 0.0006, and among ROI, stimulus location, and attention condition, F(9, 99) = 5.83, p = 0.0002. 

Figure 4.

 

Classification of center and surround stimulus types. We used separate classification analyses to decode the stimulus type in the center and surround of each stimulus, within each of the two attention conditions, for each ROI. Plotted classifier accuracy is averaged across all pairwise comparisons. Error bars indicate the 95% confidence intervals of the between-subject (N = 12) mean. Symbols to the right of each plotted line indicate cases where there was a significant difference in decoding of the stimulus across the attention conditions (^◆◆p < 0.01; ^◆p < 0.05) or a marginally significant difference (⁺p < 0.10). All significance values were corrected for multiple comparisons using FDR correction.

Permutation-based post hoc contrasts (FDR-corrected for multiple comparisons) revealed significant differences in the decoding across attention conditions for the ROIs and stimulus locations shown in Figure 4. The data in Figure 4 show that all significant differences with the attention condition indicate better decoding when the stimulus was attended. However, we have found that attention tends to produce larger boosts in decoding for the surround stimulus, whereas decoding of the center stimulus is affected less or shows no significant difference. Notably, attention did not produce a significant increase in decoding of the central stimulus in areas V1, V2, V3, and hV4, despite the cortical surface in this region showing a change in responsiveness (Figure 2). 

Because there were differences in the average response across stimulus conditions (see Figure 3), classification performance could have been driven by these large spatial-scale differences, rather than differences in the finer spatial pattern of response within each ROI. To test this, we performed classification analyses based on average response, which revealed above-chance but much lower overall performance (see Appendix). This shows that for most ROIs the pattern of response within the ROI contained stimulus-related information beyond that in the mean response alone. 

To explore differences in these attention effects across different stimulus types, we grouped the data according to classification type in Figure 5. Because stimulus types were decoded pairwise, data from each bar includes half the total data set (e.g., decode AM includes AM vs. RG, AM vs. AP, and AM vs. BY), meaning that bars within each attention condition are not orthogonal. Overall, attention most consistently improved discriminations that paired the Ach (M-type) from the remaining stimuli. Conversely, attention was least consistent in improving discriminations pairing the Ach (P-type) from the remaining stimuli. We performed a permutation-based, three-way, repeated-measures ANOVA of the effects of ROI, stimulus location (center or surround), and stimulus type on the difference values shown in Figure 5. The ANOVA revealed a significant main effect of stimulus location, F(1, 11) = 29.01, p = 0.0006, and significant interactions between ROI and stimulus location, F(9, 99) = 5.91, p = 0.0002; between stimulus type and location, F(3, 33) = 3.17, p = 0.042; and among ROI, stimulus type, and location, F(27, 297) = 1.55, p = 0.045. Overall, attention tended to induce greatest effects for pairwise comparisons including the Ach (M-type) stimulus and smallest effects for comparisons including the Ach (P-type) stimulus. Permutation-based post hoc pairwise contrasts between different stimulus types (FDR-corrected for multiple comparisons) revealed that only three of these differences reached statistical significance. For the LO center stimulus, the Ach (M-type) stimulus showed a greater attention effect than the RG stimulus (estimated difference = 0.47, p < 0.0001). For both the V1 and LO surround stimuli, the Ach (P-type) stimulus showed less attentional modulation than the Ach (M-type) stimulus (V1: estimated difference = 0.73, p < 0.0001; LO: estimated difference = 0.53, p = 0.040). 

Figure 5.

 

Attentional change of classifier performance within each stimulus type. Data from Figure 4 are replotted, with changes in decoding (attend-surround – attend-center classifier performance) shown for each stimulus type and location. Error bars indicate the 95% confidence intervals of the between-subject (N = 12) mean. Symbols above each pair of stimulus locations indicate cases where there was a significant difference in these values across stimulus locations—that is, a significant interaction between the effects of stimulus location and attention condition on decoding (^◆◆p < 0.01; ^◆p < 0.05) or a marginally significant difference (⁺p < 0.10). All significance values were FDR corrected for multiple comparisons.

Discussion

We measured changes in both BOLD response amplitude and pattern classification accuracy induced by spatial attention and compared these effects across foveal and peripheral parts of the visual field. We found a number of interesting dissociations between the effects of attention on overall response and on classifier accuracy, as well as differences between early and later cortical regions in how responses to foveally presented stimuli were affected by spatial attention. 

Occipital foveal confluence (V1, V2, V3, and hV4) showed increased response but no increase in stimulus information with attention

The contralateral visual hemifield maps of areas V1, V2, V3, and hV4 formed a cluster, with a confluent foveal representation centered on the occipital pole (Wandell, Brewer, & Dougherty, 2005). Attention to the center versus surround of the stimulus changed the amplitude of responses across voxels according to their retinotopic preference, such that responses were greater when attention was directed to their preferred location. Because participants were always attending to either the center or the surround part of the visual field, we cannot rule out that these effects were driven primarily by one condition. For example, if attention to the central stimulus did not induce any response amplitude changes, this pattern of results could reflect a boost of surround response and suppression of the center response when attention is directed to the surround. However, Smith et al. (2000), who compared attending to the fovea with a diffuse attention (passive fixation) condition, reported that attending to the fovea induced both increased response at the foveal representation and suppressed response elsewhere for these early cortical regions. This makes it likely that the response differences we found at the fovea reflects a boost in response when attended, possibly combined with suppression of response when the participants attended to the surround. 

Unlike the amplitude differences, the changes in classifier accuracy reveal an asymmetry in the effects of attention on the representations of the center and surround stimuli. Across each of the areas that make up the occipital foveal confluence we found robust effects of attention on decoding of the surround stimulus but no significant effects on the decoding of the center stimulus, despite the fact that both center and surround responses were modulated by attention. We think this is unlikely to be a ceiling effect, where classifier performance was too high in the unattended condition to reveal an increase with attention, as classifier performance in the unattended condition was comparable across center and surround stimuli in these areas. 

Previous work has demonstrated that classifier accuracy is typically tightly correlated with BOLD response amplitude; for example, when stimulus contrast is increased there are proportional increases in BOLD response and classifier accuracy (Smith, Kosillo, & Williams, 2011; Tong, Harrison, Dewey, & Kamitani, 2012). However, changes in classifier accuracy with attention are not necessarily coupled with a change in response amplitude (Cohen & Tong, 2015; Jehee et al., 2011). Of particular relevance here, Jehee et al. (2011) found that spatial attention only increased decodable information for task-relevant features and not task-irrelevant features at the same location. It is possible here that the lack of increase in decoding the stimulus type at the center is related to the fact that participants were making judgments related to stimulus contrast rather than stimulus identity. However, this explanation does not account for why spatial attention enhanced the decoding of the surround stimulus in these areas or why higher order areas (LO, VO1, and VO2) showed effects of attention on decoding of the center stimulus. 

Alternatively, our results may indicate that spatial attention does not produce the same changes in stimulus coding for the foveal confluence at the occipital pole. One possibility is that when we are attending to peripheral locations attention is not disengaged from the fovea in the same way that attention can be disengaged from the periphery when we attend to the foveal part of the visual field. In this view, the foveal visual field could be a site of compulsory attention for this earliest cortical cluster of visual field maps, so that even when attending to the surround stimulus the stimulus in the center of the visual field is processed with high fidelity by these early visual areas. The central bias of attention (Bao et al., 2013; Staugaard et al., 2016; Valsecchi et al., 2013; Wolfe et al., 1998) could perhaps be related to such a tendency for these early visual areas to process the foveal visual field regardless of where attention is directed. This may not mean that attention effects are categorically absent in this part of the visual field; for example, Poletti, Rucci, and Carrasco (2017) provided behavioral evidence that spatial attention effects can operate within the foveola (central 1° of the visual field) in a manner analogous to spatial attention to peripheral stimuli, but at a minimum it suggests an asymmetry where unattended stimuli in the central visual field are not suppressed to the same extent as peripheral stimuli. It is important to note that our current results provide only modest support for this speculation, as the main evidence in favor of this view is a null result—namely, the lack of an effect of attention on decoding the central stimulus for these visual areas. However, as mentioned above, the fact that these regions show an attentional effect on surround stimulus decoding, combined with the fact that higher visual areas show an effect of attention on the central stimulus, suggests that there could be a genuine effect that is worth testing in future work. 

Previous work has shown attentional modulation of LGN responses (O'Connor et al., 2002), including retinotopically organized changes in response (Schneider & Kastner, 2009). We found some attentional modulation of the LGN response, but no clear evidence of retinotopically organized effects, likely related to the lower signal strength from this small, deep structure and perhaps the tendency for eccentricity mapping to be weaker than polar angle mapping in previous fMRI studies (Schneider, Richter, & Kastner, 2004). As in our previous work with similar stimuli (Goddard & Mullen, 2020), we found that classifier decoding of a stimulus was significantly above chance but was lower for the LGN than for cortical areas. Perhaps because of this lower overall performance, classifier accuracy did not show any significant modulation with attention. 

Effects of attention on higher visual areas were more similar across the fovea and periphery

Across the higher visual areas, we found that the effects of attention on decoding performance were more uniform across the center and surround stimuli. A notable exception was area hMT, for which we failed to find significant attention effects for either stimulus location. Areas V3A/B and LO both showed some evidence of response changes, and for LO there were attentional effects on decoding for both stimulus locations; however, for V3A/B, there was a marginally significant effect on decoding of the surround stimulus only. 

In areas VO1 and VO2 there was no evidence of attention producing a systematic change in response across the cortical surface (Figure 2) or in the average response (Figure 3), but attention boosted the stimulus information for both center and surround. One mechanism by which spatial attention is proposed to enhance information about stimuli at the attended location is by causing shifts in the location and size of receptive fields (Anton-Erxleben & Carrasco, 2013). This is supported by evidence from a range of fMRI studies demonstrating that spatial attention shifts the population receptive fields of voxels toward the attended location for visually responsive voxels in occipital, temporal, and parietal cortices (Kay, Weiner, & Grill-Spector, 2015; Klein, Fracasso, van Dijk, Paffen, Te Pas, & Dumoulin, 2018; Klein, Harvey, & Dumoulin, 2014; Sheremata & Silver, 2015; van Es, Theeuwes, & Knapen, 2018; Vo, Sprague, & Serences, 2017). Spatial attention has also been shown to narrow spatial tuning of responses (Fischer & Whitney, 2009). Neurons in higher ventral cortex (from hV4 and more anterior regions, including VO1 and VO2) contain larger receptive fields that change according to the attended stimulus to a greater degree than in the early visual cortex. For example, population receptive fields in ventral visual cortex are more eccentric and greater in size when attention is directed to peripheral stimuli (Kay et al., 2015). Klein et al. (2014) suggested that the larger effects of attention observed for higher visual areas might be mediated primarily by the larger receptive fields in these regions. If changes induced by spatial attention in VO1 and VO2 are primarily mediated by shifting receptive fields, this would account for the fact that the large attentional effects on classifier performance are not accompanied by response changes. 

Minimal differences were observed across stimulus type

We found only fairly subtle differences between stimulus types in terms of how their decoding varied across attention conditions in different regions and at different eccentricities, despite the fact that we previously used the same stimuli to revealed inter-area differences in stimulus specialization (Goddard & Mullen, 2020). Ventral regions (hV4, VO1, and VO2) showed very similar patterns of attentional enhancement across all stimulus types. The areas that showed the largest differences across stimulus types were V1, V2, and V3A/B. In each of these cases, there was a tendency for the Ach (M-type) to show relatively larger modulations in effective decoding with attention. In the case of V3A/B, this could possibly reflect the strength of the magnocellular projection to the dorsal visual pathway (Maunsell, Nealey, & DePriest, 1990; Nassi & Callaway, 2006; Nassi & Callaway, 2007). However, V1 and V2 also showed a similar pattern across stimulus types, including no attentional modulation for the Ach (P-type) stimulus, despite these areas showing a strong preference for this stimulus in our previous work (Goddard & Mullen, 2020). In this way, the difference in attentional modulation across stimulus types did not clearly relate to the stimulus preferences of each area. 

Conclusions

We examined the effects of spatial attention on BOLD response amplitude and stimulus decoding for stimuli sampling a range of spatiotemporal achromatic and chromatic contrasts. We found a number of dissociations between the effects on response amplitude and classifier accuracy, highlighting the utility of measuring the impact of attention using complementary metrics. Spatial attention produced asymmetric effects on the information present in the pattern of responses to the center and surround stimuli for regions in the occipital foveal confluence (V1, V2, V3, and hV4), with attention boosting decoding of the surround but not the center stimulus. Higher visual areas showed attentional enhancement of information about both the center and the surround stimuli, implying qualitative differences across visual cortical areas in how attention effects vary with eccentricity. 

Acknowledgments

The authors thank Michael Ferreira for his advice on fMRI protocols; the imaging technicians at the McConnell Brain Imaging Center at McGill University (David Costa, Ron Lopez, and Louise Marcotte) for their assistance with MRI data collection; and our participants. 

Funded by grants from the Canadian Institutes of Health Research (153277) and Natural Sciences and Engineering Research Council (RGPIN 183625-05) to KTM. EG was partly supported by an Australian Research Council DECRA Fellowship (DE200100139). 

Commercial relationships: none. 

Corresponding author: Kathy T. Mullen. 

Email: [e-mail]. 

Address: Department of Ophthalmology & Visual Sciences, McGill Vision Research, McGill University, Montreal, Quebec, Canada. 

Appendix

To estimate the extent to which classifier performance was driven by differences in average response (seen in Figure 3), we repeated our classification analyses using the average response of each ROI on each trial, rather than the pattern across voxels. The results of this analysis are shown in Figure A1. Classification performance tended to be above chance but much reduced from performance in the original analysis, showing that the pattern of responses within each ROI carried stimulus-related information in addition to that from differences in mean response. An exception to this trend was in the LGN, where the classifier based on mean response tended to outperform the original, suggesting that, in the case of the LGN, the pattern of responses was not informative about the stimulus and/or that averaging across voxels reduced noise to a greater extent than it reduced stimulus-related signal. A permutation-based, three-way, repeated-measures ANOVA of the effect of ROI, attention (task) condition, and stimulus location (center or surround) on classifier accuracy based on mean response revealed no significant main effects: ROI, F(9, 99) = 1.89, p = 0.061; attention condition, F(1, 11) = 0.082, p = 0.779; stimulus location, F(1, 11) = 0.092, p = 0.767. There were no significant interactions (p > 0.05 in each case). 

Figure A1.

 

Classification of center and surround stimulus types based on mean responses within each ROI. Plotting conventions as in Figure 4. Results from Figure 4 are replotted as lighter, dashed lines for comparison. For classification based on mean response, there were no significant differences in decoding of the stimulus across the attention conditions, in any ROI.