Distinct effects of attention on the neural responses to form and motion processing: A SSVEP source-imaging study

Abstract

Abstract

Abstract:

Abstract  We measured neural responses to local and global aspects of form and motion stimuli using frequency-tagged, steady-state visual evoked potentials (SSVEPs) combined with magnetic resonance imaging (MRI) data. Random dot stimuli were used to portray either dynamic Glass patterns (Glass, 1969) or coherent motion displays. SSVEPs were used to estimate neural activity in a set of fMRI-defined visual areas in each subject. To compare activity associated with local versus global processing, we analyzed two frequency components of the SSVEP in each visual area: the high temporal frequency at which the local dots were updated (30 Hz) and the much lower frequency corresponding to updates in the global structure (0.83 Hz). Local and global responses were evaluated in the context of two different behavioral tasks—subjects had to either direct their attention toward or away from the global coherence of the stimuli. The data show that the effect of attention on global and local responses is both stimulus and visual area dependent. When attention was directed away from stimulus coherence, both local and global responses were higher in the coherent motion than Glass pattern condition. Directing attention to coherence in Glass patterns enhanced global activity in areas LOC, hMT+, V4, V3a, and V1, while attention to global motion modulated responses by a smaller amount in a smaller set of areas: V4, hMT+, and LOC. In contrast, directing attention towards stimulus coherence weakly increased local responses to both coherent motion and Glass patterns. These results suggest that visual attention differentially modulates the activity of early visual areas at both local and global levels of structural encoding.

Introduction

The ability to combine local cues for elementary stimulus features is central to visual perception, yet the cortical mechanisms involved are only beginning to be understood (Sasaki, 2007). In primates, orientation, direction of motion, and disparity are robustly coded for the first time in the primary visual cortex (Livingstone & Hubel, 1987; Sincich & Horton, 2005), whereas the integration of features into coherent textures and objects is mainly accomplished by higher visual mechanisms residing in the extra-striate cortex (Pasupathy, 2006). 

Random-dot stimuli have been used widely to study the high-level integration of local cues. Random-dot stimuli enable creation of stimuli that differ in their global structure, while preserving the same local structure. The global structure of the stimulus is controlled by the relationship between local cues. These local cue relationships can be graded between highly organized and completely random without altering the structure of the local cues. This permits manipulations that are invisible to the small receptive fields in early areas of the visual system. For example, random dot stereograms elicit depth perception from horizontal disparity cues in the absence of monocular depth cues (Julesz, 1971). Glass patterns (Glass, 1969) portray large-scale texture flows through orientation cues from locally paired dots. Presenting the individual dots in these dipole pairs in a fixed temporal order can, similarly, give rise to a sensation of global coherent motion (Newsome & Pare, 1988). 

Interestingly, sequential brief presentations of Glass patterns induce a perception of implied or illusory motion that is categorized as “dynamic.” Dynamic Glass patterns have no net global motion energy, and the motion direction of each dot pair is, at best, ambiguous. However, dynamic Glass patterns interact with real motion psychophysically (Ross, Badcock, & Hayes, 2000) and show a degree of sensitivity in the motion processing area, hMT+ (Krekelberg, Dannenberg, Hoffmann, Bremmer, & Ross, 2003; Krekelberg, Vatakis, & Kourtzi, 2005). This suggests that the analysis of visual form and motion, often assumed to be managed by largely independent streams (Mishkin & Ungerleider, 1982), interact. 

This study asks whether directed attention modulates neural responses generated at different hierarchical levels of visual processing in the same way. Specifically, we ask whether attention to global structure modulates responses of neurons tuned to both local and global properties of the stimuli or only the global properties. We also ask whether directed attention is equally important for processing global form and motion. To answer these questions we compared visual evoked responses generated by “dynamic” Glass patterns (Ross et al., 2000) and by coherent motion displays across a set of cortical visual areas. Identical global structures were used. The only difference between our form and motion stimuli was that Glass pattern extraction involved the detection of a local orientation cue from simultaneously presented dot pairs, while coherent motion extraction involved the detection of systematic organization in the spatiotemporal relationship between dot pairs. By imposing different temporal frequency tags on the local and global aspects of our stimuli, we were able to separate responses to these distinct levels of processing using spectrum analysis. In addition, because we recorded these responses within individual visual areas defined by retinotopic mapping and functional localizer MRI scans, we were able to localize the frequency tagged local and global response components within the anatomical hierarchy of visual areas. 

Our main result is that directed attention enhances local and global neural responses of form and motion differently within the same anatomical location: (1) The effect of directed attention on the global responses was stronger for Glass patterns than for coherent motion, particularly in the lateral occipital complex (LOC), but the effect on the local responses was effectively uniform across both stimulus types. (2) The modulatory effect of directed attention on global responses was generally greater than the effect on local responses. (3) In the absence of directed attention, both the local and global responses evoked by the spatiotemporal sequences underlying the coherent motion displays were generally larger than those evoked by the simultaneously presented dots of the Glass pattern displays. Together, these data suggest that the effects of directed attention on global coherence are greater for stimuli defined by coherent form rather than coherent motion. 

Methods

Participants

We recorded from sixteen well-practiced participants (with over 10 participation hours in other experiments, aged 21–65 years; three females), who have participated at least twice previously in other electroencephaolography recording (EEG) studies. Our participants had visual acuities of better than 6/6 in each eye, with correction if needed, and stereoacuity of 40 arc s or better on the Titmus and Randot stereoacuity tests. Acuity was measured using the Bailey-Lovie chart, which has five letters per line and equal log increments in the letter sizes across lines. Informed written consent was obtained before experimentation under a protocol approved by Smith-Kettlewell Eye Research Institutional Review Board, in compliance with national legislation and the Code of Ethical Principles for Medical Research Involving Human Subjects of the World Medical Association (Declaration of Helsinki). 

Stimuli

Stimuli were presented in a dark and quiet room on a color CRT monitor (Sony MultiScan 420GS, Sony, USA; screen resolution of 1024 × 768 pixels, refresh rate of 60 Hz) by in-house software running on a Macintosh G4 computer (Apple, Inc., Cupertino, CA). Mean luminance was 96 cd/m² and contrast was 90%. Dynamic Glass patterns and coherent motion stimuli consisted of random dot kinematograms covering a 20 × 20 deg square with 5% density (about 9 dot pairs/deg²). Each dot was a 3.2 × 3.2 min white square. 

The dynamic Glass stimuli consisted of a sequence of fields comprised of dot pairs, with each dot in the pair being separated by 19.2 arc min. The orientation of the dot pairs was constrained to be tangential to imaginary circles centered on fixation during half of a stimulus cycle (0.6 s of a 1.2 s total cycle time). During the other half of the cycle, the dot pairs had a random orientation. All of the dots were updated every 33 ms (30 Hz, a frequency hereafter referred to as F2). A full cycle of alternation of the global structure between a series of Glass patterns and a series of random patterns lasted 1.2 s, yielding a stimulus frequency of 0.83 Hz, which we will refer to as F1). 

The coherent motion stimuli used the same dot sizes and number on each frame, but here the members of the dot pairs were presented sequentially rather than simultaneously as in the dynamic Glass patterns. Each dot in the pattern had a partner dot presented 19.2 arc min away with a temporal offset of 33 ms (30 Hz). The direction of the second dot was tangential to an imaginary circle centered on fixation during half of a stimulus cycle (0.6 s of a 1.2 s total cycle time). During the other half of the cycle, the dot pairs had a random direction of motion. Half of the dots were randomly redrawn every 33 ms (limited lifetime). 

In both the dynamic Glass pattern and coherent motion conditions, the participants saw a globally coherent pattern for 600 ms and then a random pattern for 600 ms (Figure 1). These alternations were presented continuously for 12 s. The presentation of Glass patterns/motion coherence stimuli alternated within a single recording (12-s Glass pattern, 12-s coherent motion)¹. These 24-s cycles of Glass pattern/motion stimuli were repeated 10 times in a continuous run for a total run duration of 240 s. The order of the alternation was randomized. 

Figure 1

 

Coherent motion and Glass pattern stimuli. The global patterns periodically changed from coherent to random every 600 ms, while the position of the local dots updated every 33 ms.

Behavioral tasks

During separate 4-min runs, the observers were asked to either attend to the global coherence of the dot patterns (referred to as the “directed attention” conditions for both Glass patterns and coherent motion stimuli) or to an array of letters (referred to as the “diverted attention task”). The two behavioral tasks were overlaid on the screen at all times, and only the instructions to the observer differed across blocked trials. In the directed attention conditions, the participants were asked to detect threshold-level changes in the coherence of the dots. During the periods of global structure stimulation, the observers detected threshold level decrements in coherence. This procedure ensured that the participants were equally vigilant during structure-present and structure-absent periods of stimulation and that they were devoting their attention to global structure in both intervals. The coherence decrements were selected on the basis of pilot studies to yield approximately 80% correct detections, and we verified that this performance level was attained by our participants after the experimental sessions. The decrements were presented for 600 ms with a duty cycle of 20%. 

The diverted attention task was designed to redirect attention away from the dot stimuli in order to assess the role of sustained attention in processing both local and global activity. In the diverted attention task, the participants determined whether an array of randomly oriented letters contained a single “T” among 4 “L's,” or whether the array consisted of 5 L's. The target arrays were preceded by an array of five randomly oriented “F's” presented at the same locations as the T's and L's. The letter detection task was maintained at a constant high level of difficulty by controlling the presentation duration of the letters using a staircase procedure that held performance near the 82% correct level. The target letter array was presented parafoveally at 5°, and masked by a set of five randomly oriented F's to control exposure duration. The timing of the letter task was randomized with respect to the 0.83 Hz stimulation rate of the Glass pattern and coherent motion stimuli and responses to the letter task did not survive the averaging process associated with the dot pattern responses. In both the coherence and letter tasks, the participants indicated their choices by pressing keys. 

Data acquisition

EEG data were collected with whole-head, 128-channel HydroCell Sensor Nets (Electrical Geodesics, Eugene, OR), which provided uniform spatial sampling of ∼2 cm (sensor to sensor). The EEG was amplified at a gain of 1,000 and was referenced to a vertex physical reference. Signals were bandpass filtered between 0.1 and 50 Hz and were digitized at 779 Hz with a precision of 4 bits/μV (Electrical Geodesics NetAmp 200). The 16-bit analog-to-digital converter was clocked externally via a hardware interface to the video card that used the horizontal synch of the video monitor as its base clock. The video stimulation computer also sent a digital trigger to mark the recording of the first active video frame to indicate the precise beginning of the trial. 

Artifact rejection and spectral analyses were done after the data acquisition session. Raw data were evaluated sample by sample to determine the number of samples that exceeded a set threshold (∼30 μV). Noisy channels that had a large percentage of samples exceeding the threshold were replaced by the average of their six nearest spatial neighbors. Noisy 1-s epochs were also excluded. After artifact rejection, EEG data were re-referenced to the common average of all the channels. 

Signal and statistical analysis

The time averages were converted to complex-valued amplitude spectra via a discrete Fourier transform. These amplitude spectra were then evaluated at harmonics of the global-structure update frequency of 0.83 Hz and the dot update frequency of 30 Hz. We adopt the following convention for describing response components: nFm, where n is the harmonic number and m is 1 for components that are phase locked to the global-structure update rate and 2 for components that are phase locked to the dot-update rate. Thus, 1F1 and 2F1 refer to the first and second harmonic components of the response to the global structure alternation, and 1F2 refers to the first harmonic response associated with the rapid updating of the dot patterns. The 1F1 and 2F1 were the most consistent and statistically reliable components associated with the global structure updates. The 1F2 component dominated the responses to the dot update rate. Therefore, we focus the main analysis on these response components. For illustration purposes, local and global response waveforms are shown in the time-domain in Figure 2 top. The global response waveform (Figure 2, third row) was reconstructed from the spectrum by an inverse Fourier transform that used the first 10 harmonics of F1 (i.e., from 0.83 to 8.3 Hz). The waveform for the dot-update response (Figure 2, fourth row) was reconstructed by inverse transforming the Fourier coefficients in a ±6.64 Hz range of frequencies centered on the dot update frequency (30 Hz). The Fourier coefficients were chosen so that they were multiples of the global alternation frequency (0.83 Hz/1F1). This range indexes both the response at the dot update frequency (1F2) and the modulation of the response caused by the global alternation (1F1). 

Figure 2

 

Overlay of average responses across 128 channels as a function of time (first, third, and fourth rows) and temporal frequency (second row). Responses reflect the structure of the stimulus presentation. The update of the local dots is represented in the time domain by the fast oscillations in the waveform (fourth row) and is represented in the frequency domain by the peak at the 30 Hz bin (second row). The update of the global pattern is represented by the envelope of the waveform (filtered up to 10 Hz; third row) and the peaks at lower temporal frequencies, at 0.83 and 1.66 Hz in particular (second row). Responses in the attended conditions (blue and black traces) were higher than in the unattended conditions (red and orange traces) for the low-frequency global components of the response, but not in the high-frequency components.

To test for differences in cortical current density across stimulus-type and regions of interest (ROIs), we performed repeated-measure ANOVAs on the amplitude of the local responses (1F2) and on the global response computed as the quadrature sum of the amplitudes of the dominant harmonic components related to the global update rate: (1F1² + 2F1²)^1/2. 

Head conductivity model

As part of our steady-state visual evoked potentials (SSVEP) source-imaging procedure, we created realistic volume conductor models from MRI data for each of our observers. We used the boundary element method ([BEM] Hamalainen & Sarvas, 1989), which assumes homogeneous conductivity within a tissue type and models the head as three concentric compartments (brain/CSF, skull, and scalp). Individualized models were created using MNESuite (Martinos Center for Biomedical Imaging, Charlestown, MA) after a customized tissue segmentation was performed. 

The three relevant meshes used in the BEM model were derived from T1 and T2 whole-head anatomical MRI scans collected on a 3T Siemens TIM-Trio scanner (Siemens Corp., Princeton, NJ) (3D SPGR or MPRAGE pulse sequences). All anatomical head volumes were composed of sagittal slices with a resolution of 0.94 × 0.94 × 1.2 mm slices. These images were aligned, averaged, and resampled into 1 × 1 × 1 resolution 3D anatomical volume using the FSL toolbox (FMRIB, Oxford, UK, http://www.fmrib.ox.ac.uk/). Head models were based on tissue segmentation of contiguous regions of the scalp, outer skull, inner skull, and cortex. Using the FreeSurfer package (Martinos Center for Biomedical Imaging, Charlestown, MA), gray and white matters were defined by voxel intensity thresholding and smoothing. The gray matter surface was then derived from the boundaries of the segmented white matter and pial surfaces. The sensors were aligned to the MRI anatomical positions using the three digital fiducial markers and were fitted using a least-squares algorithm. The locations of the sensors and three fiducial markers (nasion and left and right peri-auricular sites) were digitized using a 3Space Fastrack 3-D digitizer (Polhemus, Colchester, VT) at the end of each recording session. 

FMRI-based regions of interest

Functional MRI (fMRI) scans were collected on the same scanner used for the anatomical scans. Retinotopic mapping was performed via a combination of rotating wedge and expanding/contracting ring scans (Engel, Glover, & Wandell, 1997; Wandell, Brewer, & Dougherty, 2005). The ROI corresponding to human middle temporal area (hMT+) were determined using low-contrast moving versus stationary random dot patterns (Huk & Heeger, 2002), while a ROI corresponding to the lateral occipital complex (LOC) was determined by contrasting activation to intact versus scrambled photographs of common objects (Kourtzi & Kanwisher, 2000). 

Cortical current density estimation

The spatial distribution of the SSVEP across the cortex was modeled with a cortically constrained L2 minimum norm method. This model assumes that the SSVEP at the scalp is generated by multiple dipolar sources in the gray matter oriented perpendicularly to the cortical surface. The estimates of the current densities are calculated by optimizing the fit to the data under the constraint of a minimal L2 norm for the source magnitudes (i.e., minimizing the sum square of the dipole magnitudes; Ales & Norcia, 2009; Cottereau, Ales, & Norcia, 2012; Hamalainen & Ilmoniemi, 1994). The optimal regularization parameter, lambda (λ, was determined using a generalized cross-validation approach (GCV). This technique uses a leave-one-out procedure to robustly estimate the noise in the measurements and computes the value of lambda that minimizes it (Reeves, 1994). It has been adopted in a number of estimation problems, including magnetoencephalography (MEG) brain imaging (Cottereau, Jerbi, & Baillet, 2007). In our study, the computation of the GCV and the corresponding optimization of lambda was performed using the sine and cosine components of the signal at the first harmonic of the global update rate (1F1) using previously published routines (Hansen, 1994). In the L2 minimum norm procedure, estimates of the cortical current density were calculated by linearly optimizing sum square voltages (Cottereau, McKee, & Norcia, 2012). The source imaging approach proposed here is similar to techniques in reconstructing retinotopic areas (Im, Liu, Zhang, Chen, & He,, 2006). Given that the Euclidean distance between our chosen ROIs is at least 2 cm, the resolution of the inverse should be sufficient to resolve responses in those areas. 

Results

Response spectra and waveform

Our hierarchical stimuli elicit evoked responses that are driven by two intrinsic frequencies in the displays—the frequency F2 at which the dots are updated and the frequency F1 at which the global structure is updated. These responses can be examined both in the time and temporal frequency domains. 

In the time-average, the two components overlap throughout the 1.2 s base period of the alternation between globally coherent and random structure. Time averaged waveforms for the main comparisons of interest are shown as 128-channel overlays (Figure 2). In the top left plot of Figure 2, the waveforms for the Glass pattern onset/offset conditions are shown. The blue traces plot group average responses when the Glass pattern coherence was task relevant (labeled “attended”), and the red traces plot the responses to the same stimuli when the Glass patterns were irrelevant and the observers were performing the letter task (labeled “unattended”). Data from the coherent motion onset/offset conditions are plotted in the upper right panel of Figure 2 using the same convention. Here the black traces are from the trials during which motion coherence was task-relevant, and the orange traces are from the trials when motion coherence was irrelevant. 

Activity associated with the local and global update rates are well separated in the frequency domain, and this makes it possible to distinguish precisely how the different response components (global and local) are being affected by the task instructions. 

As shown in the second row of Figure 2, the response to the fast dot update rate is located primarily in the single bin in the Fourier transform at 30Hz (1F2). By contrast, responses evoked by the global structure alternation are confined to single bins centered on the first harmonics of the global update rate (1F1 & 2F1). The third row of Figure 2 depicts waveforms which were lowpass filtered up to the tenth harmonic of the 1F1, by zeroing all frequency components above 8.33 Hz. The fourth row was bandpass filtered centered on 30 Hz. 

It is apparent from Figure 2 that responses locked to the dot update rate (F2 = 30Hz) are not affected by the observers' task, but that responses that are locked to the 0.83 Hz global-structure update rate (F1) are. For both Glass pattern and coherent motion stimuli, the first harmonic of the global update rate (1F1) is largest, followed by the second harmonic (1F2), and we thus focus the remainder of the analysis on the 1F2, 1F1, and 2F1 components. 

Scalp topography

There are three notable differences between local and global responses at the sensors. First, the responses to the first (0.83 Hz) and second (1.66 Hz) harmonics of the global update rate (1F1 & 2F2) were more broadly distributed over both parietal and lateral occipital areas, while the 30 Hz local dot update response (1F2) was more restricted to the occipital pole. Second, directed attention to global coherence appears to only modulate global responses, but not local responses. Third, the effect of attention on global responses was stronger for Glass patterns than for motion, whereas the effect of attention on local responses had the opposite pattern. The spatial distribution of the grand-average responses on the scalp is shown in Figure 3 for the 1F2 (local) and the 1F1 and 2F1 (global) components. These data suggest that global and local responses differ in both their stimulus-driven and task-dependent selectivities, indicating that different underlying mechanisms/cortical sources are involved. 

Figure 3

 

Scalp topographies at critical temporal frequencies. Global responses were generally more distributed than local responses, which were more localized in occipital areas. The local response at 30 Hz appears less susceptible to the effects of directed attention than global responses at 0.83 and 1.66 Hz. Yellow areas indicate higher activity. Scale is in microvolts (μV).

ROI-based analyses

Scalp topographies only provide a qualitative overview of the differences of the spatial distribution of the responses. To determine where the local and global structures of Glass and motion patterns are processed in cortex, we also localized the SSVEP responses using a cortically constrained L2 minimum norm inverse (see Methods). Figure 4 shows surface-based averaged (Fischl, Sereno, & Dale, 1999) evoked responses from our 16 observers projected onto the cortical surface of one observer at critical temporal frequencies. This observer's fMRI-derived ROIs are also shown for reference. These data suggest that local responses (1F2) arise from more medial occipital sources than global responses (1F1, 2F1) that arise in more lateral-occipital areas. 

Figure 4

 

(Top) Example regions of interest for one observer. (Bottom) Surface averages of VEPs (attended condition) from our 16 observers projected onto the cortex of the same observer. Local responses (red areas) were concentrated in occipital areas, particularly in V1. Global responses were more lateral, with a concentration in the right LOC. Red areas indicate higher activity. Global responses were the quadratic sum (i.e., root mean square [RMS]) of 1F1 and 2F1. Units are in current density and are proportional to μAmp/mm².

For each of our observers, we also collected fMRI scans in order to localize evoked responses within their individually mapped ROIs (see Figure 4 for an example). We then averaged these ROI-localized responses across observers. This allowed us to assess quantitatively whether directed attention increases neural activity uniformly across visual areas and to evaluate which visual areas are selective for form or motion stimuli. We examined responses in five well-separated visual areas, V1, V3a, V4, LOC, and hMT+ that were driven by updates of the local and global structure of the coherent motion and Glass patterns. 

Sensitivity to global transitions

Because our behavioral task required observers to track changes in the global structure of our stimuli, we first quantified the attention effect on the global responses directly. 

Feed forward effects of stimulus type

We determined whether cortical responses were different for form or motion even when attention was directed away from the stimulus dots (labeled as “unattended” in Figure 5). Results show that responses to coherent motion were higher than responses to Glass patterns. We conducted a 5 (ROI) × 2 (stimulus type) repeated measures ANOVA only on responses in the unattended condition. We found a significant main effect of stimulus type, F(1, 15) = 11.244; p = 0.004, but no main effect of ROI, F(1, 15) = 1.811; p = 0.139 or interaction, F(1, 15) = 0.945; p = 0.444. These results show that cortical responses to motion coherence were generally greater than cortical responses to form coherence when attention was targeted to an unrelated letter identification task. 

Figure 5

 

Global and local responses in fMRI-defined ROIs. (Top) While global responses to motion coherence were higher than global responses to Glass patterns in the unattended cases, directed attention modulated responses to Glass patterns more than responses to motion coherence. (Bottom) Directed attention modulated local responses to form and motion stimuli similarly. Global magnitude is the quadrature sum of the 1F1 and 2F1. All units are in current density, proportional to μAmp/mm². Error bars ±SEM.

Effect of attention

We carried out several analyses to determine the effects of directed attention on the neural correlates of our dot stimuli, and we generally found that attention to Glass patterns elicited a greater modulation than attention to coherent motion. This differential pattern of enhancement between form and motion coherence was particularly consistent in V1 and in the LOC. (Conditions where attention was directed towards the coherence of the stimulus dots are labeled as “attended” in Figure 5.) 

We conducted independent repeated measures 2 (stimulus type) × 2 (attention) ANOVA for each ROI on response magnitude (Figure 5). In V1, we found a reliable effect of attention, F(1, 15) = 5.531; p = 0.033, but no effect of stimulus type, F(1, 15) = 0.004; p = 0.949. There was a significant interaction between attention and stimulus type, F(1, 15) = 6.043; p = 0.023. These results show that directed attention did not affect responses to coherent motion but increased responses to Glass patterns. We found a similar pattern of results in the LOC: a significant effect of attention, F(1, 15) = 6.333; p = 0.024, no effect of stimulus type, F(1, 15) = 0.004; p = 0.949, and a reliable interaction between them, F(1, 15) = 6.314; p = 0.024. We only found significant or marginally significant main effects of attention in V4, F(1, 15) = 3.549; p = 0.079, and hMT+, F(1, 15) = 5.886; p = 0.028, but no other main effects or interactions (p > 0.10). There were no effects in V3a (p > 0.10). These results show that directed attention differentially affected cortical responses to form and motion coherence in V1 and LOC. 

An omnibus 5 (ROI) × 2 (stimulus type) × 2 (attention) ANOVA resulted in a significant main effect of attention, F(1, 15) = 10.483; p = 0.006 and a significant interaction between stimulus type and directed attention, F(1, 15) = 8.680; p = 0.010. Together, these analyses suggest that the effect of directed attention on cortical responses is stronger in Glass patterns than in motion coherence (Figure 5). 

We calculated an attention index by subtracting the magnitudes of the unattended condition from the attended condition normalized by the magnitude of the unattended condition ([Attended − Unattended]/Unattended). A significant attention effect corresponds to an attention index different from zero. These analyses show that the effects of attention were more extensive to Glass patterns than to coherent motion. We conducted one-sample t-tests (two-tailed) on the attention index. These analyses show that attention significantly affected global Glass pattern responses across all ROIs: V1 (p < 0.001), V3a (p = 0.017), V4 (p = 0.004), LOC (p < 0.001), and hMT+ (p = 0.015). However, attention significantly affected global motion coherence responses in V4 (p = 0.024), hMT+ (p = 0.038), and marginally in LOC (p = 0.057) (Figure 6). 

Figure 6

 

Attention index at each ROI. Significant attentional modulation of neural responses to global structure was found in across all ROIs for Glass patterns, but only in V4, hMT+, and LOC for motion coherence (Left). However, attentional modulation of the local responses to local dots was only significant in V4 an LOC for motion coherence (Right). The attention index is the normalized difference of the responses in the attended and unattended conditions, [(attended − unattended)/unattended]. An index greater than zero represents an attentional enhancement of response amplitude *p < 0.05, ^∧ p < 0.10. Error bars ±SEM.

Sensitivity to the dot update rate

Feed forward effects of stimulus type

When directed attention was away from the stimulus dots (i.e., unattended), we determined whether responses different across stimulus type or ROI. Results show that that local responses to coherent motion were slightly higher than responses to Glass patterns. We conducted a 5 (ROI) × 2 (stimulus type) repeated measures ANOVA only on responses in the unattended condition. We found a marginally significant main effect of stimulus type, F(1, 15) = 4.179; p = 0.059, but no main effect of ROI, F(1, 15) = 0.709; p = 0.589 or interaction, F(1, 15) = 0.992; p = 0.419. 

Effect of attention

Similar to the analyses performed on global responses, we carried out ANOVAs on the response amplitudes and t-tests on the attention index. These analyses suggest that unlike the global responses, the effect of directed attention on local responses is similar in form and motion stimuli, but that cortical responses to motion coherence were generally greater than cortical responses to form coherence. 

We conducted independent repeated measures 2 (stimulus type) × 2 (attention) ANOVA for each ROI on response amplitude (Figure 5). In V1, we found significant no reliable effects of attention, F(1, 15) = 2.912; p = 0.109, but a marginal effect of stimuli, F(1, 15) = 3.378; p = 0.086. The interaction between these factors were nonsignificant, F(1, 15) = 0.022; p = 0.883. These results in V1 were similar in hMT+, where there was a significant effect of stimulus type, F(1, 15) = 6.028; p = 0.027, but no other effects or interactions (p > 0.20). In V4, we also found a significant main effect of stimulus type, F(1, 15) = 9.367; p = 0.008, with an additional interaction between stimulus type and attention, F(1, 15) = 6.069; p = 0.026. The effect of attention was nonsignificant, F(1, 15) = 2.582; p = 0.129. In LOC, the main effects of stimulus type, F(1, 15) = 5.399; p = 0.035 and attention F(1, 15) = 7.380; p = 0.016, were reliable, but not their interaction, F(1, 15) = 0.006; p = 0.939. In V3a, we found significant effects of attention, F(1, 15) = 9.549; p = 0.007, but no effects of stimulus type, F(1, 15) = 3.063; p = 0.101 or interaction between these factors, F(1, 15) = 0.974; p = 0.339. While directed attention modulated local responses in the V3a, V4, and LOC, these analyses show an effect of stimulus type across V1, V4, LOC, and hMT+ ROIs, suggesting that cortical responses to the local structure of motion coherence are greater than the responses to form coherence. An omnibus 5 (ROI) × 2 (stimulus type) × 2 (attention) ANOVA resulted in a significant main effect of attention, F(1, 15) = 10.162; p = 0.006 and a significant main effect of stimulus type, F(1, 15) = 5.238; p = 0.037. 

To determine the effect of directed attention to the local responses at 30 Hz (1F2), we also calculated an attention index ([Attended − Unattended]/Unattended). We carried out planned t-tests against zero. We found marginal or significant effects of attention to motion stimuli in V3a (p = 0.058), V4 (p = 0.024), and LOC (p = 0.013), and marginal effects of attention to Glass patterns in V1 (p = 0.055), hMT+ (p = 0.075), and LOC (p = 0.070). These results suggest that directed attention to the global structure of the dot patterns weakly modulated the cortical responses to their local structure for both form and motion coherence patterns when compared to the global responses to our stimuli, (Figure 6). 

Attention index of global versus local responses

Finally, we directly compared the attention effect on global and local responses (Figure 6), and our statistical analysis confirm that attentional modulation of global responses were overall stronger than the modulation of local responses. A 2 (hierarchical level) × 5 (ROI) × 2 (stimulus type) ANOVA was conducted and resulted in a marginal main effects of hierarchical level, F(1, 15) = 4.087; p = 0.061, and attention, F(1, 15) = 3.685; p = 0.074. Crucially, there was a reliable interaction between hierarchical level and attention, F(1, 15) = 17.406; p = 0.001. No other main effects or interactions were significant, (p < 0.20). 

More detailed analyses show that the attention index to the local level of coherent motion was reliably higher than the local level of Glass patterns in V1 and V3a. In comparison, the attention index to the global level was higher in Glass patterns than the global level of coherent motion across our ROIs, V1, V3a, V4, hMT+, and LOC. Five independent ANOVAs were conducted at each ROI, 2 (hierarchical level) × 2 (stimulus type). There was a significant interaction between hierarchical level and stimulus type in V1, F(1, 15) = 11.487; p = 0.004, and V3a, F(1, 15) = 9.007, p = 0.009; there were no main effects in these ROIs (p > 0.15). The LOC showed significant main effects of hierarchical level, F(1, 15) = 6.499; p = 0.022, stimulus type, F(1, 15) = 21.348; p = 003, and a marginal interaction, F(1, 15) = 3.157; p = 0.096. In hMT+, there was a marginal main effect of stimulus type, F(1, 15) = 3.439; p = 0.083. In V4, there were no reliable effects. 

Discussion

Random-dot kinematograms and Glass patterns are useful experimental tools because they have a logical hierarchical structure (local vs. global cues) that can be used to find and characterize cortical mechanisms that reflect a similar hierarchical relationship. Our use of the frequency-tagging paradigm with these stimuli provides simultaneous access to responses at both the local and global levels of the stimulus, and our source-imaging procedure allows us to record both types of response in visual areas that lie at different levels of the visual hierarchy. We are thus in a position to study both cue and attention effects across the visual pathway. 

Attention effects across the cortical hierarchy

By recording responses to both the local and global components of form and motion stimuli, we found that the hierarchical level (global or local) of the stimulus encoding is more predictive of where attention will modulate the response than is anatomical level (e.g., V1, V4): Significant modulatory effects of attention were seen, for example, for the global responses to Glass patterns in all areas, but not for the local responses from the same areas. The prevailing consensus is that attentional effects become stronger in higher-order visual areas (Maunsell, 2004). Specificity to global features such as the pattern of optic flow has primarily been observed in areas that are hierarchically distant from V1 (Bartels, Zeki, & Logothetis, 2008; Duffy & Wurtz, 1995; Wall, Lingnau, Ashida, & Smith, 2008). Our results with hierarchical stimuli suggest that a more nuanced view is in order: both high- and low-level signals can coexist in the same anatomical structure, but attention can selectively modulate the higher-level, global information, rather than all of the information available in an area. 

Attention to local versus global features

Why might attention affect global versus local levels of processing differently? One factor may be differences in salience of the two types of cues. Previous research indicates that both coherent motion (Burr & Santoro, 2001) and Glass pattern detection thresholds (Palomares, Pettet, Vildavski, Hou, & Norcia, 2010) do not depend on contrast once the local elements themselves are just barely visible. In other words, local cue extraction does not appear to be the critical limitation on performance in global coherence tasks, and this alone would suggest separate detectors for processing local and global features. More importantly, previous research has suggested that attention can act to increase response amplitude in much the same way as increasing stimulus contrast does (Martinez-Trujillo & Treue, 2002; Pooresmaeili, Herrero, Self, Roelfsema, & Thiele, 2010; Reynolds, Pasternak, & Desimone, 2000; Reynolds & Desimone, 2003; Thiele, Pooresmaeli, Delicato, Herrero, & Roelfsema, 2009; Treue, 2004) and that the effect of attention is maximal for weak signals (Reynolds et al., 2000). However, if the contrast of local, low-level cues is already high, as in our stimuli, boosting the neural representation of the local cues via attention would be expected to have little effect. 

Directed attention to feature conjunctions of separate features such as color and orientation has been found to boost SSVEP responses in succession by task relevance: response amplitude to a conjunction target is greatest, followed by responses to non-targets that shared features to the target, and then to non-targets with non-shared features (Andersen, Hillyard, & Muller, 2008). Similar effects of directed attention have been found for evoked potentials to color and spatial location (Andersen, Fuchs, & Muller, 2009) and for auditory and visual information (Saupe, Schroger, Andersen, & Muller, 2009). Our current results are consistent with these findings—albeit dependent on stimulus type. Attending to the global coherence of Glass patterns robustly enhanced the neural responses to the task-relevant global attribute, whereas neural responses to the task-irrelevant local feature are not. Attending to the global coherence of motion stimuli only modestly increased the neural responses to the global attributes, but reliably increased the neural responses to the local features of the stimuli. 

Intrinsic salience of global form and motion

One of our major results is that attention has a greater effect on responses to global structure than it does on the responses to local structure—particularly in the case of the global form stimulus. The same mechanism by which attention operates when salience is low may also underlie this effect as well. Recall that when directed attention was targeted towards the letter task, responses to global motion were larger than those to global form. This suggests that coherent motion is an intrinsically more salient stimulus than global form. 

The idea that motion processing proceeds largely in the absence (or division) of attention was a touchstone of the older literature on attention and motion (McLeod, Driver, & Crisp, 1988; Nakayama & Silverman, 1986). Neural responses to motion coherence (Hou, Gilmore, Pettet, & Norcia, 2009) are seen earlier in development than global form responses (Palomares et al., 2010), and are more robust to prolonged visual deprivation (Fine et al., 2003). Moreover, motion captures attention automatically (Franconeri & Simons, 2003), suggesting that the coherent motion system is, perhaps, computationally simpler or more ‘hard-wired' than the form system. Indeed our own data, in conditions where participants withdrew their attention from the dots, shows that the global structure of motion coherence elicits stronger neural responses than the global structure of Glass patterns. 

However, some evidence suggests that attention does play a role in motion processing (Raymond, 2000). For example, attention to a high-load task at fixation reduces hMT+ activation in fMRI, but a low load task does not (Rees, Frith & Lavie, 1997). Within the framework of the normalization model of attention (Reynolds & Heeger, 2009), a low salience target will be more susceptible to attentional modulation than a high salience target. In this model, salience is the effective strength of a feature relative to a baseline condition. If we equate salience with coherence, attention to dot motion with 100% coherence produced less neural enhancement than to dot motion with 50% or 20% coherence (Handel, Lutzenberger, Thier, & Haarmeier, 2008). In our case, global form in Glass patterns is less salient than global motion due to impoverished representation of local orientation cues. 

The present study is the first to compare form and motion processing directly. Due to the very high degree of similarity between our coherent motion and Glass pattern stimuli and corresponding tasks, the comparison is a very focused one and we clearly show that extraction of global structure in Glass patterns is more dependent on focused attention than is the extraction of coherent motion. This is perhaps not surprising, as it has been known for many years that attention, in the form of priming, can strongly influence the interpretation of degraded or ambiguous form information (Porter, 1954). More recently Kourtzi and Huberle (2005) have found that integration of collinear elements into contours, a task that bears a resemblance to the Glass pattern task, is modulated by attention in LOC, as we find here. Effects of attention were also observed in early visual areas for oriented elements, formed by dipoles in Glass patterns. Attentional modulation of global form in V1 is likely due to feedback mechanisms (e.g., Thielscher, Kolle, Neumann, Spitzer, & Gron, 2008) that mediate segmentation of target elements from their background as well as facilitate integration of global form. 

The human neuroimaging literature has suggested that the largest effects of attention on a given feature are seen in areas that are specialized to process that feature (Corbetta, Miezin, Dobmeyer, Shulman, & Peterson, 1990; Huk & Heeger, 2002; Schoenfeld et al., 2007; see also Katzner, Busse, & Treue, 2009 for a single-unit study in macaque). In our data, hMT+, an area strongly involved in motion processing (Huk & Heeger, 2002), does not stand out as the sole area with attentional modulation of motion coherence stimuli. Instead, we found that the attentional effect on the global responses to coherent motion was distributed across V4 and hMT+ and marginally in the LOC. Our finding that attentional modulation of motion stimuli in ventral visual areas responsible for shape processing are consistent with recent findings of motion direction selectivity in V4 (Hong, Tong, & Seiffert, 2012) and the strong percept of shearing spatial structure that is engendered by rotation of dots through a constant displacement. 

Conclusion

Our study examined two classic dichotomies in spatial vision—local versus global cue processing and form versus motion processing—using a method that allows direct and unambiguous access to the relevant neural substrates. By coupling high-density frequency tagged EEG and fMRI, we were able to quantify attentional modulation of local and global signals due to form and motion stimuli in well-defined cortical regions of interest. Directed attention appears to modulate strongly global responses in Glass patterns and more weakly in motion coherence. Local responses were modulated by attention in both form and motion stimuli. The differential effects of attention at the different hierarchical levels of the stimulus and visual pathway are consistent with a model in which attention acts on the subset of the feature-specific neurons that are required to perform the global integration task. These subsets appear to be distributed in different cortical locations for the different tasks and stimuli we examined here. 

Acknowledgments

This research was funded by the National Institutes of Health (EY014536, EY06579, EY19223) and the C. V. Starr Fellowship Fund. The authors thank Margaret McGovern for recruiting participants. Preliminary data from this manuscript were presented at the 2009 Vision Sciences Society Conference in Naples, FL. 

Commercial relationships: none. 

Corresponding author: Melanie Palomares. 

Email: [e-mail]. 

Address: University of South Carolina, Columbia, SC, USA. 

Footnotes

¹  In five of our observers, the run duration of 240 s consisted of an alternation between 12 s of coherent-random pattern and 12 s of random-random pattern. During periods of random stimulation, the observers detected threshold-level increments in global coherence. Data from the random-random stimuli were not included in the analysis here.