Effects of attention on perceptual direction tuning curves in the human visual system

Abstract

In sensory neurophysiology, reverse correlation analyses have advanced our understanding of the spatio-temporal structure of receptive fields (RFs) and the tuning properties of individual neurons. Here, we used a psychophysical variant of the motion reverse correlation technique to investigate how visual selective attention influences human perceptual tuning curves for direction of motion. Direction tuning functions were computed by reverse correlating speeded target-present responses of human observers with a random sequence of brief, fully coherent motion impulses. We found that attention enhanced the amplitude of perceptual tuning curves for direction of motion, while tuning width remained unaffected. Furthermore, the full direction tuning profile across time could be well fitted by a separable model of direction and temporal tuning. Attention enhanced both the direction tuning and its temporal profile, without shifts or changes in shape. Thus, attention exerts a multiplicative effect on human perceptual tuning curves for direction of motion. An analysis of second-order correlations revealed a boost in the likelihood of responses to the target direction when it was followed by a motion impulse in the opposite direction. This perceptual effect might be mediated by biphasic neurons that are preferentially activated by a rapid succession of opposite motion directions.

Introduction

At any given moment, our visual system is deluged with much more information than can be fully processed. To overcome this limitation, we can use attention, which selectively modulates the processing of sensory information according to behavioral relevance. While it is well known that attention can speed behavioral responses (e.g., Posner, 1980) and improve performance in detection and discrimination tasks (e.g., Carrasco, Penpeci-Talgar, & Eckstein, 2000), the mechanisms underlying these attentional benefits have been subject of substantial debate. Here, we examined the mechanism by which attention modulates perceptual tuning curves for direction of motion in human observers. 

Tuning curves, i.e., the average activity of individual neurons as a function of the value of a particular stimulus parameter, have been widely used to characterize the selectivity of neurons in visual and other sensory areas of the brain (Adrian, 1926). For instance, a direction tuning curve describes a neuron's response to visual motion in various directions (Albright, 1984; Dubner & Zeki, 1971). Typically, these tuning curves have a bell-shaped form and can be well fitted by a Gaussian model function. Two important parameters of such tuning curves are their amplitude, i.e., the difference between the strongest and weakest response, and their width (Butts & Goldman, 2006). While the amplitude reflects the dynamic range, the width of the tuning curve represents the selectivity of a neuron. Tuning curves are not a static property of neurons but can be modulated by cognitive influences, such as selective attention. In general, it has been found that the attentional modulation consists of a multiplicative scaling of tuning curves, without substantial changes in the tuning width (Cook & Maunsell, 2004; Martinez-Trujillo & Treue, 2004; McAdams & Maunsell, 1999; Treue & Martinez-Trujillo, 1999). 

Spike-triggered averaging (STA) is one of several different methods for characterizing a neuron's receptive field properties (Marmarelis & Marmarelis, 1978). Here, the visual system is presented with a pixel-by-pixel contrast-modulated white noise stimulus. The spike-triggered average stimulus, or kernel, corresponds to the best linear approximation of the neuron's spatio-temporal impulse response (Ringach & Shapley, 2004; Victor, 2005). Furthermore, subspace reverse correlation can be used to assess the temporal dynamics of feature selectivity such as tuning for orientation (Ringach, Sapiro, & Shapley, 1997) or direction of motion (Borghuis et al., 2003). To analyze motion tuning at high temporal resolution, the motion reverse correlation (MRC) paradigm uses very brief, fully coherent motion impulses in one of several different directions that are rapidly presented in a random sequence. Cross-correlating the neural response with this motion impulse sequence reveals the temporal dynamics of motion selectivity, expressed as the probability of observing a specific motion impulse direction at a specific point in time preceding an action potential. Based on these correlation functions, direction tuning curves can be constructed at each instant of time. 

Analogously, behavioral reverse correlation methods measure tuning functions of psychophysical observers (e.g., Ahumada & Lovell, 1971; Eckstein, Shimozaki, & Abbey, 2002; Mareschal, Dakin, & Bex, 2006; Murray, Sekuler, & Bennett, 2003; Neri & Levi, 2008; Ringach, 1998; Tadin, Lappin, & Blake, 2006). Such perceptual tuning curves assess the characteristics of information processing on a macro scale, i.e., on the level of the entire observer, and allow a comparison with neurophysiological tuning curves (Neri & Levi, 2006). Here, we use a psychophysical variant of the MRC paradigm to obtain perceptual tuning curves for direction of motion in human observers and compare the tuning profiles between two attentional conditions. We find that attention multiplicatively modulates perceptual tuning curves in human observers without changes in shape or shifts in time. 

Materials and Methods

Apparatus

The experiment was conducted in a dimly illuminated and quiet testing room. Stimuli were presented on a VGA monitor (Lacie, electron22blueIV) operating at a refresh rate of 85 Hz and a spatial resolution of 40 px/deg. Stimulus presentation and recording of responses was controlled by custom-made software running on an Apple Power Mac G4. Using a chin-rest or bite-bar, subjects were placed at a viewing distance of 114 cm from the monitor. 

Stimulus

Figure 1 shows a schematic drawing of the stimulus and the trial sequence. On each trial, subjects fixated a central fixation square (0.05 × 0.05 deg, 75 cd/m²) presented on a black background (luminance 0.05 cd/m²). A stationary, circular aperture (3.75 deg diameter) containing moving gray dots (luminance 5.5 cd/m²) was shown centered 3.75 deg to either the left or the right of the fixation square. To minimize the effects of adaptation, the position of the circular aperture was alternated from trial to trial. Before the start of a trial, the dots (density: 15 dots per deg²; dot size: 0.2 × 0.2 deg) were stationary to indicate the location of the upcoming stimulus. After initiation of the trial, the dots moved in a random sequence of brief (94 ms), fully coherent motion impulses in one of 24 different directions, sampling the full 360 deg in steps of 15 deg. The speed of motion was 12 deg/s. Individual motion impulses were separated by a blank screen of 47 ms. This blank interval between motion impulses improves the sensitivity (d′) for the target direction (see Paradigm), as determined in a pilot study in which the duration of the blank interval was varied systematically between 0, 47, and 94 ms (ANOVA, p = 0.012, Greenhouse–Geyser corrected). Each trial was composed of 107 motion impulses and lasted 15 seconds. During the course of the trial, the fixation square could briefly change in luminance up to 3 times for a duration of 120 ms. In two different conditions, run in different sessions, the luminance changed by 75 (the fixation square disappeared completely) or 40 cd/m², respectively. 

Figure 1

 

Stimulus and task. Observers performed a dual task. The primary task consisted of counting the number of luminance changes in the fixation point and reporting the outcome (0–3 changes) at the end of each trial (15 s duration). For the secondary task, observers were instructed to make speeded responses to a designated motion direction embedded in a random sequence of brief, fully coherent motion impulses. In this example, the target direction is rightward motion, which is indicated by the black arrowhead pointing to the right in the stream of motion impulses. The dot pattern containing the motion impulse sequence was presented to the left or the right of the fixation point, with the side changing systematically from trial to trial to minimize effects of adaptation. In two different conditions, the luminance change in the fixation point was either substantial or more attenuated, thus leaving almost “full” or only “poor” attentional resources for the motion impulse task.

Paradigm

To manipulate the amount of attention, a dual-task design was used. For the primary task, subjects were instructed to count the number of luminance changes of the fixation point and to indicate, at the end of the trial, the number of changes by pressing one of four keys with the right hand on a computer keyboard (NumPad keys 0, 1, 2, 3 corresponding to the number of counted changes; non-speeded task). After each trial, subjects received auditory feedback indicating performance in this primary task. Additionally, subjects were instructed to covertly direct their attention to the peripheral RDP in order to make speeded responses to motion impulses moving in a designated target direction (secondary task) using their left hand. During the trials, eye-movements were monitored using an infrared eye-tracking system (ISCAN ETL-200). Due to the long duration of the trials, it was not feasible to automatically abort trials when the gaze left the fixation window, e.g., due to eye blinks. However, the experiment was interrupted and subjects were re-instructed if we observed drifts or saccades away from the center and toward the target location (a fixation radius of 1.5 deg was plotted around the fixation point for online comparison of eye movement signals on a control monitor). 

The target direction was rightward motion for half of the subjects; for the other half, leftward motion was used. Since the sequence of motion impulses was random, on average 1/24 of the motion impulses were targets. Each key press for the secondary task was acknowledged by a neutral sound. For online analysis, key presses within 100–1000 ms after a motion impulse in the target direction were counted as correct responses, while all other key presses were counted as “false alarms”. After the last motion impulse in each trial, a blank screen was presented for 550 ms to allow for responses to the last few motion impulses. At trial end, a summary of the subjects' performance for the current trial was displayed in the upper left corner of the screen, giving the number of hits, the number of targets presented, and the number of responses outside the response time window. 

The experiment was run in two sessions on different days. Each session lasted for about 1.5 h and consisted of 3 blocks of 32 trials each. At the beginning of each session, subjects performed an additional practice block (32 trials) to get used to the task requirements. The practice blocks were not included in the data analysis to ensure that tuning functions were based on a constant level of performance. The two sessions differed in the magnitude of the luminance change of the fixation point. The difficulty of the smaller luminance change (40 cd/m²) was chosen to yield about 75–80% correct responses as determined in a pilot experiment in which we systematically varied the intensity of the change while subjects also performed the motion impulse task. To maximize the difference between conditions, we made the luminance change as large as possible (75 cd/m², i.e., a complete disappearance) in the other session. We termed the condition with the larger luminance change “full attention” condition because the size of the luminance change left almost full attentional resources for the motion impulse task. In contrast, the condition with the smaller luminance change was called “poor attention” condition because counting the more attenuated luminance changes left less attentional resources for the motion impulse task. While it might be argued that both luminance changes are rather large, the secondary, motion detection task, was very difficult, such that it could not be performed in combination with more difficult luminance changes. The order in which subjects performed the two attentional conditions was counterbalanced across subjects. Eighteen subjects participated in the experiment (age 15–34, 13 females). One subject was excluded from data analysis because he performed very poorly in the luminance detection task despite of substantial training (less than 40% of correct trials). 

Data analysis

Behavioral performance

For both luminance and motion impulse tasks, we first performed an analysis of behavioral responses independent of the reverse correlation analysis. In the luminance task, we calculated the percentage of correct responses by dividing the number of trials in which the correct number of luminance changes was reported by the total number of trials. In the motion impulse task, the percentage of correct responses corresponds to the total number of target key presses within the response time window (200–800 ms after target impulse) divided by the number of targets in the sequence summed across trials. Furthermore, we computed the percentage of false alarms by dividing the number of target key presses occurring outside any response time window by the total number of non-target motion impulses. Subtracting the z-transformed false alarm rate from the z-transformed hit rate yielded a sensitivity measure d′ for each subject and attentional condition (Green & Swets, 1966). 

Correlograms

For each attention condition separately, we calculated correlograms including only those trials with correct performance in the primary task. Excluding the trials with incorrect performance in the primary task is important because subjects might not have devoted enough attentional resources to the luminance task when they gave a wrong response, thus undermining the intended manipulation of attention. Analogously to the MRC method in single-unit physiology (Borghuis et al., 2003), we determined the onset times and directions of all motion impulses preceding and following each response by −1000 to 300 ms. For each direction of motion separately, these motion impulses were counted at their specific onset time relative to the response (50 ms bin width). These correlograms were normalized by dividing the number of impulses in each bin by the sum of button presses in the two attentional conditions (Ringach, 1998). Using a single value for the normalization is crucial for comparing tuning curves across time and across attentional conditions. The correlograms were low-pass filtered in time using a cut-off of 4 Hz, which removes most of the noise without affecting the global shape of the correlation function. The amount of remaining noise can be assessed at the time bins following the response; the distribution of values should ideally be flat since responses cannot be influenced by future motion directions. 

Tuning curves

Perceptual tuning curves were constructed for each observer by plotting the probability values of the 24 correlograms at the time of the peak of the most likely direction as a function of direction of motion. Across observers, these tuning curves typically exhibited two peaks, with the main peak located at the target direction, and a second much smaller peak located at the opposite direction of motion. These tuning curves were fitted with a sum of two Gaussians, shifted by 180 deg and differing in amplitude, but not in width:  

$p\left(x\right)=B+A_1{e}^{-0.5{\left(\frac{x-C_1}{\sigma }\right)}^2}+A_2{e}^{-0.5{\left(\frac{x-(C_1\pm 180)}{\sigma }\right)}^2},$

(1)

where x are the 24 directions used in the experiment, B is the baseline, A₁ and A₂ are the amplitudes, C₁ is the main peak of the sum of the Gaussians, and σ the half width or selectivity of the tuning curve. To obtain the location of the second peak, 180 deg were added to or subtracted from the main peak, respectively, depending on whether the subject was assigned a rightward (90 deg) or leftward (270 deg) target direction. Parameters B, A₁, and A₂ were allowed to vary between 0 and 1, C₁ was restricted to values within ±90 deg of the assigned target direction (to converge on the main, and not the second peak), and σ was unrestricted. Additional free parameters for the location or the width of the second peak, or both, did not significantly improve the amount of explained variance as assessed by nested hypothesis testing using the F-test (see Auxiliary Material Table A1). We assessed the significance of the attentional effects on the parameters of the tuning curves in two different ways. First, we fitted each attention condition separately and compared the fitted parameters across conditions using paired sample t tests. Second, we performed successive nested hypothesis tests using the F-test. Here, we examined, for each individual subject, if the fit to the data in the poor attention condition significantly improved by scaling the values obtained for the parameters in the full attention condition. 

Second-order correlations

To investigate whether the second peak at the opposite (anti-target) motion direction in the perceptual tuning curves was the result of an interaction with the preceding motion impulse, we correlated responses to combinations of two successive motion directions (Perge, Borghuis, Bours, Lankheet, & van Wezel, 2005). This second-order analysis investigates how specific stimulus pairs interact to affect the behavioral response. The steps involved in the second-order analysis were identical to those used for the first-order correlation described above, with the following exceptions. First, we pooled three neighboring directions of motion to increase the signal/noise ratio, resulting in 8 different direction bins. Thus, the 0 deg bin contained the target bin and those deviating +/- 15 deg from it, the 45 deg bin contained bins deviating 30 to 60 deg from the target, etc. Second, we correlated the responses with pairs of successive motion impulses, resulting in a total of 8² correlation functions. The temporal bin for each pair of motion impulses was determined by the latency between the second impulse in the combination and the response. To quantitatively analyze the second-order correlation profiles, we performed two analyses. First, we correlated the fitted amplitude of the second peak with the peak probability of responses to the target-opposite pair. Second, we tested whether responses were more likely to target-opposite pairs than to pairs in which the target was followed by any other direction of motion. Again, we chose the latency corresponding to the peak probability for the target-opposite impulse pair. 

Separability analysis

To investigate the effects of attention on the time course of direction processing, we modeled each subject's full time–direction tuning surface in each attentional condition by multiplying two functions representing a temporal and a direction tuning curve, respectively (separable model). First, we normalized each time–direction tuning surface by subtracting the mean across time and direction. Then, we applied singular value decomposition (SVD) (Benucci, Frazor, & Carandini, 2007; Mazer, Vinje, McDermott, Schiller, & Gallant, 2002), transforming the measured time–direction tuning surface into the form U^TsV, where U and V are sets of orthogonal vectors that completely reconstruct the original matrix when combined with the singular values in s. In case of full separability, the complete tuning surface can be represented by multiplying the first row eigenvector by the first column eigenvector, so only the first diagonal term of s will be nonzero. Separability was quantified by the relative magnitude of the first singular value using the following index:  

$si=\frac{\lambda (1{)}^2}{\sum _i\lambda (i{)}^2},$

(2)

where λ(i) is the ith diagonal term of s; si ranges from near 0 (nonseparable) to 1 (separable) (Mazer et al., 2002). The first singular values were compared across attention conditions using a paired sample t test. The direction and temporal tuning functions consisting of the first row and first column eigenvectors were compared across attention conditions using paired sample multivariate permutation tests (Blair, Higgins, Karniski, & Kromrey, 1994). 

Results

Behavioral performance

In both attentional conditions, performance for counting the luminance changes of the fixation point was very high. On average, counts were correct in 88.5% (s = 7.0) and 85.0% (s = 10.0) of the trials in the full and poor attention condition, respectively (paired t test, p = 0.21). In case of incorrect counts, subjects missed changes in 8.1% (s = 5.1, full) and 12% (s = 9.4, poor attention condition) of the trials, while their counts exceeded the number of changes in 3.4% (s = 2.7, full) and 3% of trials (s = 2.3, poor attention condition). None of the incorrect outcomes differed between attentional conditions (paired t tests, all p > 0.14). 

In the motion impulse task, observers correctly detected more target impulses in the full attention condition (37%, s = 9.5) compared to the poor attention condition (32.6%, s = 8.7) (paired t test, p = 0.0019). At the same time, the percentage of false alarms, i.e., button presses outside the response time window, was not different between the full (2.5%, s = 1.2) and the poor attention condition (2.3% responses, s = 1.2; paired t test, p = 0.34). This pattern of results was confirmed by the calculation of sensitivity (d′), which was higher in the full (1.67, s = 0.19) compared to the poor attention condition (1.56, s = 0.2; paired t test, p = 0.0006). Reaction times (RTs) to targets did not differ between attentional conditions (full attention: 491.1 ms, s = 28.9; poor attention: 493.1 ms, s = 34.6; paired t test, p = 0.67). Comparable results were obtained when choosing different limits for the response time window; including the limits used for online feedback during task performance (100–1000 ms). On average, observers responded to 4.2% and 4.0% of the motion impulses in the full and poor attention condition, respectively. This behavior is optimal in the sense that there is no difference between the average number of responses per impulse and the target probability of 4.2% (p > 0.3, for both attention conditions). In summary, these results indicate that we successfully manipulated attention: in order to achieve the same level of performance in the primary task (luminance detection), observers' motion sensitivity was degraded in the poor compared to the full attention condition. This cannot be explained by an overall reduction in the observers' willingness to give a response in the poor attention condition since the number of false alarms was not different between the two conditions. 

Correlograms and tuning curves

Correlograms of a representative observer are depicted in Figure 2. They show that this observer based his responses on motion impulses which had occurred between 600–200 ms before the response. Most importantly, the correlation is strongest for motion impulses in the target direction (0 deg), yet the observer also responded frequently to adjacent directions (±15 deg). Interestingly, directions opposite to the target direction lead to responses more frequently than directions deviating by ±60–150 deg from the target (thin black lines). Furthermore, the amplitude of the correlation functions is lower in the poor attention condition than in the full attention condition. The correlogram is extended to positive latencies where the stimulus is presented after the response. Since, here, responses cannot be causally related to the stimulus, the variability of the curves provides a measure of the noise inherent to the experiment (Borghuis et al., 2003). 

Figure 2

 

Correlograms for a representative observer obtained by cross-correlating the behavioral responses with the motion impulse sequence in conditions of full (A) and poor attention (B). Each line represents the time course of the probability of observing a particular direction of motion, with time 0 being the time of the behavioral response (solid vertical line). The correlation functions in both conditions change prominently between 600 and 200 ms preceding the response, indicating that observers based their response on motion impulses occurring in this time window. Correlations are strongest for the motion impulses in the target direction (0 deg, red line) and decrease as directions deviate from the target motion impulse. Remarkably, responses to the direction opposite to the target direction (cyan line) are not as strongly suppressed as those to directions differing only by 90 deg. Amplitudes in the full attention condition are higher than amplitudes in the poor attention condition.

We computed direction tuning curves to quantify the effects of attention. First, we plotted the probabilities for all directions at the time corresponding to the peak amplitude of the most likely direction (i.e., the time slice indicated by the dashed vertical line in Figure 2), which coincided with the target direction in all subjects. Figure 3A shows the tuning curve constructed from the correlograms in Figure 2 at time −425 ms. The main peak of the resulting tuning curve is centered on the target direction, and a second smaller peak is located at the opposite direction. Tuning functions for each of the observers are plotted in Auxiliary Figure A1. We fitted these tuning curves with a sum of two Gaussians (see Methods, Tuning curves). The fit captured the data well with an average R² between observed and fitted values of 0.96 (s = 0.04) and 0.92 (s = 0.13) in the full and poor attention condition, respectively (paired t test, p = 0.17). Across observers, the amplitude of the second peak was significantly different from zero (0.0075, s = 0.004, t test, p = 9.56·10⁻⁷) but also significantly smaller than the amplitude of the main peak (0.031, s = 0.009, paired t test, p = 9.08·10⁻¹⁰). The designated target direction was very close to the best-fit center of the main peak, with an average absolute difference of 1.6 deg (s = 1.7) and 2.3 deg (s = 1.8), in the full and poor attention condition, respectively, indicating that the psychophysical variant of the MRC method reliably extracts the designated target direction. Averaged across attentional conditions and observers, the half width of the tuning curve (σ in Equation 1) was 17.5 deg (s = 4.8). 

Figure 3

 

Effects of attention on perceptual tuning curves. Probabilities in the full attention condition are plotted as asterisks, whereas probabilities in the poor attention condition are indicated by open circles. (A) Perceptual tuning curves for direction of motion obtained from the correlograms of the observer shown in Figure 2 at the time of the highest peak (−425 ms). Tuning curves were fitted using a sum of two Gaussian functions, shifted by 180 deg. The fit is depicted as a solid vs. dotted trace for the full and poor attention condition, respectively. The amplitude of the fitted model function is ∼41% higher in the full compared to the poor attention condition. (B) Average across observers. Note that the gray lines only connect the data points and do not represent a fit. Bars represent bootstrapped within-subjects 95% confidence intervals for each data point (Loftus & Masson, 1994; Wright, 2007) and are displaced by 3 deg in the poor attention condition for display purposes. Similar to the individual observer, across observers the amplitude in the full attention condition is higher than the amplitude in the poor attention condition.

Comparing the fitted parameters across attention conditions revealed a significant difference only in the amplitude of the main peak without concomitant changes in the tuning width. Full attention on the motion detection task increased the amplitude of the main peak by ∼21% (full attention: 0.033, s = 0.008, poor attention: 0.028, s = 0.011, paired t test, p = 0.003), while the average width was the same in both attention conditions (full attention: 17.2 deg, s = 4.3, poor attention: 17.8 deg, s = 6.1, paired t test, p = 0.56). We also did not find significant differences between the attentional conditions for any of the other fitted parameters. The distribution of effects across observers is summarized in Figure 4. In perfect agreement with these results, the nested hypothesis tests (see Methods, Tuning curves) revealed that allowing different amplitudes for the main peak in the two attentional conditions significantly improved the model fits in 15 of the 17 subjects tested (median p value of the F-Test = 6.5·10⁻⁶). In contrast, letting any of the remaining parameters vary between attentional conditions did not consistently improve the model fits across subjects. 

Figure 4

 

Distribution of attentional effects for each of the 5 tuning parameters. The circles represent the data for each individual subject, the box spans the upper and lower quartiles, and the median is indicated by the thick line. The notch of the box represents a 95% confidence interval for the median. Whiskers extend to 1.5 times the interquartile range. Effects of attention on (A) amplitude of the main peak, (B) width, (C) baseline, (D) amplitude of the second, smaller peak, (E) difference between designated target direction and center of the fit. Full attention to the motion task only increased the amplitude of the main peak without affecting any other aspect of the tuning curve.

Second-order correlation

To explore whether the unexpected, second peak at the direction of motion opposite from the target reflects an influence of the motion impulse directly preceding this “anti-target” we performed a second-order correlation (see Methods, Second-order correlations). Figures 5A and 5B show the results of this analysis for two different observers (subjects 1 and 12; see Auxiliary Figure A1) in the full attention condition. The high values in the third of the major columns at ∼450 ms (framed in red in Figures 5A and 5B) show that the subjects responded to the target direction irrespective of the direction of the preceding impulse. The parallel light-gray streaks in the third subcolumn of each column at ∼300 ms represent responses when the first motion impulse in the pair moved in the target direction. Responses to the first impulse appear at a shorter latency in this figure since latencies are determined by the difference between response and onset of the second motion impulse. The latency difference between responses to the first and second impulse in the pair directly depends on the interval between successive motion impulses (here: 94 ms + 47 ms blank = 141 ms). Most important, the bright streak in the third subcolumn of the seventh major column of subject 1 (marked by a white arrow in Figure 5A) compared to the corresponding subcolumns in the other major columns shows that this subject's pronounced 2nd peak in the tuning function reflects an increased response probability when the target was immediately followed by the “anti-target” impulse. This specific enhancement is much less evident in the data of subject 12 (Figure 5B) who has only a weak second peak (if at all). Across subjects, the likelihood of such responses to the target-opposite pair is strongly correlated with the amplitude of the second peak (full attention condition: r = 0.75, p = 1.64·10⁻⁵; poor attention condition: r = 0.85, p = 5.77·10⁻⁴). Furthermore, in both attentional conditions, subjects responded more often when the target direction was followed by the opposite direction compared to when it was followed by the average across all other directions of motion (paired t test, full: p = 0.0001; poor: p = 0.002). Thus, the second peak observed at the opposite direction of motion indicates that an “anti-target” stimulus following a target impulse boosts the subjects' likelihood to respond. 

Figure 5

 

Results of the second-order correlation for a subject with a pronounced peak (A, subject 1) and for a different subject whose data did not show a second peak (B, subject 12). Brightness indicates the probability of a specific pair of motion impulses preceding the behavioral response. The latency is determined by the onset of the second motion impulse in the pair relative to the response time. The columns of the x-axis (major divisions) are the directions of motion of the second impulse in the pair relative to the target direction; the subcolumns (minor divisions) are the directions of motion of the first impulse in the pair relative to the target. The large red box indicates responses when the second impulse moves in the target direction. The arrow in A highlights the enhanced probability of responses when the first impulse moves in the target direction and the second impulse moves in the opposite direction.

Separability analysis

So far, we have only assessed the effects of attention at the time of strongest direction tuning. One interesting aspect of reverse correlation methods, however, is the possibility to investigate the effects of attention across time. To do so, we fitted the direction tuning data across time with a separable model in which the tuning surface in each attentional condition is reconstructed as a product of two separate functions, one for direction and one for time. These direction and temporal tuning curves can then be compared across attentional conditions. Figure 6 depicts the direction tuning surface across time and the best separable model for direction and temporal tuning for the same subject as shown in Figures 2 and 3. 

Figure 6

 

Time–direction tuning surface, separable model and residuals for each attention condition of the observer shown in Figures 2 and 3. The top row shows the time–direction surface obtained in the full attention condition (A, same data as in Figure 2A), the prediction of the separable model (B) and the residuals (C), i.e., the difference between the data and the predictions of the separable model. The bottom row uses the same format for the poor attention condition (D, same data as in Figure 2B; E, prediction of the separable model; F, residuals). All plots are scaled to the same values, white represent higher, black lower probabilities. For this subject, separability indices were 0.71 and 0.67 in the full and poor attention condition, respectively.

The separable model captures the tuning data well because the residuals do not show a regular pattern. To quantify the separability of the data, we computed the separability index si which ranges from 0 (inseparable) to 1 (separable) (Equation 2). The data shown in Figure 6 have an index of 0.71 (full attention condition) and 0.67 (poor attention condition), indicating a high degree of separability. Averaged across observers and conditions the si was 0.67 (s = 0.12). 

Figure 7 compares the separable model for the two attentional conditions. The effect of attention is captured by the model in a higher singular value scale factor for the full attention compared to the poor attention condition (full attention: 0.08 (s = 0.016), poor attention: 0.069 (s = 0.022); paired t test, p = 0.005). The two average modeled direction tuning curves (Figure 7A) are very similar to the averaged tuning curves at the peak of the temporal tuning (compare to Figure 3B). Before multiplication with the singular value scale factor, there is no difference between the modeled direction tuning curves in the two attentional conditions (multivariate permutation tests, p > 0.18). The modeled temporal tuning curves (Figure 7B) peak at the same time bin in each attention condition (paired t test, p = 0.67). Moreover, multivariate permutation test also do not reveal any change in shape or shift in the temporal tuning curves by attention (p = 0.28). Since a resolution of 50 ms might be too coarse to reveal subtle shifts in the temporal tuning function we repeated the analyses with a temporal binning of 10 ms, and obtained qualitatively similar results, i.e., a highly significant difference in the scale factors (p = 0.0073) between the attentional conditions, without any difference in shape and any shift of the modeled tuning functions (multivariate permutation test, p > 0.17, difference in peak bin p = 0.21). In summary, according to the separable model attention multiplicatively enhances both direction and temporal tuning curves without a significant change in shape or temporal shift. 

Figure 7

 

Effects of attention according to the separable model. (A) Modeled direction tuning curve. (B) Modeled temporal tuning curve. For illustration purposes, the square root of the modeled singular value for each attention condition is multiplied with each subject's direction and temporal tuning curves before averaging. The effect of attention consists of a higher scale factor in the full compared to the poor attention condition in the absence of any changes in shape or any temporal shifts of the tuning curves.

Discussion

In this experiment, we used a psychophysical variant of the MRC technique (Borghuis et al., 2003; see also Srinivasan, Jin, Stange, & Ibbotson, 1993, for an earlier foundation and Neri, 2006, for an application of the method in the invertebrate) to obtain perceptual tuning curves for direction of motion in human observers with high temporal resolution. Comparing these perceptual tuning curves between conditions differing in attentional load, we found that visual selective attention increased the amplitude of both direction and temporal tuning curves of motion processing in the absence of changes in tuning width or shifts in time. 

Comparison to physiological studies

These results are consistent with previous single-unit studies of attention. Electrophysiological studies of neurons in area MT have reported that both spatial and feature-based attention exert a multiplicative influence on tuning curves for direction of motion without changes in the tuning bandwidth (Cook & Maunsell, 2004; Treue & Martinez-Trujillo, 1999), and similar results have been obtained for different visual stimulus dimensions and in different areas of visual cortex (V1, V4) (McAdams & Maunsell, 1999; McAdams & Reid, 2005). Also, it has been shown that attention multiplicatively modulates the temporal RF of neurons in areas MT and V1 without any shifts or changes in shape (Cook & Maunsell, 2004; McAdams & Reid, 2005). The amount of gain change found in electrophysiological studies of attention depends on the exact stimulus configuration, type of attention, and visual area. Estimates of gain changes in area MT range from 10% for purely spatial attention to 25% for combinations of feature-based and spatial attention (Treue & Martinez-Trujillo, 1999). In V4, a median response modulation of 26% has been reported for a combination of feature-based and spatial attention (McAdams & Maunsell, 1999). Our psychophysical task requires attention directed to the target feature at a particular location in space. The observed average modulation of 21% is very similar to the size of the effect found in single-unit electrophysiological studies. 

Comparison to psychophysical studies

Previous psychophysical studies investigating attentional influences on sensory tuning have yielded inconsistent results (Neri & Levi, 2006), with some reporting a sharpening (Dosher & Lu, 2000; Lee, Itti, Koch, & Braun, 1999; Yeshurun & Carrasco, 1998) and others showing no effect of attention on the width of perceptual tuning curves (Baldassi & Verghese, 2005; Eckstein et al., 2002; Lu & Dosher, 1998; Murray et al., 2003; Neri, 2004; Talgar, Pelli, & Carrasco, 2004). Among those, the reverse correlation studies (Eckstein et al., 2002; Murray et al., 2003; Neri, 2004) consistently concluded that attention does not alter the shape of perceptual tuning curves, in line with both the results reported here and those of electrophysiological studies. In particular, Murray et al. (2003) have previously investigated the effects of feature-based attention on perceptual tuning curves for direction of motion. The authors computed the perceptual weights allocated to spatially overlapping target and distractor dots defined by contrast polarity in a random dot stimulus, in which a few signal dots moved coherently while all other dots moved in random directions. The subject's task was to attend to the dots of one contrast polarity and indicate their global motion direction. The authors constructed perceptual tuning curves by calculating, separately for attended target and to-be-ignored distractor dots, how much the different random dot directions influenced the observer's judgment. A sinusoid fitted to the perceptual tuning curves for target dots had a ∼14% higher amplitude compared to the tuning curve for the distractor dots, indicating that attention increased the perceptual weight allocated to target compared to distractor dot displacements. The authors also concluded that attention did not affect the selectivity for direction of motion, although sinusoids do not have a free parameter for width. Our study, using a different experimental approach and testing a different type of attention, confirms these conclusions. The size of the attentional effect found by Murray et al. is within the 95% confidence limit of the median modulation in our experiment (lower limit: 13.85%). 

Differences to physiological tuning curves

There are two major differences between physiological tuning curves of direction-selective neurons in area MT and the perceptual tuning curves we obtained. First, the average half width of the perceptual tuning curves is much lower (17.5 deg) than that of direction selective neurons in area MT (50 deg) and V1 (30 deg) (Albright, 1984), and that obtained in two other reverse correlation studies (90 deg, Murray et al., 2003; 36 deg, Neri & Levi, 2008). This difference does not stem from the MRC method per se because neuronal tuning width is not different when obtained with classical methods compared to the MRC method (Borghuis et al., 2003). Rather, the small tuning width is most likely related to the combination of the specific task given to the subjects and the duration of the motion impulses. Our subjects were instructed to respond to a particular target direction, and since the motion impulses contained fully coherent motion, the perception of direction was limited only by the short presentation time of each impulse and its embedding in a rapid series of impulses. Since a difference of 15 deg is much larger than the perceptual discrimination threshold for direction of motion when presented in isolation (De Bruyn & Orban, 1988), we would expect an extremely sharp peak at the target direction under such conditions. To test this prediction, we conducted a control experiment, in which we systematically varied the duration of the motion impulse (N = 5 subjects; 47, 94, 141 ms duration). Here, we found that the width of the resulting direction tuning curves (30.1, 21.2, 15.2 deg) decreased with increasing duration of the motion impulses (ANOVA with factor duration, p = 0.019, Greenhouse–Geyser corrected). This suggests that the psychophysical variant of the MRC paradigm does not target the neuronal direction tuning per se but is related to it by a more complex operation. Noise-limited global motion tasks, such as used by Murray et al. (2003) and Neri and Levi (2008), might measure other aspects of population tuning, which more closely reflect the activity of individual neurons contributing to it. 

A further major difference is the second, smaller peak in the perceptual tuning curves which arises because observers make a significant number of responses to directions of motion opposite to the target direction. The second-order analysis revealed that this unexpected peak reflects an increased probability for responses to the target direction when it was followed by an “anti-target” impulse in the opposite direction. Neurons with a biphasic response profile optimized for the detection of motion changes of 180 deg (Bair & Movshon, 2004; Perge et al., 2005) could be a neuronal mechanism mediating the perceptual facilitation for target followed by “anti-target” impulses. Tentative evidence for perceptual “opposite-same” interactions has been discussed in a study investigating the temporal dynamics of surround suppression (Tadin et al., 2006). A related hypothesis is that observers partially base their responses on the orientation of the motion vector rather than on the specific direction of motion. Such orientation signals would be particularly strong when target and anti-target motion directly follow each other. Since the strength of the “motion streak” (Burr, 1980; Burr & Ross, 2002; Geisler, 1999) critically depends on fast speeds of motion, this interpretation is testable by varying the speed of motion impulses. 

Attentional modulation of temporal tuning curves

Reconstructing perceptual tuning curves by reverse correlation methods has a major advantage compared to other psychophysical methods, such as critical band-masking or adaptation paradigms, namely, the availability of temporal information. So far, however, previous classification image studies on attentional influences have temporally averaged frames to obtain the estimates of the linear kernel (Eckstein et al., 2002; Murray et al., 2003; Neri, 2004). Singular value decomposition of the time–direction tuning surface enabled us to assess the effects of attention on temporal processing of motion information. Consistent with a recent study by Neri and Levi (2008), conducted in parallel and independently from our experiment, we find a high degree of time–direction separability. Specifically, the separability index reported by Neri and Levi is 0.72, which is in very close agreement to the separability index of 0.7 reported here for the full attention condition (our full attention condition is most comparable to the experiment of Neri and Levi, in which attention was not manipulated). As reported in single-unit experiments in areas V1 and MT (Cook & Maunsell, 2004; McAdams & Reid, 2005), we find that attention does not alter the shape or shift the temporal tuning function, even when we use a temporal resolution of 10 ms for binning of response latencies. Consistent with a multiplicative scaling of the temporal tuning function by attention is the fact that we also do not find any speeding of average RTs in the full compared to the poor attention condition. Note however, that small shifts or changes in shape might go undetected because RTs have a broad distribution, and we bin and low-pass filter them in time. In summary, within the temporal resolution of our method, attention enhances the discriminability of motion directions without affecting the speed of reaction times. 

Shape of the temporal tuning curves

The shape of the temporal tuning function in our experiment is different from that reported by Neri and Levi (2008), which peaks between 0 and 100 ms, decreases during the next 100 ms, and increases slightly again at 300 ms. A second peak in temporal tuning, although at an earlier latency (70–90 ms after the first peak), seems also to be present in a study using motion reverse correlation to investigate the dynamics of surround influences (Tadin et al., 2006). Furthermore, the temporal width of the primary peak ranges from 40 to 120 ms in Tadin et al. (2006) and is about 120 ms in Neri and Levi, while our temporal tuning is much broader (∼400 ms). For comparison, temporal tuning width for single neurons in MT is about 40–50 ms (Bair & Movshon, 2004; Perge et al., 2005), at least for fast motion at high contrast and optimal spatial frequency (Bair & Movshon, 2004). It is difficult to directly compare the dynamics of the temporal tuning between the three psychophysical studies because we use RTs for correlation of responses with brief motion impulses while both of the other studies perform a frame-by-frame averaging of a longer noise motion stimulus depending on whether the observer correctly classified the global direction of motion in a non-speeded 2-alternative forced choice design. In electrophysiological subspace reverse correlation studies, the latency of the temporal kernel (e.g., ∼60 ms in MT; Perge et al., 2005) reflects the time from stimulus presentation to the arrival of the information at the recorded neuron, while the width of the kernel reflects the processing time and, to a much smaller degree, temporal variability. Analogously, in our study, the latency of the temporal kernel (∼450 ms) reflects the time from stimulus presentation to the motor response of the subjects, encompassing all stages in between. While it has been shown that temporal tuning of direction selective neurons can be substantially broader with weaker signals (Bair & Movshon, 2004), we think that the perceptual temporal kernel in our experiment is dominated by the spread in time of the responses instead of mostly reflecting processing time. Neri and Levi propose that their bimodal temporal tuning is due to the delayed self-normalization of underlying neuronal filters which have a duration of 30 ms. The absence of such bimodal temporal tuning in our data might be explained by the short duration and rapid succession of our stimuli: sensory processing of a single impulse in the target direction would need to be finished before the self-normalization process applies at ∼300 ms because the subsequent impulses prohibit the accumulation of evidence. In contrast, in Neri and Levi's study, the perceptual decision can be based on the entire duration of a 300-ms motion signal, such that processes at that time scale can be revealed. Alternatively, the distribution of RTs might be too broad to reveal two distinct peaks in the temporal profile measured in our task. 

In summary, we show that attention multiplicatively modulates perceptual tuning curves of human observers for direction of motion across time. Furthermore, we find evidence that the detection of rapid target impulses is facilitated when they are followed by impulses in the opposite motion direction. 

Supplementary Materials

Supplementary File - Supplementary File 

Table A1 - Table A1 

Table A1. p values resulting from the F-tests for the different model comparisons. 

Figure A1 - Figure A1 

Figure A1. Individual tuning curves for all observers in the full (A) and poor (B) attention condition. The data are stacked on top of each other for display purposes. For each observer, the tuning curves are expressed relative to the target direction. The data presented in Figure 2 and 3 are from subject #14. Figure 5A and 5B show data from subject #1 and #12, respectively. 

Acknowledgments

This work was supported by BMBF Grant 01GQ0433 (Bernstein Center for Computational Neuroscience, Göttingen). LB is currently supported the German Academy of Sciences Leopoldina (Fellowship Program, BMBF-LPD 9901/8-165). We thank Peter Neri and two anonymous reviewers for their useful comments. 

Commercial relationships: none. 

Corresponding author: Laura Busse. 

Email: [e-mail]. 

Address: German Primate Center, Kellnerweg 4, 37077 Göttingen, Germany.