Speed encoding in human visual cortex revealed by fMRI adaptation

Abstract

In macaque visual cortex, the conventional view is that image motion is initially detected by direction-sensitive neurons that are tuned in terms of local spatial and temporal frequency (TF), from which speed is encoded later. We used functional magnetic resonance imaging (fMRI) adaptation to seek evidence for speed or TF tuning in human visual cortex. Drifting sine-wave gratings were presented in pairs (S1: adapter, 100% contrast; S2: probe, 15, 40 or 80% contrast). In each trial, either speed or TF was the same for S1 and S2, whereas the other dimension changed. We investigated whether the response was weaker (adapted) for repetitions of the same speed, indicating speed coding, or for repetitions of TF, indicating TF coding. For high-contrast (80%) probes, we observed clear speed coding in MT and MST with similar but weaker trends in several earlier visual areas. For medium- and low contrast probes, our data indicated a trend towards temporal frequency coding in most visual areas studied. In a second experiment, we adjusted stimuli in terms of perceived rather than physical speed and found a trend for speed coding even for low-contrast probes. Our results suggest that speed coding dominates in MT/MST for high contrast stimuli, and possibly also in other visual areas and/or at lower contrasts.

Introduction

Encoding the speed of a moving object is essential for understanding and properly reacting to the events in a visual scene. Whereas the coding of movement direction has been investigated substantially, much less is known about the coding of movement speed. The speed (in degree/second) of a spatio-temporally narrow-band stimulus depends on the ratio between temporal frequency (in Hz) and spatial frequency. Standard models of motion detectors (Adelson & Bergen, 1985; Stone, Watson, & Mulligan, 1990) give rise to motion signals that are divorced from the identity of the object, and a later stage is required for identifying speed. Much evidence suggests that visual stimuli are initially detected by neurons whose responses are separable in space and time. Such neurons selectively respond to a specific range of temporal frequencies and are oriented parallel to the x-axis in spatiotemporal frequency space. In contrast, neurons that are truly tuned to speed respond optimally to a specific speed, irrespective of the underlying spatial frequency. These neurons have receptive fields that are tilted in spatiotemporal frequency space. 

Neurons in the primary visual cortex of cats have a fixed optimum temporal frequency that is invariant with spatial frequency (Holub & Morton-Gibson, 1981; Tolhurst & Movshon, 1975). Similarly, Foster, Gaska, Nagler, and Pollen (1985) found evidence for temporal frequency coding in most neurons in macaque visual cortical areas V1 and V2. These results were consistent with the view that in the early stages of visual processing, neurons behave as detectors of spatial and temporal frequency. They imply that further processing is needed in order to extract speed. 

In contrast to these results for cat and macaque V1 and V2, Perrone and Thiele (2001) reported that most macaque MT cells (61%) are tuned for speed, not temporal frequency. This finding was based on an analysis that classifies neurons according to their orientation in spatiotemporal space. The authors classified neurons as speed-tuned if t-tests revealed a significant difference from 0° orientation. Priebe, Cassanello, and Lisberger (2003) classified neurons as speed tuned if their 95% confidence intervals included a slope value of 1, as frequency-tuned if it included 0, and unclassified otherwise. Based on this analysis, they reported that only 25% of MT neurons show speed tuning. A similar number were classed as temporal-frequency tuned and the remainder had intermediate properties. Stimulation with gratings consisting of two components of different spatial frequencies increased the proportion of MT neurons showing speed encoding, suggesting that speed coding may be more in evidence in MT for natural stimuli than for sine gratings. Comparable results have been reported for marmoset MT (Lui, Bourne, & Rosa, 2007). 

Priebe, Lisberger, and Movshon (2006) revisited the issue of speed tuning in V1. They demonstrated that most simple cells in V1 show separable tuning for temporal and spatial frequency (i.e. temporal frequency coding). On the other hand, only 8/33 complex cells in V1 showed frequency coding, while 9/33 complex cells showed speed coding, reflecting a continuum resembling that found in MT. Perrone (2006) argues that the proportion of speed-tuned cells in V1 may be even higher. These results challenge received wisdom, suggesting that speed coding in MT may be inherited from V1 complex cells, rather than (as widely assumed) being generated in MT. 

Most of our knowledge about the processing of speed is based on macaque physiology. Behavioral data on humans suggest that both speed-tuned mechanisms and separable spatial and temporal frequency mechanisms exist (Reisbeck & Gegenfurtner, 1999). Here we use functional magnetic resonance imaging (fMRI) to examine whether speed tuning or temporal frequency tuning dominates in human MT. We also address the same question in various other visual areas (V1–V4). There are two motives. First, it is important to establish whether macaque and human differ in this respect. Second, fMRI methods allow the assessment of the overall balance of tuning properties in MT as a whole, and the evolution of those properties through earlier visual areas. 

We employed a fast event-related fMRI adaptation or repetition suppression paradigm (Grill-Spector, 2006; Grill-Spector, Henson, & Martin, 2006; Grill-Spector & Malach, 2001; Krekelberg, Boynton, & van Wezel, 2006). In this paradigm, each trial consists of an adaptation (S1) and a probe (S2) stimulus. Previous studies suggest that if both stimuli are encoded by the same (or heavily overlapping) neuron populations, the response to the second will be attenuated by the first. Thus, the observed degree of attenuation gives an index of the extent to which two stimuli are co-processed. This approach has now been used in numerous studies to examine neuronal specificity in the visual system, at both high and low processing levels (Ashida, Lingnau, Wall, & Smith, 2007; Boynton & Finney, 2003; Engel, 2005; Grill-Spector & Malach, 2001; Kourtzi, Erb, Grodd, & Bulthoff, 2003; Larsson, Landy, & Heeger, 2006; Lingnau, Gesierich, & Caramazza, 2009; Wall, Lingnau, Ashida, & Smith, 2008). Typically, S1 and S2 are the same in some trials and differ in one respect in other trials. The adaptation index then indicates the sensitivity of neurons in a given voxel to the parameter changed. In our case, unusually, S1 and S2 were always different. We presented participants with a drifting sine-wave grating (adapter, S1), followed by a second drifting grating (probe, S2; see Figure 1). We designed S1–S2 pairs such that either speed or temporal frequency was the same for S1 and S2, whereas the other dimension changed (Figure 2a). Within each visual area, we investigated whether responses were weaker (adapted) for repetitions of temporal frequency, indicating temporal frequency coding (Figure 2b), or for repetitions of the same speed, indicating speed coding (Figure 2c). We found that speed coding dominates. 

Figure 1

 

Sequence of events in one trial. Each trial consisted of the presentation of two Gabor patches of different spatial frequencies, separated by a blank inter-stimulus interval (ISI). Each was temporally ramped on and off. Trials were separated by an inter-trial-interval (ITI) of 10 s.

Figure 2

 

a: Illustration of all combinations of spatial and temporal frequency used in Experiment 1. Arrows indicate the change in speed or temporal frequency from S1 to S2 within a given trial type (e.g. 1 cpd/4 Hz followed by 2 cpd/4 Hz for a “same TF” trial). Blue indicates same-speed trials; green indicates same-TF trials. See Table 1 for further details. b: Predicted BOLD time course for areas that contain more neurons that are tuned for temporal frequency in comparison to neurons that are tuned for speed. In such areas, we expected that the repetition of the same temporal frequency would lead to a stronger adaptation of the BOLD response than the repetition of the same speed (i.e., the green curve will be below the blue curve). c: Predicted BOLD time course for areas that contain more neurons that are tuned for speed than for temporal frequency. The BOLD response is expected to adapt more strongly for the repetition of the same speed than for the repetition of the same temporal frequency.

Experiment 1: Methods

Participants

The results presented reflect data from seven participants. Data from five others were discarded because of excessive head movements. 

Visual stimulation

All stimuli were drifting sine gratings. The contrast of each was spatially shaped by a static 2D Gaussian, to give a Gabor patch (drifting grating within a stationary envelope). Various grating contrasts were used. Contrast values are specified in terms of Michelson contrast:  

$C_M=\frac{L_{\max }-L_{\min }}{L_{\max }+L_{\min }}$

(1)

where C is contrast, L _max and L _min are the maximum and minimum luminances in the image. 

Gratings subtended 10° of visual angle in diameter before spatial shaping. The standard deviation of the spatial Gaussian envelope was 2.2°. The gratings were presented for either 3 (adapter) or 2.5 (probe) s ( Figure 1). During the initial and final 500 ms of each presentation, contrast was ramped on and off to reduce the effect of high frequency components introduced by abrupt changes. Stimuli were generated with the MATLAB Psychtoolbox-3 (Brainard, 1997) for Mac OSX. 

Stimuli were back-projected onto a screen by a liquid-crystal projector at a frame rate of 60 Hz and a screen resolution of 1024 × 768 pixel. Participants viewed the stimuli binocularly through a mirror above the head coil. The screen was visible as an approximately circular aperture of 30° diameter. The luminance profile was linearized, with a mean luminance of about 2000 cd/m ². 

At the center of the stimulus, a disc (diameter: 0.7°) was presented that served as the fixation target. To (a) ensure that participants fixated at the center of the screen and (b) control the level of attention, participants were required to monitor the color of the disc, which changed at a rate of 2 Hz. At the end of each block of trials, participants reported the number of blue discs they had observed. 

Design

The experiments employed an fMRI adaptation paradigm. Each trial consisted of a pair of stimuli (denoted S1 and S2) presented successively, with an inter-stimulus-interval (ISI) of 1.5 s (see Figure 1). S1 may be thought of as an adapter and S2 as a probe. Within each trial, S1 and S2 differed in spatial frequency by one octave. In addition, they differed either in drift speed (deg/s) or in drift temporal frequency (Hz), but not in both (see Figure 2a and Table 1 for further details); if speed was different, temporal frequency was the same and vice versa. This allowed us to determine whether our stimulus pairs activated neurons that encode motion of sinusoidal gratings in terms of temporal frequency or speed (or could not distinguish the two). If, after counterbalancing for absolute differences in response magnitude, a same-temporal-frequency pair gives a smaller compound response than a same-speed pair, coding must be predominantly in terms of temporal frequency, and vice versa. 

Table 1

 

Construction of pairs of drifting sine-wave gratings (S1: adapter, S2: probe) used for Experiment 1. As an example, combining S1A and S2B in a particular trial results in a change in temporal frequency, whereas speed is the same (4 deg/sec) in both stimuli. Note that these values are within the range of optimal values as derived from fMRI studies in retinotopic areas (Henriksson, Nurminen, Hyvarinen, & Vanni, 2008; Singh, Smith, & Greenlee, 2000).

			S2 (probe)
			A	B
			2 cycles/deg 4 Hz 2 deg/sec	2 cycles/deg 8 Hz 4 deg/sec
S1 (adapter)	A	1 cycle/deg 4 Hz 4 deg/sec	same TF	same speed
	B	4 cycles/deg 8 Hz 2 deg/sec	same speed	same TF

Previous adaptation studies of early visual processing have found that the technique works best when the adapter has a high contrast but the probe has a lower contrast (e.g. Larsson et al., 2006). This parallels a number of psychophysical adaptation phenomena (e.g. Georgeson, 1985). In our application, interpretation is simplest if S1 and S2 have similar contrasts because perceived speed is markedly altered by contrast changes (Thompson, 1982). But we were concerned that we might fail to see adaptation with this method. We therefore used a high-contrast adapter (100%) and probes of various contrasts (15, 40 and 80%). High-contrast probes provide a straightforward interpretation while low-contrast probes provide insurance against finding no effects of adaptation. 

Data acquisition

Data acquisition was done using a Siemens (Erlangen, Germany) Trio 3T scanner and a Siemens 8-channel array head coil. For each participant, scanning was divided between two sessions of about an hour each, conducted on different days. 

Functional images were acquired with a T2*-weighted gradient-recalled echo-planar imaging sequence. We used 28 slices, acquired in ascending interleaved order, slightly tilted to run parallel to the calcarine sulcus, with a TR of 2000 msec, (voxel resolution 3 × 3 × 3 mm, TE: 31 ms, FA: 90°, FOV: 192 × 192 mm). In experiment 1, each participant completed six scans of 175 volumes and 12 scans of 167 volumes. In experiment 2, each participant completed 8 scans of 183 volumes. In addition, a high-resolution T1 weighted anatomical scan (MP-RAGE, Siemens; 1 × 1 × 1 mm; FOV: 256 × 256) was acquired in each of the two scanning sessions. These were used for co-registration of functional data and for segmentation and flattening of the cortex. 

Definition of regions of interest (ROI)

Boundaries between retinotopic areas V1, V2, V3, V3A, V3B, and V4v were identified using standard procedures (Sereno et al., 1995; Sereno, McDonald, & Allman, 1994; Tootell et al., 1997). The polar angle of the rotating checkerboard wedges was 24 degrees and the contrast reversed at a rate of 8 Hz. Each participant completed eight cycles (64 s per cycle). 12 s at the beginning and 12 s at the end of stimulation served to allow the hemodynamic response to reach a stable baseline. Retinotopic maps were created by projecting the temporal phase of the response onto segmented and flattened hemispheres. Borders between visual areas were marked manually at the reversals between phase-map colors. From these regions, a volume-of-interest was defined for each visual area. V3B was selected according to the criteria of Smith, Greenlee, Singh, Kraemer, and Hennig (1998) and thus corresponds to area LO1 (Larsson & Heeger, 2006). V4v was defined as a hemifield representation adjacent to V3v (Wade, Brewer, Rieger, & Wandell, 2002). 

The human MT+ complex was defined by contrasting moving random dots with static dots. The moving dots consisted of a circular patch (diameter 8 deg) of dots that moved alternately inwards and outwards along the radial axes, creating alternating contraction and expansion. The static stimulus had the same image characteristics but no motion. The stimuli were presented unilaterally at an eccentricity of 10 deg to both the right and, in separate scans, left of fixation. A block design was used, moving and static stimuli being presented alternately for 15 s each. MT+ was identified as a 3D volume-of-interest and then divided into two sub-regions. For each hemisphere, MST was defined as the sub-region activated both by ipsilateral and contralateral stimulation, due to the presence of large receptive fields that straddle the midline, whereas MT was defined as the region activated by contralateral stimulation only (Dukelow et al., 2001; Huk, Dougherty, & Heeger, 2002). Since previous research (Dukelow et al., 2001; Huk et al., 2002; Smith, Wall, Williams, & Singh, 2006) has shown that MST is located anteriorly with respect to MT, any MT voxels situated further anterior than the median value of the MST ROI were removed from the MT ROI and discarded (Wall et al., 2008; Wall & Smith, 2008). 

Data analysis

Data analysis, including cortical segmentation and flattening, was performed using BrainVoyager QX 1.7 (BrainInnovation, Maastricht, The Netherlands) in combination with MATLAB 7.5. 

Preprocessing

Prior to further analysis, the first four volumes (8 s) were discarded to preclude T1 saturation effects. Slice timing correction was applied using sinc interpolation. Next, 3D motion correction with trilinear interpolation (six parameters), using the first slice as reference, was applied. To reduce temporal drifts, functional data were temporally high-pass filtered at 3 cycles/scan (approx. 0.01 Hz). No spatial smoothing was applied to avoid blurring the boundary between MT and MST. 

Functional data from the same experimental session were aligned to the first volume of the first functional run of that session. Next, this reference run was aligned to the anatomy obtained in that session. Both functional and anatomical data were transformed into Talairach space (Talairach & Tournaux, 1988), using trilinear interpolation. 

Retinotopic data were additionally temporally low-pass filtered by convolution with a Gaussian of FWHM 8 seconds. 

Cortex reconstruction

Within each hemisphere, the border between gray and white matter was segmented and reconstructed. The resulting surfaces where then smoothed, inflated and cut along the calcarine sulcus. To improve the quality of the cut, retinotopic data were projected onto the surface, such that the cut could be aligned to the border between V1d and V1v. The surface was then flattened. To correct for spatial distortions, distortion correction was applied to reach a linear distortion of <15%. 

Statistical analysis

To identify voxels within individual regions of interest that are sensitive to our experimental manipulations, we convolved each function describing event times with a canonical hemodynamic impulse response function, where ∂ = 2.5 and t = 1.25 (Boynton, Engel, Glover, & Heeger, 1996). The resulting reference time-courses were used to fit the time course of each voxel by means of a general linear model, separately for each participant. Parameters from 3D motion correction (translation and rotation) were included in the model as regressors of no interest, to increase power. Only those voxels that were revealed by the full model, surviving a statistical threshold of a false discovery rate of p < 0.001, were included for computing event-related averages. 

Event-related average time-courses were computed separately for each participant from the activated voxels within the ROI defined in separate localizer runs (see Definition of regions of interest). Computation of the baseline was performed separately for each condition, using a time window of 0–4 seconds, where 0 corresponds to the onset of S1. 

To quantify the change of the measured response after adaptation, relative to the overall response, we computed an adaptation index based on a time window from 8 to 12 s, corresponding to when the response to S2 was prominent (see Figure 3). First of all, within each ROI, we collapsed the time courses across all time points within the time window of 8 to 12 s, separately for each condition and participant. Based on the resulting mean BOLD amplitudes the adaptation index (AI) was then computed as:  

$AI=\frac{\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{T}\mathrm{F}-\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{e}\mathrm{d}}{\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{T}\mathrm{F}+\mathrm{s}\mathrm{a}\mathrm{m}\mathrm{e}\mathrm{S}\mathrm{p}\mathrm{e}\mathrm{e}\mathrm{d}}$

(2)

such that 0 < AI < = 1 if adaptation is stronger for ‘same speed’ trials than for ‘same TF’ trials, indicating speed encoding, whereas −1 < = AI < 0 indicates that adaptation is stronger for ‘same TF’ than for ‘same speed’ (i.e., TF encoding). The adaptation index was computed separately for each ROI and for each stimulus contrast condition. Next, we computed 95% confidence intervals using a nonparametric bootstrapping procedure (Efron & Tibshirani, 1993). In brief, we created 10,000 samples by randomly sampling hemispheres with replacement to estimate the empirical variance of the data using the MATLAB function ‘bootstrp’ contained in the statistics toolbox. The resulting values were used to compute the 95% confidence intervals. A statistically significant adaptation index was assumed if the 95% confidence interval derived from this procedure did not include 0. 

Figure 3

 

Event-related averages from area MT (a–c) and V1 (d–f). Each plot shows the mean response to one trial of a given type, averaged across 14 hemispheres and 42 trial repetitions per participant (sampled from individually defined ROIs as described in the methods section). The contrast of S1 was always 100%, whereas the contrast of S2 was varied (a, d: 15%, b, e: 40%, c, f: 80%). Time zero represents the onset of S1, the two vertical gray bars show the durations of S1 and (when present) S2. Results for three trial types are shown: baseline trials (gray), same-temporal-frequency trials (green), and same-speed trials (blue). Error bars show ±1 standard error. Note that the curves representing the response during baseline trials are identical under all three S2 contrasts (see text for details).

Procedure

Within each block of trials, there were two core trial types: ‘same-temporal-frequency’ and ‘same-speed.’ In addition, there were ‘baseline’ trials where S2 was absent. The contrast of S1 was always 100% (nominal figure; actually about 97%). Each of the two core trial types had three variants, in which the contrast of grating S2 was 15, 40 or 80% (nominal). Thus, there were 7 trial types in all. 

The details of the grating spatial and temporal frequencies used are described in Table 1 and Figure 2a. S1 and S2 always differed in spatial frequency by 1 octave. Because visual neurons typically have a spatial frequency bandwidth of 1–2 octaves (e.g. Foster et al., 1985; Priebe et al., 2006), this difference is sufficient to allow reliable dissociation of speed from temporal frequency while preserving some degree of overlap in the neural populations responding to them. S2 was higher in spatial frequency than S1 in half the trials and lower in the other half. Each S1 and S2 stimulus appeared equally often in same-speed and same-temporal-frequency trials; thus, the effects of the inevitable differences in response magnitude among gratings of varying spatial and temporal frequencies were perfectly balanced. 

Because of the possibility of carry-over of adaptation from one trial to the next, S1 was the same stimulus in every trial during a given run of trials. The use of two different variants of S1 therefore necessitated two different types of scan run. For type-A runs, grating 1A (see Table 1) served as the adapter (S1). This was paired with grating 2A (‘same-temporal-frequency’), grating 2B (‘same-speed’) or no probe stimulus (‘baseline’). For type-B runs, grating 1B was used as S1, paired with grating 2A (‘same-speed’), grating 2B (‘same-temporal-frequency’) or no stimulus (‘baseline’). 

Within each run of trials, each trial type was preceded and followed by all seven trial types (including itself) equally often, to exclude differential effects on the time-courses of the history of a specific trial. Due to this restriction, the minimum block length was 49 trials (7 ²). Since a single trial lasted 17 seconds, a run would last a minimum of 49 × 17 seconds (13.9 minutes) under these conditions. Initial pilot sessions showed that this was too demanding for participants, resulting in occasional head movements even in well-practiced participants. Therefore, each series of 49 trials was divided into three shorter runs of 17, 16 and 16 trials. Within each run, an additional trial was added at the beginning, which was later discarded from the analysis. This ensured that the first analyzed trial had a history that fitted the counterbalancing pattern. Furthermore, 34 seconds were added at the beginning of each run and 10 seconds at the end of each run to allow the hemodynamic response to stabilize. Thus, one scan run lasted either 350 or 333 seconds. 

Each participant completed 18 runs (9 for each type, A or B), with run type ordered AAA, BBB, AAA, etc. in half the participants and BBB, AAA, etc. in the remainder. These were distributed across two experimental sessions on different days. Retinotopic data were either collected at the end of the second session or in a separate session. 

Experiment 1: Results

Sample response time-courses are shown in Figure 3, in terms of percent signal change from baseline. Each plot reflects the average of 42 trials of the same type in each observer and 14 hemispheres from 7 participants (total 588 measurements at each time point). The results are collapsed across spatial frequency variants and across A and B blocks. Each panel shows results for 3 conditions: ‘same-speed’ (blue), ‘same-temporal-frequency’ (green) and ‘baseline’ (gray). 

The top row shows results for MT, for each of the three different S2 (probe) contrasts (A–C). The response to S2 is represented by the difference between each main condition and the baseline condition and is most prominent at around 8–12 s after the onset of S1. At 15% contrast, the response to S2 is relatively small and there is no difference between the green and blue curves. At 40% contrast, there is a slightly larger response to S2, incidentally suggesting that responses in MT are not completely contrast-invariant. The response to same temporal frequency is slightly reduced relative to same speed. At 80% contrast, the response to same-speed is reduced compared to same-temporal-frequency, suggesting the presence of speed-tuned neurons in larger numbers than temporal-frequency-tuned neurons. These results provide evidence of speed tuning in human MT, but this is only evident when the probe contrast is high (i.e. close to S1 contrast of 100%). 

The bottom row of Figure 3 shows the corresponding results for V1. We expected to find strong signs of temporal frequency coding in V1 (green curve attenuated relative to blue curve). Instead, the results are quite similar to MT, with clear signs of temporal frequency tuning at 40% contrast, but speed coding at 80% contrast. 

Figure 4 shows the full set of results for all visual areas studied, in terms of the adaptation index defined in the Methods section. When S2 has 80% contrast, there is a consistent trend towards speed coding (positive adaptation index). This effect is weakest in V1 and V2, where it fails to reach significance (95% confidence intervals include 0), whereas it gradually increases towards MT/MST, where the adaptation index is greatest. At 40% contrast, there is a more variable but consistent trend towards temporal frequency coding (negative adaptation index). This effect is most pronounced in V1, V3 and V4v, whereas it fails to reach significance in MT and MST. At 15% contrast, results are inconclusive, with a tendency for temporal frequency coding that does not reach significance in any area. It should be borne in mind that the S2 response is relatively small at 15% and so the index is more vulnerable to noise than at higher contrasts. In summary, there is a strong effect of contrast and a weak trend towards temporal frequency coding in the early visual areas and towards speed coding in higher visual areas. The latter trend is the pattern that might be expected on physiological grounds, but if so it is expected at all contrasts. Also it is odd to find the direction of the effect reversing, in all visual areas, between 40% and 80% contrast. Priebe et al. (2006) noted that high contrast favors speed coding in both V1 and MT, and this provides a possible explanation of our results. 

Figure 4

 

Adaptation indices for Experiment 1, averaged across hemispheres (n = 14). The index is shown for each visual area at each of three probe contrasts (a: 15%; b: 40%; c: 80%; adapter contrast was always 100%). Positive indices indicate speed coding, negative values indicate temporal frequency coding. The adaptation index is assumed to be significant if the boot-strapped 95% confidence interval does not include zero.

Experiment 2: Methods

General

In Experiment 1, which used a high-contrast adapting pattern, speed coding was demonstrated at 80% probe contrast but not at lower probe contrasts. However, equating speeds across stimuli of different contrasts poses a special problem. It is known from psychophysics that perceived speed typically decreases at lower contrasts and so the adapter and probe in the ‘same-speed’ condition of Experiment 1 may not appear to have the same speed. This raises the question of whether, in this condition, speed should be equated in physical terms, as it was in Experiment 1, or in perceived terms. The data of Stone and Thompson (1992) show, on average, a 30% drop in perceived speed for a one-log-unit reduction in contrast. On this basis, we can expect a reduction in perceived speed (relative to 100% contrast) for S2 in Experiment 1 of 3% for the 80% contrast condition, 12% for the 40% contrast condition and 25% for the 15% contrast condition. When both S1 and S2 had high contrasts (100% and 80% respectively), the effect of contrast on perceived speed is small and the result of Experiment 1 stands, irrespective of whether our data result from perceived or physical speed. But it is possible that the lack of evidence for speed coding at 40% and, particularly, 15% contrast in Experiment 1 might reflect the fact that the gratings of the ‘same-speed’ condition actually had quite different encoded speeds. This is potentially a considerable problem. 

It is difficult to decide whether to equate stimuli in terms of physical or perceived speed in our experiments. The answer turns on whether or not the known perceptual error is reflected in the behavior of the neurons affected by our adaptation stimuli. We do not know the answer to this question and it could be different in different visual areas. It is known that contrast does alter tuning functions in V1 and MT. We therefore conducted Experiment 2 in which the ‘same-speed’ probes were matched for perceived, rather than veridical, speed (see Figure 5 for an illustration). 

Figure 5

 

Overview of stimuli used in Experiment 2. a: Combinations of spatial and physical temporal frequency. b: Combinations of spatial and perceived temporal frequency. Arrows indicate possible pairs of S1 and S2 within a given trial. The color code is the same as in Figure 2. See Table 1 for further details.

The effect of contrast on perceived speed is largely independent of spatial frequency but depends strongly on temporal frequency. It is strongest at low frequencies and is absent or even reversed at high frequencies. The reduction of perceived speed at low contrast is seen only below about 8 Hz, depending to some extent on spatial frequency (Thompson, Brooks, & Hammett, 2006). We based our adjustments to speed on these findings. Since, in Experiment 1, our S1 stimuli were 4 Hz and 8 Hz, we assumed that a 4 Hz grating is reduced in speed by 25% at 15% contrast and we increased physical speed accordingly, but we assumed that the effect of contrast on perceived speed at 8 Hz is negligible and therefore made no adjustment. The stimuli used are shown in Figure 5 and Table 2. Six of the participants who also participated in Experiment 1 were scanned. Only a single S2 contrast (15%, the most critical condition) was tested, and no control trials were included. Scanning was performed in a single session, resulting in 48 repetitions per condition per participant. Apart from these adjustments, the method was the same as for Experiment 1. 

Table 2

 

Construction of pairs of drifting sine-wave gratings for Experiment 2. The two variants (A and B) of S1 were the same as in Experiment 1. For S2, two different types of grating were constructed (S2A, S2B). 4 Hz stimuli are assumed to appear slower at low contrast whereas 8 Hz stimuli are assumed to be unaffected by contrast (see text for details). For each type of S2, physical as well as estimated perceived speed and temporal frequency are shown. Pairs of gratings can be constructed that either have the same perceived speed (e.g., S1A and S2A) or the same temporal frequency (e.g., S1A and S2B).

			S2 (probe, 15% contrast)
			A		B
			Physical	Perceived	Physical	Perceived
			2 cycles/deg 8 Hz 4 deg/sec	2 cycles/deg 8 Hz 4 deg/sec	2 cycles/deg 6 Hz 3 deg/sec	2 cycles/deg 4 Hz 2 deg/sec
S1 (adapter, 100% contrast)	A	1 cycle/deg 4 Hz 4 deg/sec	same speed		same TF
	B	4 cycles/deg 8 Hz 2 deg/sec	same TF		same speed

Experiment 2: Results

The results are expected to follow one of two patterns in each visual area. If a given visual area codes speed in a way that mirrors perception, and does so at all contrasts, then adaptation should occur for ‘same-speed’ with a low-contrast probe even though we did not find any signs of speed coding at low contrast in Experiment 1. If speed is encoded veridically in a given area, despite the perceptual error, then there is no ‘same’ condition, either in terms of speed or temporal frequency, which will lead to lower levels of fMRI adaptation than in Experiment 1 and the lack of a clear distinction between the conditions. 

As can be seen in the illustrative time-courses in Figure 6, our data followed the former pattern. The results point towards speed coding in V1, the same-speed condition giving a reduced S2 response compared to the same-temporal-frequency condition. In MT, there is only a weak trend for speed coding. These results are summarized in Figure 7, showing the adaptation index separately for each ROI. In all areas except MT, there is a trend for speed encoding at 15% contrast. Note, however, that the 95% confidence intervals include 0 in most cases except V3B and V4v. 

Figure 6

 

Event-related averages from Experiment 2 for MT (a) and V1 (b), sampled from individually defined ROIs. Each plot shows the mean response to one trial of a given type, averaged across N = 12 hemispheres and 48 repetitions per participant. The contrast of S1 was always 100% and that of S2 was always 15%. Time zero represents the onset of S1 and the two vertical gray bars show the durations of S1 and S2. Results for two trial types are shown: same-temporal-frequency trials (green) and same-speed trials (blue). Error bars show ±1 standard error.

Figure 7

 

Adaptation indices of all visual areas in Experiment 2. Error bars show the 95% bootstrapped confidence intervals (see text for details).

Discussion

Speed encoding at high contrast

Using an fMRI adaptation paradigm in which an adapter is followed by a probe, we have examined tuning for speed and temporal frequency in various visual areas in human occipital cortex. Our data suggest that speed encoding dominates over temporal frequency coding in all visual areas studied (though only marginally in some early areas), including MT/MST, at least for high-contrast stimuli. It is not possible to assert that speed coding is ubiquitous at the neuronal level. Indeed, it is very possible that a continuum of encoding strategies is employed, as in macaques, and our results do not preclude the existence of sub-populations (e.g. simple cells in V1; Priebe et al., 2006) that have strictly separable spatial frequency and temporal frequency sensitivities. Indeed, the dominance of speed over frequency was weak in V1–V3 (Figure 4c) and did not reach significance in V1, V2 and V3B, suggesting mixed populations. These weak effects in early visual areas are compatible with data by Orban, Kennedy, and Bullier (1986) who observed that only about 10% of all examined cells in central V1 and V2 were tuned to a particular preferred speed, whereas the majority of cells were classified as low-pass speed tuned. Yet, the mean tuning of the population that contributes to the BOLD response is closer to speed tuning than to temporal frequency tuning, at least in MT/MST and possibly in earlier areas. It should be borne in mind that our choice of stimulus parameters may make some areas more responsive than others and therefore more likely to reveal selectivity. 

Site of adaptation

Since our results at high probe contrast are similar in all areas, with only a weak progressive increase in adaptation across areas in our ‘same-speed’ condition ( Figure 4c), it is possible that adaptation arises first in V1 and that both adaptation and speed sensitivity are largely inherited by other areas, including MT, from V1, with only modest refinement. fMRI adaptation alone may not allow determination of the precise site of adaptation (Bartels, Logothetis, & Moutoussis, 2008; Sawamura, Orban, & Vogels, 2006). It cannot be ruled out that some of the effects we observed (e.g. speed coding in V4) actually originated further downstream. However, the idea that our adaptation effects are all largely inherited from V1 is made plausible by the fact that the macaque V1 cells that project to MT resemble MT cells, in terms of receptive field properties, much more closely than V1 cells as a whole (Movshon & Newsome, 1996), suggesting that the reorganization that occurs in MT may be relatively limited. On the other hand, there is evidence that properties like direction selectivity may be computed autonomously in MT (Thiele, Distler, Korbmacher, & Hoffmann, 2004). In line with this view, McKeefry, Burton, Vakrou, Barrett, and Morland (2008) demonstrated that repetitive transcranial magnetic stimulation (rTMS) applied to MT and V3A impairs performance in speed perception, whereas the application of rTMS to V1 produces no deficit (but see Matthews, Luber, Qian, & Lisanby, 2001). Furthermore, given the existence of fast feedback connections from MT to V1 (Hupe et al., 2001), the possibility cannot be excluded that the effects we observe in early visual cortex are in fact generated in MT. 

Effects of attention

It has been shown many times in many contexts that the magnitude of the BOLD signal elicited by a given stimulus can be increased by attention to that stimulus. In the case of the fMRI adaptation paradigm, which relies heavily on documenting small differences in response magnitudes between different trial types, there is a risk that different stimuli will attract attention to different degrees so that effects of attentional modulation are confounded with effects of adaptation. In particular, in the standard adaptation paradigm used by many investigators, the adapter and probe are sometimes the same and sometimes different and it is possible that the different probe may attract attention more strongly because it is more novel. An attention-related increase in the ‘different’ condition could then be mistaken for suppression by adaptation in the ‘same’ condition. Two factors suggest that our results cannot be explained in this way. First, we used a demanding task at fixation to divert attention away from the grating stimuli. This approach has been used successfully in a number of other fMRI adaptation studies (Ashida et al., 2007; Larsson et al., 2006; Wall et al., 2008), though it should be noted that drawing attention away from the stimulus may reduce the size of observed effects (Murray & Wojciulik, 2004). Second, our paradigm is not susceptible to novelty effects because we did not use a ‘same’ condition. The adapter and probe were always different, by one octave of spatial frequency, and the direction of that difference was perfectly balanced. 

Effect of contrast on perceived speed

In fMRI adaptation studies, it is desirable to use a high-contrast adapter (Larsson et al., 2006). Only when the probe also has a high contrast can the issue of perceived speed be avoided. In this case, perceived and veridical speed can be assumed to vary in the same way between adapter and probe. But when there is a large contrast change between adapter and probe, that assumption is violated and it becomes difficult to know whether it is the difference in veridical or perceived speed between adapter and probe that is important. 

If we ignore perceptual data and assume that speed-tuning functions of visual neurons reflect physical speed, then we should take the results of Experiment 1 with speed defined in physical terms at face value and we are then led to the conclusion that speed coding dominates only at high contrasts. For a 40% probe contrast, temporal frequency coding dominates ( Figure 4b). This would be in line with a psychophysical study suggesting that speed-tuned mechanisms are only in evidence at high contrast (Reisbeck & Gegenfurtner, 1999). It is also in line with the neurophysiological work of Priebe et al. (2006) in which single-unit responses were shown to shift towards temporal frequency coding at low contrast. But there is an inherent contradiction in interpreting our data in these terms. If speed encoding gives way to temporal frequency encoding at medium and low contrasts, then the assumption that visual neurons encode speed in a contrast-invariant fashion cannot be correct. Moreover, there is other physiological evidence that speed coding is not contrast-invariant, either in V1 (Livingstone & Conway, 2007) or MT (Krekelberg, van Wezel, & Albright, 2006; Pack, Hunter, & Born, 2005), but see also Priebe and Lisberger (2004). Thus, to assume that contrast does not affect neural speed selectivity, and to base our conclusions on the results obtained with speed defined in physical terms, is problematic. 

Alternatively, we could assume that the pronounced failure of contrast invariance in speed perception (Thompson, 1982) reflects a similar failure in early visual neurons. In this case, the results of Experiment 1 for low-contrast probes have limited validity and we should base our conclusions on the results of Experiment 2, in which compensation for effects of contrast on perceived speed was applied. This would lead us to the conclusion that speed coding dominates over frequency coding across a range of different contrasts, not just at high contrasts. Of course, this conclusion relies on the accuracy of the compensation we applied. This cannot be guaranteed, since relevant psychophysical estimates (Stone & Thompson, 1992; Thompson, 1982) vary considerably across studies and across observers. But our finding that with the compensation applied in experiment 2, results at low contrast shifted towards speed coding, consistent with results obtained with high-contrast probes, suggests that the underlying neural populations are dominated by speed estimates that are not invariant with stimulus contrast. 

Although our Experiment 2 provides an interesting result, it clearly does not fully resolve the complex issue of what happens to speed and frequency coding at medium and low contrast. In fact there are a couple of factors that complicate the interpretation of the low and medium contrast conditions, as will be discussed in the following section. 

Effect of contrast on perceived temporal frequency

The interpretation of the effects of manipulating contrast are further complicated by the fact that perceived temporal frequency is not affected in the same way as perceived speed. Thompson and Stone (1997) measured the perceived temporal frequency of a 2 c/deg, 4 Hz flickering (counterphasing) grating and found that it increases at low contrast, whereas the perceived speed of the equivalent drifting grating decreases. From this, it might be argued that we might equally well have adjusted our stimuli in the opposite direction in Experiment 2. It should be noted, incidentally, that the downward shift in optimum speed seen at low contrast in V1 and MT neurons (Livingstone & Conway, 2007) is in the wrong direction to explain changes in perceived speed on a labeled line model since these models predict that perceived speed should increase rather than decrease at lower contrast (Krekelberg, van Wezel et al., 2006). However, this downward shift is in the right direction to explain changes in perceived temporal frequency (Thompson & Stone, 1997). This might be construed as evidence for temporal frequency coding (these studies used dots or bars, not sine gratings, and so do not distinguish the two). 

Effect of contrast on speed tuning functions

Speed tuning functions have been reported to shift towards lower speeds with decreasing contrast (Krekelberg, van Wezel et al., 2006; Livingstone & Conway, 2007). This could mean that if we adapt with a high contrast and then test with medium or low contrast, we are actually testing a somewhat different population of neurons, in which case we can interpret only the data from the 80% condition with confidence. It should be noted, however, that the reported shifts of tuning functions were largest for cells with high preferred speed at high contrast (Krekelberg, van Wezel et al., 2006, Figure 5), and these speeds where clearly higher than those used in the current study. 

Differences in contrast response across areas

Contrast sensitivity in V1 and V2 is substantially lower than contrast sensitivity in V3 and MT (e.g. Tootell et al., 1995). It is likely that areas (such as MT) that are driven at peak level for all our test contrasts show different adaptation effects in comparison to areas that give unsaturated responses at some contrasts. The heterogeneity observed for the low contrast probe stimuli might reflect such differences. 

Other complicating factors

Several other factors complicate interpretation of data such as ours. 

First, it is possible that speed and TF selective channels have different contrast sensitivities and thus are differentially affected by adaptation. According to the weighted intersection mechanism (WIM) model (Perrone, 2006; Perrone & Thiele, 2002), speed tuning is implemented by combining a temporally biphasic and a temporally monophasic channel. Physiologically, these two channels might be implemented as magno- (fast and biphasic) and parvocellular (slower and monophasic) neurons. Parvocellular neurons have a lower contrast sensitivity than magnocellular neurons (Shapley, Kaplan, & Soodak, 1981). If the assumption is correct that speed tuned neurons receive input both from parvo- and magnocellular neurons, the weighting of these two inputs may change if contrast is varied. 

Second, Hietanen, Crowder, Price, and Ibbotson (2007) showed that the majority of neurons in cat V1 shift to lower speeds following adaptation to motion, irrespective of the adapting speed. When adapted at a speed below their preferred speed, they show no shift in preferred speed. The authors demonstrated adaptation both for speed and contrast, but these seemed to be uncorrelated. However, these observed shifts in preferred speed following adaptation were observed at speeds that were clearly higher than those used in the current study. 

Third, it is possible that inhibitory interneurons have different contrast thresholds than excitatory interneurons. As a result, the contribution of excitatory and inhibitory interneurons may change if contrast is varied, and this may in turn affect adaptation. 

In view of the large number of uncertain factors that may influence the results, it is difficult to draw firm conclusions about effects of contrast on the balance of speed and TF tuning at this juncture. Thus, our definitive result concerns speed tuning at high contrast in MT/MST and the remaining results have value mainly as data that will constrain future models. A direct way to address the outstanding uncertainty would be to conduct an experiment in which the adapter, as well as the probe, had a low contrast. However, it is questionable whether a low-contrast adapter would generate adaptation that could even be detected, let alone measured and compared across conditions with any precision. 

Conclusions

In summary, we suggest with some confidence that speed coding dominates in human MT and MST at high contrasts. As has been outlined above, the effects at medium and low probe contrasts need to be interpreted with care. We therefore suggest much more tentatively that speed may predominate at all contrasts. These conclusions also apply to all other visual areas studied, but with differing degrees of confidence. 

Acknowledgments

This work was supported by a grant to ATS from the European Commission. 

Commercial relationships: none. 

Corresponding author: Andy Smith. 

Email: [e-mail]. 

Address: Department of Psychology, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK.