Sensitivity of human visual cortical areas to the stereoscopic depth of a moving stimulus

Abstract

Many fMRI studies have documented motion-sensitivity in the human occipital cortex and several have examined sensitivity to binocular disparity. However, selectivity to the stereo-defined depth of a moving luminance-defined stimulus has not been examined with fMRI. We used an fMRI adaptation paradigm to examine such selectivity. On each trial of an event-related design, two brief rotating dot patterns were presented sequentially. These had either the same or opposite directions of motion and were presented in either the same or different depth planes (±1 deg disparity). There were no monocular cues to depth. Attention was controlled by a demanding task at fixation; in addition, control trials in which stimulus salience was manipulated confirmed that there was no modulation by attention. In MT and MST, the compound response was smaller (adapted) when the two had the same depth than when they were different. This suggests the presence of separate neural populations sensitive to near and far motion, consistent with physiological results. Selectivity for motion direction was also seen. The magnitude of the depth effect was similar to that of direction in MT/MST, suggesting equally pronounced tuning. Visual areas V1–V4 also showed strong selectivity for near and far depth planes, whereas direction sensitivity was weaker overall and was measurable only in V3 and beyond.

Introduction

In most visual areas of the primate cerebral cortex, neurons are found that are sensitive to stereoscopic depth (for reviews, see Gonzalez & Perez, 1998; Parker, 2007). When tested with dynamic random-dot stereograms to eliminate monocular cues, about half of all V1 cells have this property and the proportion rises to about 65% in V2 and 80% in V3 and V3A (Poggio, Gonzalez, & Krause, 1988). Even higher proportions of disparity-sensitive cells are seen in MT and MST. DeAngelis and Uka (2003) reported that 93% of MT cells are tuned for disparity. Tuning is broader than in other areas, with “near” and “far” cells dominating. Similarly in MST, over 90% are disparity sensitive, and almost all are selectively responsive to either near or far depth planes rather than to disparities near zero (Roy, Komatsu, & Wurtz, 1992). 

Most macaque visual areas also contain neurons that are sensitive to the direction of motion in the fronto-parallel plane of an appropriate stimulus. Sensitivity to motion in MT and MST has been very thoroughly documented. Virtually all MT neurons are direction sensitive (Albright, 1984; Maunsell & Van Essen, 1983b), and about 80% are strongly so (respond >5 times more strongly to the preferred than the null direction). MST shows equally strong selectivity and additionally contains some neurons (mostly in MSTd) that are selectively responsive to specific optic flow components such as expansion or rotation (Duffy & Wurtz, 1991; Tanaka & Saito, 1989). Smaller numbers of direction-sensitive cells are seen in most of the other early visual areas. In macaque V1, varying degrees of direction selectivity are seen, many cells having only a weak bias for one direction over the opposite, while some are strongly biased or even give no response at all to motion in the non-preferred direction. Estimates of the proportion of V1 neurons showing a response in the preferred direction that is at least twice that in the non-preferred direction vary in the range 20% to 40% (De Valois, Yund, & Hepler, 1982; Foster, Gaska, Nagler, & Pollen, 1985; Schiller, Finlay, & Volman, 1976). In V2, around 30% of neurons are direction sensitive, using the same 2:1 criterion, and 15–20% are strongly directional (defined as at least 3:1) (Gegenfurtner, Kiper, & Fenstemaker, 1996; Levitt, Kiper, & Movshon, 1994). This suggests a similar level of direction sensitivity in V1 and V2, but one study that compared the two areas directly (Foster et al., 1985) reported substantially stronger direction sensitivity in V2 than V1 (38% in V2 as compared to 20% in V1). In V3, direction selectivity has been reported to be stronger than in V2 (Burkhalter, Felleman, Newsome, & Van Essen, 1986; Gegenfurtner, Kiper, & Levitt, 1997), the latter claiming that 57% of cells give at least twice the null-direction response in the preferred direction. Thus, it seems that direction sensitivity probably increases from V1 to V2 and certainly from V2 to V3. In macaque V3A, however, direction specificity is less common than in V3 (Gaska, Jacobson, & Pollen, 1988) (24% > 2:1). Likewise, in V4, which is regarded as a ventral stream area involved in pattern and color rather than motion, only some 10–15% of cells are direction sensitive (Desimone & Schein, 1987). 

In humans, many studies have documented sensitivity to motion in the early visual areas using fMRI. The human V5/MT complex (Tootell et al., 1995; Wilms et al., 2005; Zeki et al., 1991) in particular exhibits robust responses to a wide variety of motion-related stimuli (Chawla, Phillips, Buechel, Edwards, & Friston, 1998; Smith, Greenlee, Singh, Kraemer, & Hennig, 1998; Smith, Wall, Williams, & Singh, 2006; Sunaert, VanHecke, Marchal, & Orban, 1999; Tootell et al., 1995). But these studies compared sensitivity to a moving stimulus with sensitivity to a static or a flickering control. They suggest the presence of neurons that respond well to moving stimuli, which is not the same as demonstrating the existence of different populations of neurons sensitive to different directions. A few human fMRI studies have studied direction sensitivity. Using an adaptation paradigm, Huk, Ress, and Heeger (2001) showed that direction selectivity is least in human V1 and V2 and strongest in MT+. V3, V3A, and V4 were intermediate. In contrast to macaque, V3A was more direction sensitive than V3, mirroring earlier observations that it has greater differential sensitivity to moving relative to stationary stimuli (Tootell et al., 1997). Surprisingly, V4 showed stronger direction-specific adaptation than V1 and V2. A similar fMRI study has been performed in macaques (Tolias, Smirnakis, Augath, Trinath, & Logothetis, 2001). These authors found direction specificity in all visual areas (V1–V4 and MT). Direction-specific adaptation was greatest in MT and, surprisingly, V4. They speculate that some V4 cells may receive input from directional cells with different sensitivities, so when one direction is adapted, sensitivity to the other is greater and directionality appears. Other human fMRI studies that have used adaptation to examine direction sensitivity include Huk & Heeger (2002), who again found direction selectivity in all areas but strongest effects in MT+ and V3A, and Nishida (2003) and Ashida et al. (2007), who found direction sensitivity in both V1 and MT+ but did not study other areas. 

Sensitivity to stereoscopic depth has been demonstrated in a number of human fMRI studies. Backus, Fleet, Parker, and Heeger (2001) demonstrated that the response to two transparent fronto-parallel surfaces formed by dynamic random-dot stereograms is larger than that to a single plane with the same total number of dots. This difference was seen in visual areas V1, V2, and V3 with equal strength. V3A was particularly sensitive, and MT gave no measurable difference, although depth sensitivity was sometimes seen in supplementary experiments. Other studies (Merboldt, Baudewig, Treue, & Frahm, 2002; Neri, Bridge, & Heeger, 2004; Nishida et al., 2001; Tsao et al., 2003) have also found sensitivity in all areas but highlighted V3A, along with neighboring areas such as V7, LOC, and posterior parietal areas, as a possible focus of stereoscopic depth processing. These studies used a variety of stimuli, including spatial and temporal depth transients, leaving unanswered the question of whether it is stereoscopic depth per se or depth edges and changes that activate these areas. A recent study of sensitivity to depth-defined shapes (Chandrasekaran, Canon, Dahmen, Kourtzi, & Welchman, 2007) found little or no such sensitivity in early visual areas, favoring an explanation in terms of depth rather than shape. However, strong sensitivity to shape was found in MT, MST, and LOC. 

All these studies employed static or dynamic depth planes or patterns; none concerned sensitivity to the stereoscopic depth of a moving pattern. The present study is concerned specifically with moving stimuli and particularly with the motion-sensitive MT complex or MT+. It is likely that this region is actually a set of inter-related motion-sensitive areas, resembling that found in other primates. We divide it into two parts, MT and MST, using the criterion of sensitivity to ipsilateral stimulation (Dukelow et al., 2001). We examine sensitivity to direction of motion as well as sensitivity to the depth of a moving stimulus, and we do so in early retinotopic areas as well as in MT/MST. We use a repetition suppression (or adaptation) technique to disentangle responses from functionally distinct neurons within the same voxel (Grill-Spector & Malach, 2001). Two stimuli differing in only one respect (in the present case, stereoscopic depth) are presented successively. If a larger response is elicited in a given cortical area by two different stimuli than by repetition of the same pattern, that area must contain neurons that are sensitive to the parameter that was changed. It is likely that the observed reduction of the BOLD response when two identical stimuli are presented is a reflection of underlying neural adaptation. Adaptation has been shown to be a powerful method for revealing the stimulus response properties of neurons within a variety of different cortical areas (e.g., Ashida, Lingnau, Wall, & Smith, 2007; Dehaene-Lambertz et al., 2006; Kourtzi, Erb, Grodd, & Bülthoff, 2003). 

Methods

Participants

Five participants took part in the studies reported here, including both authors. All had normal or corrected-to-normal vision including good stereoacuity. In selecting the participants, some volunteers were rejected (not scanned) because of poor ability to see the stereo-defined motion stimuli used in the experiment. The experiment was conducted in accordance with the Declaration of Helsinki, approved by a local ethics committee at Royal Holloway, University of London, and written informed consent was obtained. Standard MRI screening procedures were followed for all participants, and volunteers were paid for their time. 

Data acquisition

MRI images were obtained with a 3-T Siemens Magnetom Trio scanner and an 8-channel array head coil. For each participant, scanning took place over at least two sessions on different days. Each session commenced with an anatomical (3D, T1-weighted) scan. In the first scan, a sequence (MDEFT; Deichmann, Schwarzbauer, & Turner, 2004) that gives exceptional contrast between white matter and gray matter was used to facilitate segmentation and flattening of the gray matter. In the second session (and in the first if an MDEFT scan had been acquired on a previous occasion), an MP-RAGE scan (Siemens, Germany) was substituted for speed, in either case, a resolution of 160 × 256 × 256 was used to give 1-mm isotropic voxels. This was followed in each scan by six runs of functional data acquisition with a gradient echo, echoplanar sequence (TR = 2 s, 28 contiguous axial slices, interleaved acquisition order, 3 mm isotropic voxels, in-plane resolution of 64 × 64 voxels, FOV = 192 × 192 mm, flip angle = 80°, TE = 31 ms, bandwidth = 1396 Hz/pixel). Each functional scan run lasted 5 min 58 s. 

Stimuli

All stimuli were back-projected onto a screen mounted in the rear of the scanner bore by a computer-controlled LCD projector. Images consisted of dynamic stereo-pairs. The images for the two eyes were projected side-by-side. Participants viewed the stimuli via an in-house binocular viewing device that restricted each eye's field of view to one member of the pair and enabled the two eye images to be fused with a natural vergence angle to yield stereoscopic depth. The field of view was 17 degrees. 

Each stimulus consisted of circular white dots (diameter 0.5 deg) on a dark background. The luminance of the background was 4 cd/m ² and that of the dots was 609 cd/m ², except where otherwise stated. A central cluster and an outer ring of dots (see Figure 1A) were static, had zero disparity, and constituted the fixation plane. A central, colored fixation spot was present throughout the entirety of each scanning run. The moving stereoscopic stimuli filled the annular region between the two zones of the fixation plane ( Figure 1B). The diameters of the inner and outer edges of this annulus were 3 deg and 10 deg, respectively. Dots in this region had a non-zero disparity. All such dots had the same disparity, which was either +60 arcmin (crossed disparity) or −60 arcmin (uncrossed disparity). This gave the appearance of a ring of dots positioned either in front of or behind the fixation plane. The dots moved so as to give the appearance of rotation of the annulus, either clockwise or counterclockwise. Dots moved at a speed of 20 deg/sec with unlimited lifetime. 

Figure 1

 

Illustration of the stimuli. (A) A stereopair with zero disparity showing the dots that form the fixation plane and the central fixation point. (B) The same stereopair with an annular stimulus present (uncrossed disparity if free-fusing by convergence of the eyes). This represents one frame of an animation in which the stimulus ring rotates while the dots forming the fixation plane are static.

Procedure

The main experiment involved fMRI adaptation. Stimuli were presented in pairs, one trial consisting of an adapting stimulus (S1) and a test stimulus (S2). S1 was presented for 3 s, followed by a 1-s inter-stimulus interval (ISI) and finally by S2 for 1 s. This timing was based on previous studies of fMRI adaptation in the visual cortex (Ashida et al., 2007; Larsson, Landy, & Heeger, 2006). The ISI provides a degree of separation of the responses to S1 and S2 (see Results) without losing the effects of adaptation. A fixed inter-trial interval (ITI) of 8 s then followed before the start of the next trial. There were five trial types:

•  
  In “same” trials, both the direction of rotation and the depth plane of the annulus were the same for S1 and S2.
•  
  In “different disparity” trials, the direction of motion was the same for S1 and S2 but the depth plane differed. If S1 was “near” (60 min disparity) then S2 was “far” (−60 min disparity) and vice versa.
•  
  In “different direction” trials, the stereoscopic depth of the annulus was the same for S1 and S2 (either near or far) but the direction of rotation differed. If S1 rotated clockwise then S2 rotated anticlockwise and vice versa.
•  
  In “attention control” trials, both the direction of rotation and the depth plane of the annulus were the same for S1 and S2. However, the luminance (and hence contrast) of the dots changed. The luminance of the dots in S1 was 609 cd/m ², as in all other trials, being 75% of the maximum available luminance. But that of S2 either increased to 100% or decreased to 50% of maximum, with equal probability.
•  
  In “baseline” trials, S2 was absent entirely, to facilitate calculation of an adaptation index.

The rationale of the design is that when the same flow pattern is presented twice, the response to the second presentation will be attenuated by the first because of adaptation or “repetition suppression”. When two different stimuli are presented, the response to the second will be attenuated only to the extent that it activates the same neurons as the first. Thus, if there are separate populations encoding different depth planes in a given visual area, “different disparity” trials will result in larger responses than “same disparity” trials because S2 activates unadapted neurons. The “attention control” trials provide a change in stimulus between S1 and S2 that is as likely to attract attention as a change in depth or direction but is not relevant to the question of interest (see Results). 

The disparity value (±60 min) was chosen to target the “near” and “far” cells that are prevalent in MT and MST (DeAngelis et al., 1998). “Near” cells respond well over a wide range of large, crossed disparities but not well to near-zero crossed disparities or to uncrossed disparities. Similarly, “far” cells favor large uncrossed disparities. It should be noted that “tuned” neurons also exist; these respond optimally only to a narrow range of disparities. Tuned neurons in MT usually prefer zero disparity (DeAngelis et al., 1998) while neurons tuned to small non-zero disparities are common in V1 (e.g., Poggio, 1995; Cumming, 2001). Our stimuli are not expected to drive such neurons effectively. 

With an ITI of 8 s, the BOLD response to one trial does not fully decay before the next trial starts. In fMRI studies, it is common to deal with the problem of overlapping responses by using a variable ITI (e.g., Dale & Buckner, 1997). This can give efficient separation of signals for the purpose of statistical detection, but for estimating timecourses it relies on cross-trial contamination summing to zero, which may require a large number of trials. We preferred to equate such contamination across trials rather than attempting to eliminate it. To achieve this, the duration of the ITI was invariant. Several other authors have successfully used this approach in adaptation studies (e.g., Fang, Murray, Kersten, & He, 2005; Larsson et al., 2006). The order of presentation of trials within a given run was determined such that each trial type was preceded equally often by each of the five trial types, including itself. In each scan run, there were 25 trials (5 of each type) that were included in the analysis. The first of these was preceded by an additional trial that was not analyzed, so that the first analyzed trial conformed to the balanced pattern of preceding trials, giving 26 trials in total per run. Twelve scan runs were completed for each participant (over two scanning sessions), with short breaks between them. The trial order was different for each scan run but always conformed to the same counterbalancing regimen. The same set of 12 trial sequences was used for each participant; however, half the participants completed the scan runs in reverse order to eliminate any possible order effects. Throughout each run, S1 was the same in every trial. This is a precaution in case adaptation carries over, to some extent, from one trial to the next, as it ensures that there is adaptation to only one stimulus. For six of the 12 scanning runs, the adapting stimulus (S1) had crossed disparity, and for the remaining six, it had uncrossed disparity. Thus, “different disparity” trials could be either near followed by far or far followed by near, and the responses to the two versions were averaged. This balances any differences in the absolute magnitudes of the responses to the two stimuli. Similarly, the direction of rotation of S1 was counterbalanced, being clockwise in half the runs and counterclockwise in the rest. 

Attention was controlled by means of a demanding task at fixation. During scanning, the fixation point changed color at a rate of 2 Hz, the color being randomly chosen from six very different colors. Participants were instructed to count how often the fixation point turned blue and to report the total verbally to the experimenter at the end of each run. 

Scanning runs to define regions of interest (ROIs) were performed in a separate scan session. In some cases, this had been done as part of a previous project; in others, a third scan session was conducted for the purpose. MT and MST were defined by the use of an ipsilateral stimulus based on Huk, Dougherty, and Heeger (2002) and previously used in our lab (Smith et al., 2006). A circular patch of dots (8 degrees in diameter) was presented with its center placed 10 degrees to the left or right of fixation. Blocks of 15 s in which the dots were static were alternated with blocks in which the dots moved alternately inward and outward along the radial axes (thus creating alternating contraction and expansion). Sixteen blocks (8 static and 8 moving) were presented in each scanning run; one scanning run was completed with the stimulus on the left and another with it on the right. With this procedure, MT and MST can be differentiated in terms of the absence or presence, respectively, of ipsilateral drive (see Results). 

Retinotopic areas V1–V4 were identified by a standard retinotopic mapping procedure (Sereno et al., 1995), using an 8-Hz counterphasing checkerboard “wedge” stimulus (a 24-deg sector) of radius 12 degrees. Check size was scaled by eccentricity in approximate accordance with the cortical magnification factor. The wedge rotated clockwise at a rate of 64 s/cycle, and eight cycles were presented. 

Data analysis

All data were pre-processed and analyzed using BrainVoyager QX (version 1.4; Brain Innovation B.V., The Netherlands). Functional data were corrected for head motion and slice timing and were filtered with a temporal high-pass filter of 0.014 Hz. No spatial smoothing was applied to avoid blurring the boundary between MT and MST. The first anatomical scan for each participant was used as a reference to which all the functional images were co-registered. Event-related averages were then computed for each trial type and mean time courses were extracted for each ROI showing the mean response over one whole trial period, including the 8-s post-trial interval. 

MT/MST ROIs were defined as clusters resulting from the analysis of the relevant mapping data. Model regressors were defined based on the alternating moving and static blocks, and participant head-movement parameters derived from 3D motion correction were included as regressors. Since contralateral stimuli drive the entire MT+ complex, and ipsilateral stimuli drive only MST, the ROIs obtained for the two stimuli overlapped substantially. MST was defined as all contiguous voxels that were significantly active during ipsilateral motion stimulation. MT was defined as all contiguous active voxels that were active during contralateral but not ipsilateral stimulation, with one proviso. Since previous research (Dukelow et al., 2001; Huk et al., 2002; Smith et al., 2006) has shown that the center of MST is located anteriorly with respect to the center of MT, any MT voxels situated further anterior than the median value of the MST ROI on the horizontal (axial) plane were removed from the MT ROI. 

Retinotopic data were analyzed by fitting a model to the time course obtained with the rotating wedge stimulus. This consisted of a rectangular wave of appropriate duty cycle reflecting when the stimulus entered a particular portion of the visual field, convolved with the hemodynamic impulse response function. The phase of the fitted response was taken as an index of visual field location in terms of polar angle. Reversals of the direction of phase change across the cortical surface were taken as boundaries of visual areas (Sereno et al., 1995). ROIs (visual areas V1–V4) were drawn by eye based on these boundaries viewed on a flattened version of each participant's reference anatomy, transformed into Talairach space. The functional data from the main experiment were also transformed into Talairach space in order that time-course data could be extracted from the V1–V4 ROIs. 

Results

The purpose of the study was to test for the presence of neurons, primarily in MT and MST but also in other early visual areas, that are selectively responsive to the stereoscopic depth plane in which moving stimuli are presented, using the adaptation procedure described in the Methods section. For comparison, sensitivity to direction of motion was also examined. 

MT and MST

Regions-of-interest for MT and MST were successfully defined for both hemispheres in all participants. One example is shown in Figure 2. The procedure was identical to that used in our previous work (Smith et al., 2006) and other examples of the appearance and location of MT and MST ROIs can be seen in this paper. 

Figure 2

 

Regions of interest from one participant. Left: flatmap from the left hemisphere showing areas V1–V4. Right: brain sections showing the locations of MT and MST.

Figure 3 shows the mean time courses extracted from the MT (top) and MST (bottom) ROIs. Each trace represents the response to one trial of a given type, averaged over 60 repetitions per participant and across 10 hemispheres from 5 participants. Time zero represents the onset of S1 and shaded bars indicate the presence of S1 and S2. Prior to the onset of S1, responses are falling from the previous trial. About 2 s after the onset of S1, the response to S1 begins to build. Until the onset of S2, the five trial types yield extremely similar time courses since S1 is always the same and the influence of the preceding trial is counterbalanced. Thereafter, they diverge. The response in the “baseline” condition (in which S2 was absent) peaks about 6–8 s after the onset of S1 and then declines. The other four conditions all show extended responses, peaking about 2 s later, as expected, then falling. About 15 s after the onset of S1, the time courses start to rise again. This is true for all trial types and reflects the fact that (time = 13 s) for one trial is the same as (time = 0 s) for the following trial. 

Figure 3

 

Mean time courses derived from MT (top) and MST (bottom) for each trial type in the experiment. Time courses are averaged across 10 hemispheres. The vertical gray bars indicate the timings of the stimuli. Error bars show ±1 SEM.

The results for MT and MST are extremely similar in all respects. In both areas, the largest responses were obtained in the “different direction” and “different disparity” conditions. The response magnitude is reduced in the “same” and “attention control” trials. The “adaptation rebound” or increased response seen when either direction or disparity is changed indicates the presence of neurons in MT and MST that are sensitive to stereoscopic depth, as well as the expected presence of neurons sensitive to direction of motion. The fact that the increase in response compared to the “same” condition is similar in the two cases suggests that sensitivities to direction and to disparity are about equal. 

A possible alternative explanation of the “rebound” in terms of attention is ruled out by the results of the “attention control” condition. It might be postulated that trials in which S2 is different from S1 attract greater attention than those in which it is not, leading to an attention-related enhancement of the response. It would then be unnecessary to invoke adaptation in order to explain the data. In the “attention control” condition, S2 is different from S1 (in terms of dot contrast) in a way that is perceptually salient but largely irrelevant to processing in MT/MST. In macaque, responses in these areas saturate at low contrast (Sclar, Maunsell, & Lennie, 1990), and so this manipulation is not expected to have a large effect on the response. In addition, any effects of increases and decreases in contrast should tend to balance out over trials because the direction of contrast change was randomized. If the enhancement seen in the “different disparity” and “different direction” conditions is an artifact of attention, it should be equally evident in the “attention control” condition. Figure 3 shows clearly that it is not. The response in this condition follows that in the “same” condition closely. 

V1–V4

Retinotopic areas V1, V2, V3, V3A, V3B, and V4 were identified using conventional methods (see Methods section). Satisfactory ROIs were obtained for all areas in both hemispheres in all participants. Figure 2 shows a typical case on a flattened hemisphere. The positions of V1, V2d, V2v, V3d, and V3v are uncontroversial. V4 was defined in the same way as in most recent studies, as a map of the entire hemifield (where discernible) located adjacent to V3v. V3A and V3B were distinguished according to the original criteria of Smith et al. (1998), in which V3B extends posteriorly along the border of V3d to the foveal confluence, not the later definition of Press, Brewer, Dougherty, Wade, and Wandell (2001) and Tyler et al. (2005), in which it forms the more posterior/ventral portion of our V3A. Our V3B corresponds to area LO1 of Larsson and Heeger (2006). 

The time courses for V1, V2, V3, V3A, V3B, and V4 are shown in Figure 4. They are somewhat different from those in MT/MST. All five areas show disparity-specific adaptation: In all cases, the “same” condition shows smaller responses than the “different disparity” condition. This is consistent with the presence of disparity-tuned neurons in these visual areas in macaque (Poggio et al., 1988). The difference from MT/MST is in terms of the level of direction specificity observed. The results for V1 and V2 are similar to each other, and there is no sign of direction selectivity. In these areas, the responses in the “same” and “different direction” conditions are very similar, suggesting that the neurons that are adapted by S1 are, on average, not strongly direction selective. A degree of direction specificity is expected based on macaque neurophysiology since a significant minority of V1 and V2 neurons are direction selective (De Valois et al., 1982; Hubel & Wiesel, 1968; Schiller et al., 1976). One logical interpretation is that such neurons are absent in humans, but it is much more likely that differential directional adaptation is simply not large enough to be detectable with our paradigm in these areas. The results suggest that (i) direction specificity is much weaker in V1/V2 than in MT/MST and (ii) less predictably, disparity sensitivity is much stronger than direction sensitivity in V1 and V2. In V3, V3A, and V4, direction-selective adaptation is again weak or absent. In all three areas, the response is slightly bigger in the “different direction” condition than in the “same” condition but not compellingly so in any of the three areas. Surprisingly, the biggest effect (apart from MT/MST) is in V4, whereas it is expected in V3A, which has previously been shown to be strongly motion sensitive in humans (Tootell et al., 1997). 

Figure 4

 

Mean time courses for areas V1 to V4 as defined by retinotopic mapping experiments. Key as for Figure 3.

The data from the “attention control” trials are not very meaningful in V1–V4 and are not presented. This condition was designed for use in MT/MST, where contrast is largely irrelevant to response magnitude. In V1–V4, it is far from irrelevant. The design will therefore be effective only if there is a perfect balance between increases in response in trials where contrast increases and decreases in those where it decreases. This is hard to achieve because of non-linearities of response and any imbalance will result in an altered response in this condition. Since attentional modulation in visual areas tends to be least in V1 and to increase with distance from V1 (Cook & Maunsell, 2002), and we have good evidence that attention is not contaminating the results in MT and MST, it seems safe to assume that V1–V4 are also free of such contamination. 

In summary, direction selectivity could only be demonstrated convincingly in MT and MST, but robust disparity sensitivity is evident in all areas examined. 

Adaptation indices

In order to quantify the effect of adaptation and to test for statistical significance, an adaptation index was computed, based on a time window from 8 to 12 s, corresponding to when the response to S2 is prominent. First, the “baseline” condition time course was subtracted from that of each of the other conditions to isolate the response to S2 alone in each case. The S2 time courses were then adjusted by subtracting the same constant from each (the lowest value in any of the S2 time courses, for that subject) to give a meaningful baseline (0% signal change). An average was then calculated by collapsing the time courses across all points in the time window. Finally, an adaptation index was calculated at each time point by subtracting the “same” from the relevant “different” condition and then dividing by the “different” condition, i.e., (different − same) / different. This was done separately for disparity and direction. Positive scores on this index therefore represent sensitivity to a change of disparity or direction between S1 and S2. A score of 1.0 would mean that the S2 response in a particular “different” condition was completely obliterated by adaptation in the “same” condition and so can be seen as an index of signal attenuation that can be converted to percent signal attenuation by multiplying by 100. 

Adaptation indices are shown for each visual area in Figure 5 for both direction and disparity. It is clear from this figure that all areas are sensitive to changes in disparity. MST and MT are sensitive to changes in motion direction, as expected. The same is true, to varying degrees, in V3, V3A, V3B, and V4. No measurable direction sensitivity is evident in V1 and V2. Adaptation indices for disparity are generally much larger than for direction. Averaged across all visual areas measured, the mean is 0.15 for disparity and 0.05 for direction. Disparity sensitivity is evident in every area and peaks in V3A. Statistical analysis showed a significant difference between “same” and “different disparity” trials for all areas (V1: t = 2.33; V2: t = 3.90; V3: t = 5.13; V3A: t = 4.82; V4: t = 2.65; MT: t = 3.25; MST: t = 2.51; df = 9; p < 0.05 or better in all cases). A significant difference between “same” and “different direction” conditions was evident only for MT ( t = 2.45; df = 9; p < 0.05). MST narrowly failed to reach significance ( p = 0.06). The difference was non-significant ( p > 0.1) in all other areas. 

Figure 5

 

Adaptation indices (see text) for direction (gray bars) and disparity (black bars). Indices are shown separately for each visual area examined. Error bars show 84% confidence intervals generated using a bootstrap procedure.

Discussion

Numerous neurophysiological studies have shown that neurons in macaque MT and MST are sensitive to the stereoscopic depth of a moving luminance-defined visual stimulus (e.g., Bradley, Qian, & Andersen, 1995; Eifuku & Wurtz, 1999; Maunsell & Van Essen, 1983a; Roy et al., 1992). Moreover, modulatory surrounds are antagonistic in terms of disparity as well as direction (Bradley & Andersen, 1998). Disparity tuning is coarse, usually distinguishing only between near and far (crossed and uncrossed disparities) but, at least for these two broad disparity ranges, stimuli moving in fronto-parallel planes at different depths appear to be encoded by largely separate neural populations. In MST, the situation may be more complex since some neurons respond to both near and far stimuli but with opposite direction preferences (Roy & Wurtz, 1990; Roy et al., 1992). Not only does depth feed into motion computations, but direction and speed of motion can be used to disambiguate false matches and so to improve depth judgments (Bradshaw & Cumming, 1997; Van Ee & Anderson, 2001). Similarly, in macaques, microstimulation of MT can bias judgments of depth (DeAngelis, Cumming, & Newsome, 1998). In this study, we have examined selectivity for both direction and disparity in the human visual cortex, focusing on the MT complex but also documenting selectivity in the retinotopic areas V1–V4. It should be remembered that an adapted response may be inherited from an upstream area, and so sensitivity to the parameter changed may not be generated de novo in the area in question. The same is of course true of neurophysiological response properties: they are commonly inherited. 

Depth selectivity

While sensitivity to motion and depth have both been studied extensively in macaques, only motion has received detailed treatment in the human neuroimaging literature. Nonetheless, several studies, outlined in the Introduction section, have indicated that sensitivity to stereoscopic depth exists in multiple visual areas (Backus et al., 2001; Merboldt et al., 2002; Neri et al., 2004; Nishida et al., 2001; Tsao et al., 2003). These studies suggest that, among the retinotopic areas of the occipital cortex, sensitivity to stereoscopic depth is greatest in V3A. However, none of these studies used moving stimuli. The depth of moving stimuli might be encoded differently from the depth of static stimuli, for example, with a greater involvement of the MT complex. In fact, our study with moving stimuli gives results that are in line with the conclusion drawn from static stimuli, showing a higher disparity-specific adaptation index in V3A than any other area (Figure 5). Although MT and MST show clear disparity sensitivity, it is not the case that when the stimulus is moving, greatest sensitivity to its depth is provided by MT/MST. 

Direction specificity

Motion has been studied considerably more than depth in human neuroimaging studies, but most studies have focused on establishing motion sensitivity by comparing moving with static or flickering stimuli. There have been very few studies addressing direction selectivity. To show the presence of different neuron populations sensitive to different directions, intermingled on the scale of a voxel, it is necessary to use adaptation studies. Only one previous study has reported results from direction-specific human fMRI adaptation (Huk et al., 2001). Our results for direction (Figure 5, top) are generally in line with that study which, like ours, showed much lower levels of selectivity in V1–V3 than in MT+, with V3A showing intermediate selectivity. The main difference between the studies is in V4, where we found strong direction selectivity, whereas Huk et al. (2001) found only a level comparable to that seen in V3. Our finding for direction selectivity in V4 is more in line with that of Tolias et al. (2001), who conducted an MRI adaptation study in macaques. They found strong direction selectivity in V4 but suggest (because of the discrepancy with single-unit studies) that it may arise through disruption of the normal balance of directional inputs to V4 following adaptation. This interpretation is supported by studies of the effects of adaptation on direction tuning in macaque V4 (Tolias, Keliris, Smirnakis, & Logothetis, 2005). However, it should be noted that the homology between macaque V4 and what we have defined as human V4 is far from certain, so it is not possible to make a direct comparison of results. 

Independent or conjoint processing of motion and depth?

The results of our experiment do not tell us whether (i) the same neurons are tuned to both direction and disparity, or (ii) some neurons are tuned to direction but not disparity and others are tuned to disparity but not direction. In our experiments, either direction or disparity differed between S1 and S2, but not both. It would have been possible to include a condition in which both parameters changed, but this would not have addressed the question. It is expected that such trials would give a larger (less adapted) response than when only one parameter was changed, and this outcome is predicted by both models. Separate processing predicts it because at least one of two separate neuron groups would always be adapted in our experiments whereas neither would be if both parameters were changed. Conjoint processing predicts it because the extent of adaptation would be less if neither parameter of the adapter matched the preferences of a neuron than if one of the two parameters did. Current adaptation paradigms are not able to resolve such questions. In macaques, since over 90% of MT neurons are direction selective and over 90% are disparity selective, it follows logically that many neurons are tuned to both stimulus parameters. The most parsimonious explanation of our results is that the same is true in humans. Behavioural studies in humans also suggest conjoint tuning of motion and stereoscopic depth: the motion aftereffect is contingent on stereoscopic depth (Anstis, 1974), suggesting adaptation of direction-sensitive neurons that are responsive to motion only in limited ranges of stereoscopic depth. 

Conclusions

Sensitivity to the stereoscopic depth of a moving stimulus is ubiquitous in the first few visual areas of the human visual cortex. Sensitivity to the direction of motion of the same stimulus builds more gradually as processing progresses from V1 onward. 

Acknowledgments

This research was supported by a grant (GR070500) from The Wellcome Trust to ATS. 

Commercial relationships: none. 

Corresponding author: Andy Smith. 

Email: [e-mail]. 

Address: Department of Psychology, Royal Holloway, University of London, Egham TW20 0EX, UK.