Cone signal interactions in direction-selective neurons in the middle temporal visual area (MT)

Abstract

Many experimental measurements support the hypothesis that the middle temporal visual area (MT) of the rhesus monkey has a central role in processing visual motion. Most of these studies were performed using luminance stimuli, leaving open the question of how color information is used during motion processing. We investigated the specific question of how S-cone signals, an important source of color information, interact with L,M-cone signals, the dominant source of luminance information. In MT, S-cone-initiated signals combine synergistically with L,M-cone (luminance) signals over most of the stimulus range, regardless of whether the stimuli are added or subtracted. A quantitative analysis of the responses to the combination of S- and L,M-cone signals shows that for a significant minority of cells, these S-cone signals are carried to MT by a color-opponent (“blue-yellow”) pathway, such that in certain limited contrast ranges, a small amount of S- and L,M-cone cancellation is observed. Both S- and L,M-cone responses are direction-selective, suggesting that MT processes a wide range of motion signals, including those carried by luminance and color. To investigate this possibility further, we measured MT responses while monkeys discriminated the direction of motion of luminance and S-cone-initiated gratings. The sensitivity of single MT neurons and the correlation between trial-to-trial variations in single neuron firing and perception are similar for S- and L,M-cone stimuli, further supporting a role for MT in processing chromatic motion.

Introduction

The middle temporal visual area (MT) of the rhesus monkey has a central role in the processing of visual motion (Britten, Shadlen, Newsome, & Movshon, 1992; Chawla, Phillips, Buechel, Edwards, & Friston, 1998; Gegenfurtner et al., 1994; Maunsell & Van Essen, 1983; Zeki, 1974; Zeki et al., 1991). Mainly, this role has been documented using luminance motion stimuli, in keeping with the strong luminance input MT receives via the magnocellular pathway (Maunsell, Nealey, & DePriest, 1990). In previous decades, the apparent absence of color responsivity in MT (Maunsell & Van Essen, 1983; Van Essen, Maunsell, & Bixby, 1981; Zeki, 1978b) and the limited input from visual pathways carrying chromatic information led some researchers to hypothesize that MT is not involved in the processing of chromatic motion. Various proposals were entertained, ranging from the possibility that color motion is not perceived or that it is processed elsewhere in the brain (Gegenfurtner et al., 1994; Livingstone & Hubel, 1987; Maunsell & Newsome, 1987; Zeki, 1976, 1978a). 

It is now clear that color motion is perceived. Furthermore, several recent studies have documented MT responses to chromatic motion and raised the possibility that MT contains signals suitable for processing both luminance and chromatically defined motion (Dobkins & Albright, 1994; Gegenfurtner et al., 1994; Saito, Tanaka, Isono, Yasuda, & Mikami, 1989; Seidemann, Poirson, Wandell, & Newsome, 1999; Thiele, Dobkins, & Albright, 2001). However, the chromatic signals in MT have not been characterized by a quantitative model that might be compared with psychophysical performance. 

We therefore undertook an assessment of cone interactions in isolated neurons in MT, measuring how responses to S-cone drifting gratings change when L,M-cone contrast is added in phase and out of phase to the stimulus. Here, we introduce a quantitative model of the responses of MT neurons to chromatic stimuli, and we evaluate the model's performance against the data. We then characterize the relationship between S-cone responses in MT and chromatic motion perception by recording single unit activity while a monkey discriminates the direction of motion in S-cone and luminance Gabor stimuli. We quantify the neural sensitivity to motion in these stimuli and compare it to the simultaneously measured psychophysical sensitivity. We also assess the trial-by-trial correlation of variability in these neural responses with the monkey's perceived direction of motion. We find that (1) for the most part, S-cone-initiated signals combine positively with L,M-cone signals, such that the response initiated by an S-cone signal is never entirely canceled by the addition of an L,M-cone signal; (2) S-cone signals arrive in MT via pathways that are both color-opponent and summative with L,M-cone signals; (3) neural sensitivity to motion initiated in S-cone signals is roughly equal to psychophysical sensitivity; and (4) responses of MT neurons are correlated on a trial-by-trial basis with the monkey's perception of motion initiated in S cones. 

Theoretical background

We measured and modeled the responses of single neurons to a set of stimuli designed to reveal how S-cone signals combine with L,M-cone signals in MT. The stimuli were sums and differences of S- and L,M-cone gratings presented at a range of contrasts. This set of stimuli is depicted in a grid in Figure 1. In this figure, L,M-cone contrast varies along the horizontal axis and S-cone contrast along the vertical axis. The L,M-cone stimuli are created by adding equal contrast L- and M-cone stimuli in the same spatial and temporal phase; these are luminance stimuli. Positive L,M-cone values describe conditions in which the L,M-cone stimulus is combined in phase with the S-cone stimulus (i.e., added), whereas negative values describe conditions in which the L,M-cone stimulus is combined out-of-phase with the S-cone stimulus (i.e., subtracted). In Figure 1, examples of how the gratings would appear are placed within this space. 

Figure 1

 

Stimulus space. Stimuli consisted of sinewave gratings or Gabors containing varying levels of L,M- and S-cone contrast, added either in phase (right half) or out of phase (left half). The stimulus set gives rise to a grid of trial conditions. Approximations of some of the stimuli are placed within this grid. Neural responses are plotted in this space in subsequent figures.

Figure 2 illustrates the responses of an MT neuron as the stimulus varied across the space in Figure 1. The surface height and color in Figure 2A indicate the response magnitude. The three curves in Figure 2B are slices through this surface at constant S-cone contrast levels. The main features of the neural responses illustrated in this figure were present consistently across MT neurons, and these features provide the theoretical basis for our model of MT neuronal responses. First, as L,M-cone contrast is added to or subtracted from an S-cone stimulus, the response magnitude generally increases. Second, there is a small amount of response cancellation. Over a small contrast region, adding L,M-cone contrast to a particular S-cone stimulus elicits a response minimum at a nonzero L,M-cone contrast level (e.g., Figure 2B, 64% S curve). However, this response cancellation is a relatively small effect; adding or subtracting L,M-cone contrast to an S-cone stimulus almost never drives the response back to baseline. Third, the neural responses saturate at higher contrast levels. 

Figure 2

 

Neural response profile for one neuron. (A). Three-dimensional surface plot of the neural response to the grid of stimuli. Heavy lines highlight the purely L,M- and S-cone contrast response functions. The color scale indicates relative response amplitude, running from zero (dark green) to the maximum firing rate (bright yellow). (B). Two-dimensional slices through the neural response profile at three S-cone contrast levels. Error bars are standard error of the mean.

We capture these three properties of the data in a model of MT neural signals that focuses on how cone inputs interact to drive the neural response. The model predicts neural firing rate as a function of cone contrast. We refer to the S-cone contrast as s, and to the (equal) L,M-cone contrasts as k. We refer to the number of action potentials per second as r. The relationship between stimulus contrast and number of action potentials is  

$r=\mathrm{F}(s,k).$

(1)

 

Our first specification of the model reflects the fact that both adding and subtracting L,M-cone contrast to any level of S-cone contrast generally increases the response. We therefore represent the response function as comprising two positive terms (and here add a variable, a, for the spontaneous firing rate):  

$r=\left|{\mathrm{F}}_1\left(s\right)\right|+\left|{\mathrm{F}}_2\left(k\right)\right|+a.$

(2)

 

A second critical feature of the neural response concerns the location of the minimum response as L,M-cone contrast is added to a fixed S-cone contrast. In the absence of S-cone contrast, the L,M-cone contrast response function is symmetric about zero. However, in the presence of some level of S-cone contrast, for some neurons (e.g., Figure 2B), the response minimum shifts slightly toward negative L,M-cone contrasts. This asymmetry or an asymmetry of the opposite sign (i.e., a response minimum at a positive L,M-cone contrast) was evident in many of the MT neurons we recorded. Equation 2 cannot account for this. We incorporate this response feature into the model by assuming that F₁( ) depends on a weighted sum of s and k. Hence, we modify the equation by introducing a dependence on k into the first term:  

$r=\left|{\mathrm{F}}_1(s,k)\right|+\left|{\mathrm{F}}_2\left(k\right)\right|+a.$

(3)

 

To complete the model, we must define the functions F_i that relate stimulus contrast to response. The general form of the contrast response functions in these experiments is common in MT cells and many other cortical visual neurons (Carandini, Heeger, & Movshon, 1997; Simoncelli & Heeger, 1998). We model this relationship using a conventional contrast normalization expression:  

${\mathrm{F}}_1\left(x\right)=m_i\frac{x}{\sqrt{c_i+x^2}}.$

(4)

 

The equation contains one variable, x, which is contrast, and two parameters. The parameters are m_i, which is response magnitude, and c_i, which is most often called the semisaturation constant. These parameters are constrained to be positive. Inserting the contrast function into Equation 3, along with a scaling parameter b, which describes the relative influence of s and k within the left hand term, completes the response model:  

$r=m_1\left|\frac{s+bk}{\sqrt{c_1+{(s+bk)}^2}}\right|+m_2\left|\frac{k}{\sqrt{c_2+k^2}}\right|+a.$

(5)

 

This response model is the basis for our quantitative analysis of how S- and L,M-cone modulations determine the responses of MT neurons. Each of the model parameters summarizes a simple property of the neural response. The positive semisaturation constants characterize the contrast response properties of the two types of input signals. The positive coefficients characterize the relative significance of the cone inputs to the response. 

The coefficient b is of particular theoretical significance for evaluating whether color opponency is evident in the neural response. When b is zero, the model is purely symmetric and does not exhibit the lateral shift of the response minimum as S-cone contrast increases. When b is positive, however, S- and L,M-cone inputs sum, and the response minimum occurs when S- and L,M-cone signals are subtracted from each other (i.e., the stimuli are out-of-phase), yielding a response minimum to the left of zero (e.g., Figure 2). Conversely, a negative value of b indicates opponent interactions between S- and L,M-cone signals, and the response minimum shifts to the right of zero (see Results). A significant portion of our analysis focuses on measuring the sign and value of this parameter and understanding its distribution across our sample of MT neurons. 

We tested other models of the neural response, including a standard polynomial equation, and a series of exponentials, one for each level of S-cone contrast, offset by a certain amount. We also tested other versions of the cone-interaction model. We settled on the model in Equation 5 because (a) it was optimal in terms of being able to interpret fitted parameter values physiologically, (b) it had the fewest number of free parameters, and (c) the results, including the percentage of neurons exhibiting opponency or summation, did not change qualitatively for any of the other models. 

Note that our paradigm is more than a standard response minimization or color mixture experiment; rather, it represents an extensive examination of the cone interactions. By running a full grid of cone contrast combinations, we measure color across a broad range of neural responses. This approach allows us to model the cone interactions much more completely than measurements restricted to estimating a single point in this grid (isoluminant) or the neural responses to any single contrast series (a single line through this grid). 

Methods

We conducted experiments in two adult rhesus monkeys (Macaca mulatta, both female, weight 7–10 kg). Before the experiments, we surgically implanted each animal with a head-holding device (Evarts, 1968), a scleral search coil for measuring eye movements (Judge, Richmond, & Chu, 1980), and a recording cylinder (Crist Instruments, Damascus, MD) that provided access to MT. During experiments, the animals sat in a primate chair with their heads restrained, facing a CRT display. The animals performed a fixation or discrimination task for liquid rewards while visual stimuli were presented within the receptive field of a single MT neuron. All surgical and behavioral procedures conformed to guidelines established by the U.S. Department of Health and Human Services (National Institutes of Health) in the Guide for the Care and Use of Laboratory Animals (1996). 

Behavioral task

For the first experiment, two monkeys performed a simple fixation task while colored gratings were presented on the monitor. Each trial was initiated by illumination of a centrally located small black square. The monkey had 2 s to acquire this fixation point. The stimulus appeared 500 ms after the monkey began fixating and remained on the screen for 1 s. After the stimulus disappeared, another 300 ms passed before the fixation point was extinguished, which signaled the end of the trial. If the monkey successfully completed the trial, maintaining fixation within a 1.0° electronic window, she was rewarded with a drop of juice or water at the end of the trial. If the monkey broke fixation at any point or failed to initiate fixation, the trial was aborted and no reward was given. 

For the second experiment, one monkey discriminated the direction of motion in contrast-varying Gabors (see Visual stimuli). Gabors were used instead of gratings to eliminate position cues at the edge of the stimulus that the monkey might use in performing the task. Each trial began as it did in the fixation task: a fixation point was illuminated, and the monkey had 2 s to initiate fixation. Upon fixation, 280 ms passed, after which two eccentric targets appeared, placed along the axis of motion to either side of the (as yet invisible) stimulus aperture. After another 120 ms, the stimulus appeared, and remained on the screen for 1 s. A final fixation interval of 300 ms occurred before the fixation point went out, which was the monkey's cue to make a saccade to the target that lay in the direction of motion that she had perceived. The monkey was given a maximum of 800 ms to initiate this saccade; typically, saccade initiation time was about 300 ms. After a target was acquired, the monkey had to hold fixation on the target for another 150 ms before the target was extinguished. At this point, a reward was delivered for a correct choice. Neither rewards nor any negative reinforcement was given on incorrect trials. If the monkey failed to adhere to any of the time-interval requirements, either breaking fixation or failing to initiate fixation or a saccade in the allotted time, the trial was aborted and no reward was given. 

Recording methods

We recorded extracellular action potentials from single MT neurons using standard laboratory techniques (Britten et al., 1992). At the beginning of each recording session, the recording chamber was opened, a stainless steel guide tube was inserted 1–3 mm past the dura, and a tungsten microelectrode (Fred Haer, Bowdoinham, ME; 1–5 MΩ) was advanced into the brain through the guide tube using a hydraulic microdrive (Narishige, Tokyo, Japan). We identified MT by the pattern of gray and white matter transitions during descent, by the characteristic physiological properties of MT neurons, and by the topographic organization within MT. Action potentials from isolated neurons were identified using a window discriminator (Bak Electronics, Mount Airy, MD). The occurrence time of each action potential was recorded (Hays, Richmond, & Optican, 1982). 

Experimental protocol

We searched for and recorded from only direction-selective, S-cone responsive neurons. Approximately 95% of the MT neurons we encountered were direction-selective and highly sensitive to luminance contrast. Of these neurons, about 75% also responded measurably to S-cone stimuli. Of the S-cone responsive neurons, about 90% were direction-selective to S-cone motion; the few nondirection-selective neurons were not studied further. 

Each experimental session began by isolating an MT neuron. Upon isolating a neuron, we qualitatively assessed receptive field size and location and the neuron's preferred direction of motion, speed, and spatial frequency. Based on these measurements, we selected the sinusoidal grating stimulus parameters that produced the largest response. 

For the first experiment, we measured neural responses to various combinations of S- and L,M-cone contrast gratings moving in the preferred direction while the monkey fixated. The various colored stimuli were presented in pseudorandom (interleaved) order within a single block of trials. S-cone contrasts ranged from 0% to 64%, and L,M-cone contrasts ranged from −12% to 12%. Each trial type was repeated a minimum of five times, but more typically eight times. The response rate during the interval before stimulus onset served as a measure of the spontaneous firing rate. Our database for this experiment consists of 42 neurons from three hemispheres in two monkeys. 

For the second experiment, we recorded MT neural activity while the monkey discriminated the direction of motion in S-cone and luminance contrast Gabors moving in the preferred or null direction of the neuron under study. Contrast was varied in log steps to measure contrast threshold. We obtained about 20 repetitions of each trial condition (mean = 21, range = 8–40). S-cone and luminance stimuli were presented in separate blocks because we found that the monkey performed suboptimally on S-cone contrast trials when these stimulus types were interleaved. Different contrast levels were pseudorandomly interleaved within each block. Data are included for which the monkey achieved the following criteria for adequate behavioral performance: (a) at least 90% correct at the highest contrast and (b) a threshold that was good and a slope that was reasonably steep, based on the monkey's routine performance. Our database consists of 67 neurons from two hemispheres of one monkey. For 47 of these neurons, we were able to obtain data sets for both stimulus types. Our database contains a total of 59 neurons for S-cone contrast and 55 neurons for luminance contrast. 

Visual stimuli

Visual stimuli were presented on a 48-cm CRT monitor (HP910, Hewlett Packard, Palo Alto, CA). The monitor refresh rate was 100 Hz, and a 12-bit/channel graphics card controlled the primary intensities. We determined the relationship between digital value and signal output level for each of the display primaries (gamma correction), and we verified that the signals from the monitor primaries combined additively (Brainard, Pelli, & Robson, 2002) by repeated photometric measurements (J17 photometer, Tektronix, Beaverton, OR). 

The test stimuli consisted of sinewave gratings for the first experiment and Gabors for the second experiment, windowed with a circular aperture (5–12° diameter). Gabors are sinewave gratings in which the aperture is Gaussian, such that contrast is maximal at the center and falls off smoothly to effectively zero contrast at the edges. Each stimulus contained a single spatial frequency (0.1–1.0 cycles/deg) drawn from a low range to limit chromatic aberration. Stimuli were presented at a 57-cm viewing distance on an achromatic background (45 cd/m²; xy = (0.2813, 0.2941), correlated color temperature = 4036 K). 

Stimulus speed was selected to elicit robust responses from the neuron under study. The temporal frequency range was 0.7–12 Hz (median = 5.7 Hz). We did not observe any correlation between temporal frequency and the interaction between the L,M- and S-cone signals (coefficient b in Equation 5; see Figure 8). 

Using radiometric measurements of the phosphor spectra (PR650, Photo Research, Chatsworth, CA) and the color matching functions (Smith & Pokorny, 1972, 1975), we computed the relationship between display primary intensities and L-, M-, and S-cone contrast (Wandell, 1995). In addition to making this calculation, we measured the final spectral output of the monitor when displaying these stimuli and back-calculated the delivered cone contrasts to estimate the average error in cone contrast presentation. Nominal S-cone contrast errors were almost always less than 1%, and never more than 5%. Nominal L,M-cone contrast errors were always less than 0.5%. Calibration measurements were repeated every 6 months during the course of the experiments. 

Macular pigment and chromatic aberration

Two important factors, beyond display calibration, influence cone absorptions: spatial variations in macular pigment density and longitudinal chromatic aberration. The macular pigment is an inert spectral filter present at different densities across the visual field. It is present in highest density (0.35) in the central visual field, falling to half density at 2.5° (Bour, Koo, Delori, Apkarian, & Fulton, 2002; Chen, Chang, & Wu, 2001). The pigment appears to be absent beyond 8° of eccentricity (Snodderly, Handelman, & Adler, 1991). The receptive field centers in our experiments ranged from 3.2° to 18.0°, so that most receptive fields spanned regions containing a range of macular pigment densities. 

To account for the macular pigment, we calculated cone-isolating directions differentially for receptive fields in the central 5° versus those beyond. For neurons with receptive fields within the central 5°, we used the published Smith–Pokorny fundamentals. For neurons with receptive fields outside the central 5° (30/42 neurons, 71%), we corrected the fundamentals by removing a 0.35 macular pigment density (Bone, Landrum, & Cains, 1992; Stockman, Sharpe, & Fach, 1999). In control experiments with human observers, we confirmed that correcting for this difference produced good S-cone isolation. Specifically, with the macular pigment correction, adaptation to a yellow light increased L,M-cone detection thresholds by at least twofold for three human observers while leaving S-cone thresholds virtually unchanged in the periphery (data not shown). Without this correction for macular pigment density, the yellow light influenced estimated S-cone thresholds. 

In addition, chromatic aberration limits the ability to isolate different cone inputs because the optics transform the contrast separately at each wavelength. In particular, at high spatial frequency, short wavelength signals do not pass through the optics (Marimont & Wandell, 1994), such that the retinal image differs from the calibrated display image. We can substantially allay this concern by restricting the range of spatial frequencies used to below 1 cycle/deg, a cutoff below which chromatic aberration is negligible or absent (Marimont & Wandell, 1994). We were nevertheless able to present stimuli at the optimal spatial frequency for each neuron we studied because MT neurons prefer relatively low spatial frequencies (and we verified this for each neuron we isolated). 

To better understand the consequences of chromatic aberration and misestimation of the macular pigment, we used a simulation of the image formation and cone absorption processes (Cottaris, 2003; Farrell, Xiao, Catrysse, & Wandell, 2003). The simulation begins with the spectral power distribution of the CRT stimuli and simulates the passage of the light through the human optics (Marimont & Wandell, 1994) and the capture of the retinal irradiance by the three types of cone photoreceptors (simulation software can be downloaded from http://white.stanford.edu/~brian/private/JOVAnalysis.htm). Under our experimental conditions, we estimate L-, M-, and S-cone mean absorption rates to be 8.9 × 10³, 7.8 × 10³, and 1.9 × 10³ absorptions per second, respectively. For a 1-cycle/deg test stimulus, with complete certainty in the macular pigment density, a nominal S-cone contrast at the display of 64% produces a retinal S-cone contrast of 22% and L- and M-cone contrasts on the order of 0.4%. Hence, at nominal S-cone contrasts that produce robust responses, for example, 10%, chromatic aberration error introduces less than 0.07% contrast of unwanted signals in the L- and M-cone mosaics. Given that, in our experiments, we used spatial frequencies at or below 1 cycle/deg and that most of the time the preferred spatial frequency was 0.3–0.5 cycle/deg, we can eliminate chromatic aberration as a potential source of error in our calibration. 

Poor estimation of the macular pigment density is a more severe source of error. Table 1 illustrates the size of the estimated error for the case where there is no macular pigment density but where the stimuli are calculated assuming a range of (incorrect) pigment densities. The largest error corresponds to no macular pigment and a stimulus calculation using a density of 0.36. In this case, a 64% nominal S-cone contrast stimulus produces an unwanted M-cone contrast of 3.2%. In practice, an error of this size is unlikely because the macular pigment density falls off to one-half maximum (0.18 density) at 2.5° from the central fovea; the size of the typical MT receptive field excludes the possibility that the average macular pigment density is 0.35. However, at a nominal S-cone contrast of 64%, an unwanted L-cone contrast of 1.1% and an M-cone contrast of 1.7% is a real worst-case possibility. It is important to note that this unwanted cone contrast signal is out of phase with the S-cone signal and thus should appear as an opponent M-cone signal. 

Table 1

 

Estimated cone contrast errors due to chromatic aberration and misestimated macular pigment density. Estimated retinal contrasts to a nominal S-cone stimulus (64%) with no macular pigment and various assumed densities.

Assumed pigment density	Retinal contrast (%)
	L cone	M cone	S cone
None	0.4	0.4	22.0
0.09	0.6	1.2	22.4
0.18	1.1	1.7	22.1
0.36	1.7	3.2	22.0

The macular pigment correction accounts for differences between the fovea and periphery. If the estimated density is systematically wrong, we might expect to observe different S-cone contributions to cell responses when comparing foveal and peripheral receptive fields. A systematic comparison of the S-cone contributions as a function of eccentricity revealed no such differences. 

Cone isolation control experiment

MT neurons are very sensitive to luminance motion contrast (Cheng, Hasegawa, Saleem, & Tanaka, 1994; Sclar, Maunsell, & Lennie, 1990); in the conditions used here, reliable luminance responses were evoked with stimulus contrasts of 3% and, for some cells, with contrasts as low as 1.5%. Because calibrating a monitor to accurately display S-cone-isolating stimuli is notoriously difficult, during the first experiment, we performed a control experiment to verify that responses measured using S-cone-isolating stimuli were not, in fact, driven by unwanted L- or M-cone contrast. In these control experiments, we added a bright yellow light to the background to reduce L- and M-cone contrast (Chatterjee & Callaway, 2002; Seidemann et al., 1999) and measured neural responses to L,M- and S-cone-isolating stimuli in the presence and absence of this background. 

To create the yellow background light, we mounted twelve 75-W halogen lamps in front of the monitor on a track-lighting system. A filter that passes only long wavelengths (>550 nm) was placed in the optical path between the lights and the display (Wratten 15, Eastman Kodak, Rochester, NY). The lamps were arranged so as to create a uniform field of illumination on the monitor and were positioned close to the monitor such that the luminance of the monitor in the presence of the lamps was roughly five times higher than the baseline luminance of the mean gray background (215 vs. 45 cd/m²). 

We performed this control experiment for 17 cells. Two independent contrast series—one S-cone and one L,M-cone—were randomly interleaved within a block of trials. We collected three such blocks of trials, the first and the third under normal illumination and the second under yellow light illumination. Between each block, the monkey adapted to the new lighting condition for a minimum of 4 min, an interval sufficient to ensure stable retinal contrast adaptation (Perlman & Normann, 1998; Poot, Snippe, & van Hateren, 1997; Yeh, Lee, & Kremers, 1996). The stimuli were tailored to the neuron's tuning properties, just as in the main experiment (see Experimental protocol). Stationarity of the responses over time is critical for a valid comparison of responses in a block design of this nature; five additional cells were discarded from this analysis because the baseline firing rate varied by more than a factor of two between the first and third (i.e., non-yellow light) control blocks. 

For S-cone signals, the contrast response function was virtually unchanged in the presence of the yellow background (Figure 3A and 3C), whereas for L,M-cone signals, MT responses were substantially reduced (Figure 3B and 3D). Figure 3A and 3B show responses from a single MT neuron that exhibited a clear effect of the adapting light. Thick lines represent data obtained from blocks when the yellow light was on. The S-cone response function remained nearly constant between the two conditions, whereas the luminance response function was dramatically reduced by the yellow background light. Both signals returned to control levels after removal of the yellow background light. 

Figure 3

 

Cone-isolation control experiment. (A). S-cone contrast response functions, plotted against percent maximum contrast, for one neuron obtained before, during, and after adaptation to the yellow light. Heavy line: yellow light condition. Spontaneous firing rate is indicated by the horizontal dotted line. (B). L,M-cone contrast response functions for the same neuron, plotted as in panel A. (C). Population average (n = 17) of the normalized S-cone contrast response function, plotted against percent maximum contrast. Heavy line indicates the yellow light condition; the regular line indicates the average response during the pre- and postadaption conditions. Error bars are standard error of the mean. (D). Population average of the normalized L,M-cone contrast response function, plotted as in panel C. Maximum contrast varied from cell to cell to capture the dynamic range of the neuron's response, varying primarily as a function of receptive field eccentricity. Maximum L,M-cone contrast varied from 5% to 30%; maximum S-cone contrast varied from 40% to 80%. For each neuron, S- and L,M-cone contrast trials were randomly interleaved within a block.

The average response to the yellow light control for the test population of 17 cells is shown for S- (Figure 3C) and L,M-cone (Figure 3D) stimuli. Heavy lines again depict the yellow-light-adapted condition. Recall that within a particular light adaptation condition, the L,M- and S-cone data were collected on interleaved trials. Although the magnitude of the effect on L,M-cone responses varied somewhat across cells, the population average, which includes standard error bars, clearly shows a consistent effect of the yellow light on these responses and no significant effect on S-cone responses. 

To quantify this effect, for each neuron, the responses were fit with a saturating function, which is the cone-interaction model (Equation 5) reduced to one cone signal:  

$r=m\left(\frac{x}{\sqrt{c+x^2}}\right),$

(6)

where x is either an L,M- or an S-cone contrast, c is a saturation constant, and m is a proportionality constant. Maximum response and contrast threshold were extracted from the fitted equation. Maximum response was given by the response magnitude at maximum contrast; threshold was taken as the contrast at which the neural response was halfway between the spontaneous firing rate and the maximum response. 

Yellow light adaptation elevated the L,M-cone threshold 2.5-fold, on average, whereas the S-cone threshold remained unchanged; adaptation reduced the L,M-cone maximum response to about 40% of its unadapted value, on average, whereas the S-cone maximum response again remained unchanged (Table 2). 

Table 2

 

Cone-isolation control experiment. Ratio of measures of the mean neural response under the yellow-light condition to the response under the normal light condition, computed separately for each neuron. Mean and standard deviation of these ratios are given.

	L,M cone	S cone
Contrast threshold	2.43 ± 1.35	1.04 ± 0.24
Response magnitude	0.36 ± 0.34	1.00 ± 0.54

We also considered the possibility that the neural responses may be driven significantly by rod-initiated signals. There are two reasons we think that this is unlikely. First, the mean scotopic luminance in the yellow-light condition is 298 cd/m² whereas the monitor alone is 132 cd/m². Hence, the added yellow light reduced any rod contrast by a factor of two, and this contrast reduction should have been matched by a sensitivity loss; however, it was not. Second, from simulations, we estimate the unwanted rod contrast caused by the S-cone-isolating stimuli to be from 1% to 6%. Behavioral increment thresholds suggest a contrast threshold of 10% over most of the scotopic range, rising to more than 100% at background levels comparable with those of the yellow light control (Aguilar & Stiles, 1954). If behavioral thresholds measure sensitivity limited by the rods, then the unwanted rod contrast is too low to influence the MT neural responses. 

In practice, we were generally unable to maintain single-cell isolation long enough to collect responses to both the data set for the first experiment (e.g., Figure 2A) and the three blocks of the yellow-light control experiment. Thus, the interaction grids and the yellow light controls were performed on largely nonoverlapping sets of cells. It seems safe to assume, however, that the results of the yellow-light control experiments are applicable to our entire pool of MT neurons because (a) the results of the yellow light control were consistent across neurons and (b) the selection criteria, range of tuning properties, and visual stimuli were the same for the two groups of experiments. We take the results in Figure 3 as a confirmation that responses to the nominally S-cone stimulus were indeed driven by S-cone contrast, rather than by unwanted L- or M-cone contrast. 

Data analysis: Experiment 1

In the first experiment, we fit each data set with a cone-interaction model to determine the nature of the cone interactions in the neural response. 

Cone-interaction model

We fit the cone-interaction model (Equation 5) to each neural data set using a minimum-slope search algorithm (fminsearch in Matlab). The fit minimized a model error comprised of the sum of two terms: (a) the sum of squared errors between the data and the model and (b) the sum of squared errors between the derivative of the data and the derivative of the model. The first term requires the model to fit the data. The second term enforces a certain degree of smoothness onto the fitted surface. This was necessary because the five-parameter model at times produced unwanted approximations to individual data points. The derivative was calculated by taking the difference between neighboring pairs of points on the grid. This was done for the two cardinal directions (along the x and y axes), but not along the diagonals. 

We quantified the goodness-of-fit of the cone-interaction model using a metric that compares the ability of the model to predict the data (“model error”) to the ability of the data to predict itself (“data error”). We calculated the data error by dividing the individual responses to each stimulus randomly into two groups, and treating one group as the “data” and the other as the “prediction.” We considered the sum of squared deviates between the two sets to be the data error. Model error was the sum of squared deviates between the model, fit to the “data” group data set, and the data points derived from the “prediction” group data set. (This was done to keep the variability comparable in these two error metrics.) A model error that is equal to or less than the data error indicates that the model fits the data as well as can be expected given the intrinsic variability of the data. We devised this metric because we could not find one among the standard stock of statistical tests (e.g., chi-squared) that dealt adequately with two-dimensional data and for which the expected value for a good fit was easily determined. 

Ninety-five percent confidence intervals for the fitted parameters were computed using a bootstrap method (resampling each data point with replacement, then refitting the cone model) with 500 repetitions. 

Data analysis: Experiment 2

In the second experiment, we performed two analyses on the simultaneously collected neural and behavioral data. The first analysis, the neurometric–psychometric (NP) comparison, allows us to quantify the sensitivity of the neuron under study to motion relative to the monkey's psychophysical sensitivity to that same motion. The second analysis, choice probability (CP), assesses the degree of correlation between the trial-to-trial variability in the neural response and the monkey's perceived direction of motion. 

Neurometric–psychometric comparison

To assess sensitivity to motion, we measured neural responses to S-cone and luminance stimuli at a range of contrasts. We obtained psychophysical contrast threshold and slope for each contrast series by fitting a sigmoid curve to the data set (percent correct as a function of contrast; e.g., see Figure 11B). We used a cumulative Weibull function, which has the form:  

$p=1-0.5{\mathrm{e}}^{-{\left(\frac{x}{\mathrm{\alpha }}\right)}^{\mathrm{\beta }}},$

(7)

where x is the contrast and p is the proportion correct (range = 0–1). On a logarithmic x-axis, the coefficients α and β determine the horizontal offset and slope of the curve, respectively (Treutwein, 1995). We defined contrast threshold as the contrast at which the monkey performed at 75% correct (the midpoint between chance and perfect performance). 

More generally, this psychophysical function has the form:  

$p=s-(s-z){\mathrm{e}}^{-{\left(\frac{x}{\alpha }\right)}^{\beta }},$

(8)

where s is the saturating level of performance (units, proportion correct; range = 0–1) and z is chance performance level. In 2AFC paradigms, z = 0.5. When s = 1, Equation 8 reduces to Equation 7. For a few data sets, a better fit was obtained using Equation 8 (with z = 0.5 and s allowed to vary), as revealed by a better chi-squared measure of goodness-of-fit. This reflected the fact that in those experiments, maximal performance was somewhat less than 1. In these cases, Equation 8 was used to obtain estimates of α and β. 

We used a comparable metric, called the neurometric function, to compute the sensitivity of individual neurons (Britten et al., 1992). In brief, for each stimulus type and contrast, we assessed the ability of an ideal observer to distinguish between preferred and null motion by calculating the receiver operating characteristic (ROC). The ROC curve was constructed by plotting the proportion of preferred motion trials for which the average neural firing rate falls above a criterion value, against the same quantity calculated for null motion trials, for each possible criterion from zero to the maximum observed firing rate. The area under the ROC curve represents the percent of trials in which an ideal observer could correctly identify the direction of motion at this contrast (Green & Swets, 1966). The resulting data set—proportion correct as a function of stimulus contrast—was then fit with a sigmoid function, and the threshold and slope of this function was extracted using the same method as for the psychophysical data (Figure 11B). 

This analysis gives us the ability to examine the relationship between the two thresholds (or slopes) by calculating their ratio. The neural metric was divided by the psychophysical metric to yield the NP ratio. This quantity was calculated individually for each neuron. The average NP ratio was computed using the geometric mean. 

Choice probability

A neuron's response to a particular stimulus is variable. The behavioral response is also variable for contrasts in the threshold region: on some trials, the monkey guesses correctly, on other trials incorrectly. To assess whether these two variables are correlated on a trial-to-trial basis, we calculated a quantity we refer to as choice probability (Britten, Newsome, Shadlen, Celebrini, & Movshon, 1996). 

The CP calculation uses a metric similar to the NP comparison but compares the distributions of firing rates when the monkey chose one direction of motion versus the other for a single-stimulus condition (e.g., see Figure 12B). Note that these distributions are distinct from those used in the previous analysis, because they are sorted by behavioral choice rather than by the direction of stimulus motion. This analysis is only possible for stimulus conditions for which the monkey made errors and thus made different choices on different trials. We therefore only performed this analysis for conditions in which the monkey made at least three errors. CP is a metric that varies between .5 and 1 and indicates the probability with which we can predict the monkey's choice, given the neural firing rate on a particular trial. When CP = .5, we do no better than chance; when CP = 1, we can predict perfectly the monkey's response. 

To calculate a single CP value for each neuron, using all of the trials across stimulus conditions, we first normalized the responses within each stimulus condition because different conditions elicited different average levels of firing. To do so, we first converted firing rates on individual trials to z-scores, normalized to the average firing rate for that trial condition. 

Results

For the first experiment, we collected data sets from 42 cells in two monkeys. S-cone responses were smaller than L,M-cone responses (Figure 4A and 4B), much as Seidemann et al. (1999) observed. A direct comparison of the magnitude of these responses is not possible without converting S- and L,M-cone contrast to a common unit. Accordingly, we selected from our data sets the trial conditions that were purely S- or L,M-cone contrast and plotted the response magnitude as a function of the root-mean-square (RMS) of total cone contrast (Figure 4C). Dividing each measurement of response amplitude by the %RMS cone contrast with which it was elicited yielded a measurement that could be directly compared (Figure 4D). For the cell shown in Figure 4A to 4D, the mean L,M-cone response, in units of spikes per second per unit RMS contrast (response-per-RMS), was 7.36, whereas the mean S-cone response was 0.66. The response-per-RMS ratio (S:L,M) was 0.09. 

Figure 4

 

Response magnitude. Relative response magnitude to S- and L,M-cone stimuli. (A). Raster and PSTH for one neuron at 12% L,M-cone contrast. (B). Raster and PSTH for the same neuron at 64% S-cone contrast. (C). Average response (firing rate during stimulus presentation) for the same neuron for several levels of L,M- and S-cone contrast, plotted as a function of RMS cone contrast; each data point represents one stimulus condition. Pairs of points for L,M-cone contrast are for in phase and out-of-phase stimuli. (D). Response-per-unit RMS for each of the trial conditions shown in panel C. Solid lines mark average response-per-unit RMS. (E). Distribution of the ratio (S:L,M) of average response-per-unit RMS for the 42 neurons in the primary data set.

Figure 4E displays the distribution of response-per-RMS ratios for the population of neurons in this experiment. The ratios spanned a range, centering on 0.1 (mean = 0.10, SD = 0.08), which indicates that MT neural responses to S-cone contrast are, on average, 10% of that to L,M-cone contrast. It may be noted that the response to the low S-cone contrast was sometimes quite small and that the profile of responses to the range of S-cone contrasts was not necessarily linear (e.g., Figure 4C). If small responses represent a neural response that has not yet crossed a threshold of activation, including these values may skew our estimates of the average response to cone contrast. We therefore also computed our response-per-RMS ratios using only responses that exceeded 10% of the spontaneous firing rate. The overall result was very similar (mean = 0.11, SD = 0.07). 

The majority of MT neurons in our experiment (69%) were not color-opponent, exhibiting instead some degree of summation of their cone inputs. Figure 2 illustrates a data set obtained from one MT neuron. We collected data sets from 42 cells in two monkeys and observed a range of response profiles. Figure 5 illustrates data sets from three additional neurons, which span the range of response profiles that we observed. The data are displayed as three-dimensional surface plots (Figure 5A, 5C, and 5E) and as slices through constant levels of S-cone contrast (Figure 5B, 5D, and 5F). For the neuron in Figure 5A and 5B, the minimum response region fell to the left of the S-cone contrast axis. In contrast, the neuron in Figure 5C and 5D yielded minimum responses along the S-cone contrast axis, whereas the neuron in Figure 5E and 5F yielded minimum responses in the right quadrant. 

Figure 5

 

Types of neural response profiles. Three neurons are shown that illustrate the range of response profiles observed. For each cell, two views are displayed: a three-dimensional surface plot, and 3 two-dimensional slices at different S-cone contrast levels. In the surface plots, heavy lines follow the purely S- and L,M-cone contrast response functions, and data points are indicated by black dots. The flat mesh grid in the three-dimensional plots and the dashed line in the two-dimensional plots indicate the spontaneous firing rate. The color scale indicates relative response amplitude, running from zero to the maximum firing rate of each cell. (A) and (B). Summation. (C) and (D). Independence. (E) and (F). Opponency.

Model fits

Figure 6 shows the model (Equation 2) fits for each of the neurons shown in Figure 5. The model surfaces capture the main features of the data. Figure 7 is a scatter plot of the model error against data error for each cell. The model error is equal to or slightly smaller than the data error. The comparison in Figure 7 suggests that the model fits the data well and that further fitting would be beyond the reliability of the data. 

Figure 6

 

Cone model fits. Cone model fits for each of the neurons shown in Figure 5. Each panel corresponds to the same panel in Figure 5. Vertical bars in two-dimensional slice plots indicate deviation of model from data points. (Note that in panel D, the blue line falls directly behind the magenta line, and so is largely not visible.)

Figure 7

 

Goodness-of-fit metric for the model fits. The model error, computed as the SSE for the model fit to half of the data, as a function of the data error, computed as the SSE of half of the data to the other half of the data. The solid line is the identity line. Red symbols mark the values for the four neurons shown in Figures 2 and 5.

Analysis of S- and L,M-cone color circuitry

The contributions of S- and L,M-cone signals to a neuron's responses can be assessed in several ways. First, note that the two principal terms in Equation 2, one containing an S-cone-initiated signal and the other containing only an L,M-cone-initiated signal, combine positively. These two additive terms capture the fact that adding or subtracting L,M-cone contrast from a fixed S-cone stimulus primarily increases the firing rate (see Theoretical background). 

As described in Theoretical background, the sign of the coefficient b is the critical parameter for evaluating whether the cone interactions show any opponency. This is necessarily the case, because in the model, the only free parameter that can mediate any opponent interaction between cone signals before rectification is b. Asymmetries in the neural response profile, which are evident in many of the neurons we recorded (e.g., Figures 2 and 5), must arise from interactions prior to rectification. Figure 8A shows that for the majority of cells, b is greater than zero, indicating summation of S- and L,M-cone signals. However, parameter b is negative for a significant minority of cells, indicating subtraction (i.e., color opponency). The 95% confidence interval for each estimate of b is shown in Figure 8B; few of the confidence intervals include zero. Thus, our sample of MT cells contained units that reliably summed as well as cells that reliably differenced S- and L,M-cone signals. 

Figure 8

 

Range and confidence intervals for coefficient b of the model fits. (A). Distribution of coefficient b of the model fits for the population of neurons. Positive values indicate summation; negative values indicate opponency (mean = 6.25, median = 2.22, range = −56.79 to 26.59; mean significantly different from zero; t-test, p < .001). (B). Same data as in panel A, where the value and 95% confidence interval of coefficient b are plotted with a point and a line for each cell, ordered by the value of b. (C). Distribution of the magnitude (i.e., the absolute value) of coefficient b, plotted on a log scale (mean for both distributions combined = 10.26, median = 13.55). Coefficients that were positive and negative are plotted in separate histograms, as indicated in the inset.

The magnitude of b measures the relative contribution of the L,M- and S-cone contrasts to the response component represented by the first term. The distribution of the magnitude of b is plotted in Figure 8C separately for positive and negative values. Regardless of whether the cone inputs add or subtract, the magnitude of b centers near a value of 10 (10.26 for additive signals, b > 0; 13.55 for differencing signals, b < 0). 

Figure 9 shows the range of values assumed by the other four parameters in the model, as well as their 95% confidence intervals. Note that although the ranges of parameters c₁ and c₂ (Figure 9A and 9B) were quite broad, the confidence intervals were, in general, relatively narrow. Each parameter controls the curvature of the saturating contrast response function for each cone signal, and our neurons exhibited a wide range of profiles, both in terms of degree of curvature, concavity or convexity, and how quickly they saturated. This led to a large range of values for the parameters c₁ and c₂. 

Figure 9

 

Range and confidence intervals for coefficients of the model fits. (A). Distribution (upper panel) and 95% confidence intervals (lower panel) for coefficient c₁: mean = 2764, median = 1.29 × 10⁶, range <0.01 to 2.72 × 10⁷. (B). Coefficient c₂: mean = 204.9, median = 2.25 × 10⁵, range <.01 to 7.16 × 10⁶. (C). Coefficient m₁: mean = 17.78, median = 34.09, range = 0.58 to 168.22. (D). Coefficient m₂: mean = 20.92, median = 27.48, range <0.01 to 205.4.

The contribution of S-cone contrast relative to L,M-cone contrast to MT firing rates can also be estimated from the model fits. Because b is relatively small, the S-cone contribution is roughly m₁; the L,M-cone contribution is given by m₁(&z.sfnc;b&z.sfnc;) + m₂. (Note that this is only an approximation; Equation 5 does not yield a simple term that allows us to compare s and k weightings in terms of all five coefficients.) The ratio of these two quantities, m₁/(m₁(&z.sfnc;b&z.sfnc;) + m₂), is shown for each cell in Figure 10A. The distribution centers on a mean of 0.076, which, interestingly, is comparable to the ratio of S-cone to L- and M-cone photoreceptors in the retina (Curcio et al., 1991; Wandell, 1995). 

Figure 10

 

Further analysis of parameters of the model fits. (A). Distribution of the weight of S-cone contrast relative to total L,M-cone contrast in the model fits, captured by the value of m₁/(m₁&z.sfnc;b&z.sfnc; + m₂). (B). Scatterplot of the relative weight of the first and second terms of the model, captured by the value of m₁/(m₁ + m₂), plotted as a function of coefficient b. Gray ovals indicate regions of this space where neurons in different color pathways would fall (see Results).

The values obtained for the parameters of the cone model may give us a hint about the origin of the chromatic responses in MT. In Figure 10B, we have plotted the quantity m₁/(m₁ + m₂) against the parameter b. The parameter b tells us the relative weight of S to L,M within the term in which they interact, and m₁/(m₁ + m₂) measures the relative weight of that term with respect to the whole response. We then note where the different color channels, measured at the level of the retinal ganglion cells and the lateral geniculate nucleus (LGN), fall within this plot. Koniocellular LGN neurons have opposed S- and L,M-cone inputs, weighted roughly equally (b approximately −1), and little if any luminance input (m₁/(m₁ + m₂) is large). Hence, koniocellular neurons fall in the upper center part of the parameter space, indicated by a small gray ellipse. We found no cells in the region of the space that would correspond most closely to koniocellular responses. Classically described luminance (magnocellular) neurons, with no S-cone signal (m₁ approximately 0), would fall near a line at the bottom of this space. (The value of b is undefined, so all values of b are included.) Magnocellular neurons with a small S-cone signal, as described by Chatterjee and Callaway (2002), fall in the region where b is approximately 10, given the relative weight of cone signals they observed within those cells (large gray ellipse). Few of our neurons fall within the “classical” luminance region, but many fall within the ellipse that describes the magnocellular data of Chatterjee and Callaway (2002). 

Neurometric–psychometric comparison

For the second experiment, we recorded neural responses while a monkey discriminated the direction of motion in the stimulus. Figure 11A and 11B displays the data obtained from one neuron using S-cone stimuli. The neural response is shown in terms of the average firing rate (Figure 11A) and the neurometric function (Figure 11B) as a function of stimulus contrast. Additionally, Figure 11B depicts the monkey's performance. 

Figure 11

 

Neurometric–psychometric comparison. (A). Average response (firing rate during stimulus presentation) for one neuron as a function of S-cone contrast, for preferred- and null-direction motion. Error bars denote standard deviation. Dashed line indicates spontaneous firing rate. (B). Neurometric function (black) derived from the data in panel A, with the psychometric function (magenta) for the monkey overlaid. Vertical dashed lines indicate contrast threshold. Horizontal dashed line indicates chance performance level. (C). Distribution of the NP threshold ratio for S-cone contrast stimuli. Black triangle marks average NP ratio across all cells. Vertical dashed line marks a ratio of 1. (D). Distribution of the NP threshold ratio for luminance contrast stimuli. Symbol and dashed line as in panel C.

Both neurometric and psychometric functions varied from experiment to experiment, as did the relationship between them. For the neuron shown in Figure 11A and 11B, behavioral sensitivity was slightly greater than neural sensitivity, for both S-cone and luminance contrast (the latter is not shown). To visualize the general relationship between neurons and behavior, we calculated the neuronal and psychophysical threshold for each stimulus (S-cone and luminance) for each cell and compiled distributions of the threshold ratio values (Figure 11C and 11D). The geometric mean of the NP ratio was nearly identical for S-cone and luminance stimuli (threshold = 0.97 and 0.93, respectively), as were the ratios of the slopes of the neurometric and psychometric functions (S-cone slope ratio = 1.02; luminance slope ratio = 1.06). The distributions were not statistically different, either for threshold or for slope. It is reassuring to note that these values are similar to those obtained by Britten et al. (1992), using a different luminance stimulus. 

Individual MT neurons are apparently carrying motion signals of at least two types: luminance and chrominance. Is the sensitivity of a neuron to luminance motion correlated with its sensitivity to chromatic motion? We examined the relationships between S-cone and luminance threshold and between S-cone and luminance slope. Both parameters of the neurometric functions were correlated, threshold more strongly than slope (threshold, r = .62, p < .001; slope, r = .32, p < .05). Thus, we see no evidence for separate populations of cells with high sensitivity to only luminance or chromatic stimuli. 

Choice probability

CP quantifies how well the response of the neuron under study predicts the animal's perceptual decisions for a single-stimulus condition. Figure 12A displays a raster and peristimulus-time histogram (PSTH) of responses to repeated presentation of 4% S-cone contrast moving in the cell's preferred direction. Trials for which the monkey chose the preferred direction of motion are shown in blue, whereas null-direction choice trials are shown in red. Neural firing rates are variable from trial to trial, and previous studies using random dot stimuli have shown that a higher-than-average firing rate on any individual trial is associated with preferred-direction choices by the monkey. 

Figure 12

 

Choice probability. (A). Raster and PSTH for one neuron, for 4% S-cone contrast stimuli moving in the preferred direction. Rows of the raster are color-coded according to whether the monkey chose the preferred (blue) or null (red) direction at the end of the trial. (B). Distributions of average firing rate for the trials shown in panel A, for chose-preferred and chose-null subsets. Black triangles mark the mean of each distribution (blue, 30.60 sp/s; red, 26.33 sp/s). (C). Distribution of the CP value, computed across all trials, for each neuron in the data set, for S-cone contrast stimuli. Black triangle marks the mean of the distribution. Darker shading indicates neurons that were individually significant (p>0.5). (D). Distribution for luminance contrast stimuli. Symbol and shading as in panel C.

Figure 12B shows firing rate distributions for preferred- and null-direction choice trials. The blue distribution is shifted rightward with respect to the red distribution, indicating that the neuron indeed fired slightly more, on average, on trials for which the monkey chose the preferred direction. For the stimulus condition shown in Figure 12A and 11B, a signal detection analysis yielded a CP of .611 (see Methods). A CP of 1 indicates a perfect correlation between fluctuations in neural firing rate and behavior, whereas a CP of .50 indicates that there is no relationship between the magnitude of the neuron's response and the monkey's choice. 

A CP can be calculated for any stimulus condition for which the monkey divides her choices between the two alternatives. To obtain a single CP for each neuron, we analyzed all trials for that neuron after converting firing rates within each contrast level to z scores (see Methods). Figure 12C displays the CP distributions for all neurons in this data set for S-cone stimuli, and Figure 12D displays the same distribution for luminance stimuli. The average CP for S-cone stimuli was .525, and this mean was significantly higher than .5 (t-test, p < .001). Individual neurons for which CP was significantly higher than .50 are shaded (permutation test, p < .5). 

The result for luminance stimuli was very similar (mean CP = .528, p < .001). Britten et al. (1996) obtained a similar average CP (mean = .548) using a different luminance stimulus. Most importantly for our purposes here, the means of the distributions for S-cone and luminance Gabors were not statistically different (p > .05). 

An important subset of trials to examine is 0% contrast, when no stimulus is actually present on the monitor. Some artifactual causes of the CP can be ruled out if the CP for this stimulus condition alone is significant (see the Discussion section in Britten et al., 1996). CP values for the 0% contrast condition alone were significantly greater than .5 for both S-cone (CP = .532) and luminance (.520) stimuli, and these distributions were not statistically different from each other (p > .05). 

Discussion

In MT, responses initiated in the S-cones frequently sum with responses initiated in the L and M cones. In a minority of cells, these S-cone signals reliably difference from L- and M-cone signals (Figure 8A). Thus, both types of chromatic signals are present in MT. In addition, comparison of simultaneously collected neural and psychophysical responses to S-cone and luminance stimuli yields two results that are consistent with the idea that MT could mediate the processing of chromatic motion. Analysis of neurometric and psychometric thresholds shows that the relationship between neural and psychophysical sensitivity is nearly identical for chromatic and luminance stimuli, despite substantial differences in absolute sensitivity to the two types of stimuli. In addition, significant choice probabilities of equal magnitude indicate that MT neural activity is correlated on a trial to trial basis with psychophysical judgments for both S-cone and luminance stimuli. 

The middle temporal area carries a diverse range of chromatic motion responses

Signals from all cone types modulate neural responses in MT. S-cone-initiated signals are weaker, however, than L- and M-cone-initiated signals (Seidemann et al., 1999; current study). The variety of chromatic cell types (Gegenfurtner et al., 1994; Saito et al., 1989; Seidemann et al., 1999; current study) shows that motion-selective cortex responds to the motion of objects of any color, consistent with psychophysical measurements (Cavanagh & Anstis, 1991; Chichilnisky, Heeger, & Wandell, 1993; Dougherty, Press, & Wandell, 1999). It is possible that the diversity of chromatic cell types simply ensures that motion will be detected for any color, but that the chromatic information is not sufficiently organized in MT to code the color of the moving object. In this scenario, MT carries what Albright and colleagues have dubbed “unsigned” chromatic signals (Dobkins & Albright, 1998; Thiele, Dobkins, & Albright, 1999). However, the existence of color-opponent responses in MT (Dobkins and Albright, 1994; Gegenfurtner et al., 1994; current study) raises the possibility that color identity, as well as direction of motion, can be decoded from MT responses. Regardless of the answer to this question, our results are consistent with the idea that chromatic motion is processed by MT (Ffytche, Skidmore, & Zeki, 1995; Thiele et al., 2001) as a subclass of all motion stimuli, rather than by, for example, visual areas in the ventral stream. 

Sources of middle temporal chromatic responses

Our results imply that many S-cone-initiated signals arrive in MT via a pathway that sums S- and L,M-cone signals. One possible source of such a signal is the magnocellular pathway, which is known to provide the dominant input to MT (Maunsell et al., 1990). In this regard, our observations correspond with the recent findings that S-cone inputs are present in the magnocellular layers of the LGN (Chatterjee & Callaway, 2002) and that these S-cone inputs sum with the well-known L- and M-cone inputs to magnocellular neurons (Chatterjee & Callaway, 2002). Furthermore, the strength of the S-cone responses relative to L,M-cone responses (roughly 1:10) is very similar in the magnocellular LGN (Chatterjee & Callaway, 2002) and in MT (Seidemann et al., 1999; Wandell et al., 1999; current study). Alternatively, S-cone signals are known to combine in novel ways with L- and M-cone signals in V1 (De Valois, Cottaris, Elfar, Mahon, & Wilson, 2000), and MT might simply inherit signals that have been mixed at the cortical level. 

We also observed color-opponent S-cone responses in a significant minority of MT neurons. These signals could, in principle, derive from either of two established upstream sources: an S-[L,M] signal carried by the large bistratified ganglion cells (Chichilnisky & Baylor, 1999; Dacey & Lee, 1994; Martin, White, Goodchild, Wilder, & Sefton, 1997) or an S-off parvocellular cell type (Klug, Herr, Ngo, Sterling, & Schein, 2003). Area MT receives a modest input from the parvocellular pathway via V1 (Maunsell et al., 1990). In addition, a direct projection to MT from the koniocellular layers of the LGN has recently been identified (Sincich, Park, Wohlgemuth, & Horton, 2004). However, the responses of the MT color-opponent cells were fit best by a model in which the ratio of S- to L,M-cone responsivities was roughly 1:10 (see Figure 10B), which differs significantly from the roughly equal S- and L,M-cone weights described in both the small, bistratified ganglion cells of the retina and in the koniocellular layers of the LGN (Derrington, Krauskopf, & Lennie, 1984). We are not sure how to account for this discrepancy. 

Chromatic and luminance motion are processed similarly in the middle temporal area

Area MT has been shown to play a central role in the processing of luminance motion (Newsome & Paré, 1988; Salzman, Britten, & Newsome, 1990; Salzman, Murasugi, Britten, & Newsome, 1992). If MT encodes motion from a broader range of visual cues for the purposes of motion perception, we would expect to observe the same relationships between neuronal activity and psychophysical performance for different types of motion stimuli. If, on the other hand, another brain area performs the perceptually relevant processing of chromatic motion signals, we might expect to observe substantial differences in the relationship between MT activity and performance for S-cone motion stimuli. In fact, we found that chromatic motion responses in MT bear the same signature in relation to psychophysical performance as do luminance motion responses, including thresholds and slopes of the psycho/neurometric functions, and CP. These data support the notion that MT mediates the perception of motion carried by a range of visual cues. 

Presuming that MT plays an important role in processing the motion of chromatic stimuli, it is interesting to ask whether single cells respond to both luminance and chromatic motion stimuli, giving rise to cue-invariant motion signals (Albright, 1992; O'Keefe & Movshon, 1998; Stoner & Albright, 1992) or whether different subpopulations of MT neurons are dedicated to different types of motion. The fact that neuronal thresholds for the two types of motion were correlated argues in favor of the former idea. 

Conclusion

In this article, we have provided the first detailed quantitative model of the responses of MT neurons to chromatic and luminance stimuli, and we have shown that the relationship between MT activity and psychophysical performance on a direction discrimination task is indistinguishable for the two types of stimuli. Our data lend weight to the hypothesis that MT processes perceptually relevant motion signals for all types of stimuli, including chromatic ones. A definitive test of this idea will probably require inactivation studies of the kind that first demonstrated a functional role for MT in the perception of luminance-defined motion (Newsome & Paré, 1988). 

Acknowledgments

This work was supported by the National Eye Institute (EY-05603 to W.T.N. and EY-03164 to B.A.W.) and by the Howard Hughes Medical Institute (W.T.N. is an HHMI investigator). C. L. Barberini and M. R. Cohen were supported by HHMI Predoctoral Fellowships. We thank Jessica Powell and Stacy Rosenbaum for excellent technical assistance. We also thank Greg Horwitz and Robert Dougherty for helpful suggestions during editing of the manuscript. 

Commercial relationships: none. 

Corresponding author: Crista L. Barberini. 

Email: [e-mail]. 

Address: 2236 8th Street, Berkeley, CA 94710.