Many experimental measurements support the hypothesis that the middle temporal visual area (MT) of the rhesus monkey has a central role in processing visual motion. Most of these studies were performed using luminance stimuli, leaving open the question of how color information is used during motion processing. We investigated the specific question of how S-cone signals, an important source of color information, interact with L,M-cone signals, the dominant source of luminance information. In MT, S-cone-initiated signals combine synergistically with L,M-cone (luminance) signals over most of the stimulus range, regardless of whether the stimuli are added or subtracted. A quantitative analysis of the responses to the combination of S- and L,M-cone signals shows that for a significant minority of cells, these S-cone signals are carried to MT by a color-opponent (“blue-yellow”) pathway, such that in certain limited contrast ranges, a small amount of S- and L,M-cone cancellation is observed. Both S- and L,M-cone responses are direction-selective, suggesting that MT processes a wide range of motion signals, including those carried by luminance and color. To investigate this possibility further, we measured MT responses while monkeys discriminated the direction of motion of luminance and S-cone-initiated gratings. The sensitivity of single MT neurons and the correlation between trial-to-trial variations in single neuron firing and perception are similar for S- and L,M-cone stimuli, further supporting a role for MT in processing chromatic motion.

*s*, and to the (equal) L,M-cone contrasts as

*k*. We refer to the number of action potentials per second as

*r*. The relationship between stimulus contrast and number of action potentials is

*a*, for the spontaneous firing rate):

_{1}( ) depends on a weighted sum of

*s*and

*k*. Hence, we modify the equation by introducing a dependence on

*k*into the first term:

_{i}that relate stimulus contrast to response. The general form of the contrast response functions in these experiments is common in MT cells and many other cortical visual neurons (Carandini, Heeger, & Movshon, 1997; Simoncelli & Heeger, 1998). We model this relationship using a conventional contrast normalization expression:

*x*, which is contrast, and two parameters. The parameters are

*m*

_{i}, which is response magnitude, and

*c*

_{i}, which is most often called the semisaturation constant. These parameters are constrained to be positive. Inserting the contrast function into Equation 3, along with a scaling parameter

*b*, which describes the relative influence of

*s*and

*k*within the left hand term, completes the response model:

*b*is of particular theoretical significance for evaluating whether color opponency is evident in the neural response. When

*b*is zero, the model is purely symmetric and does not exhibit the lateral shift of the response minimum as S-cone contrast increases. When

*b*is positive, however, S- and L,M-cone inputs sum, and the response minimum occurs when S- and L,M-cone signals are subtracted from each other (i.e., the stimuli are out-of-phase), yielding a response minimum to the left of zero (e.g., Figure 2). Conversely, a negative value of

*b*indicates opponent interactions between S- and L,M-cone signals, and the response minimum shifts to the right of zero (see Results). A significant portion of our analysis focuses on measuring the sign and value of this parameter and understanding its distribution across our sample of MT neurons.

*Macaca mulatta*, both female, weight 7–10 kg). Before the experiments, we surgically implanted each animal with a head-holding device (Evarts, 1968), a scleral search coil for measuring eye movements (Judge, Richmond, & Chu, 1980), and a recording cylinder (Crist Instruments, Damascus, MD) that provided access to MT. During experiments, the animals sat in a primate chair with their heads restrained, facing a CRT display. The animals performed a fixation or discrimination task for liquid rewards while visual stimuli were presented within the receptive field of a single MT neuron. All surgical and behavioral procedures conformed to guidelines established by the U.S. Department of Health and Human Services (National Institutes of Health) in the

*Guide for the Care and Use of Laboratory Animals*(1996).

^{2};

*xy*= (0.2813, 0.2941), correlated color temperature = 4036 K).

*b*in Equation 5; see Figure 8).

^{3}, 7.8 × 10

^{3}, and 1.9 × 10

^{3}absorptions per second, respectively. For a 1-cycle/deg test stimulus, with complete certainty in the macular pigment density, a nominal S-cone contrast at the display of 64% produces a retinal S-cone contrast of 22% and L- and M-cone contrasts on the order of 0.4%. Hence, at nominal S-cone contrasts that produce robust responses, for example, 10%, chromatic aberration error introduces less than 0.07% contrast of unwanted signals in the L- and M-cone mosaics. Given that, in our experiments, we used spatial frequencies at or below 1 cycle/deg and that most of the time the preferred spatial frequency was 0.3–0.5 cycle/deg, we can eliminate chromatic aberration as a potential source of error in our calibration.

Assumed pigment density | Retinal contrast (%) | ||
---|---|---|---|

L cone | M cone | S cone | |

None | 0.4 | 0.4 | 22.0 |

0.09 | 0.6 | 1.2 | 22.4 |

0.18 | 1.1 | 1.7 | 22.1 |

0.36 | 1.7 | 3.2 | 22.0 |

^{2}).

*x*is either an L,M- or an S-cone contrast,

*c*is a saturation constant, and

*m*is a proportionality constant. Maximum response and contrast threshold were extracted from the fitted equation. Maximum response was given by the response magnitude at maximum contrast; threshold was taken as the contrast at which the neural response was halfway between the spontaneous firing rate and the maximum response.

L,M cone | S cone | |
---|---|---|

Contrast threshold | 2.43 ± 1.35 | 1.04 ± 0.24 |

Response magnitude | 0.36 ± 0.34 | 1.00 ± 0.54 |

^{2}whereas the monitor alone is 132 cd/m

^{2}. Hence, the added yellow light reduced any rod contrast by a factor of two, and this contrast reduction should have been matched by a sensitivity loss; however, it was not. Second, from simulations, we estimate the unwanted rod contrast caused by the S-cone-isolating stimuli to be from 1% to 6%. Behavioral increment thresholds suggest a contrast threshold of 10% over most of the scotopic range, rising to more than 100% at background levels comparable with those of the yellow light control (Aguilar & Stiles, 1954). If behavioral thresholds measure sensitivity limited by the rods, then the unwanted rod contrast is too low to influence the MT neural responses.

*x*and

*y*axes), but not along the diagonals.

*x*is the contrast and

*p*is the proportion correct (range = 0–1). On a logarithmic x-axis, the coefficients α and β determine the horizontal offset and slope of the curve, respectively (Treutwein, 1995). We defined contrast threshold as the contrast at which the monkey performed at 75% correct (the midpoint between chance and perfect performance).

*s*is the saturating level of performance (units, proportion correct; range = 0–1) and

*z*is chance performance level. In 2AFC paradigms,

*z*= 0.5. When

*s*= 1, Equation 8 reduces to Equation 7. For a few data sets, a better fit was obtained using Equation 8 (with

*z*= 0.5 and

*s*allowed to vary), as revealed by a better chi-squared measure of goodness-of-fit. This reflected the fact that in those experiments, maximal performance was somewhat less than 1. In these cases, Equation 8 was used to obtain estimates of α and β.

*neurometric function*, to compute the sensitivity of individual neurons (Britten et al., 1992). In brief, for each stimulus type and contrast, we assessed the ability of an ideal observer to distinguish between preferred and null motion by calculating the receiver operating characteristic (ROC). The ROC curve was constructed by plotting the proportion of preferred motion trials for which the average neural firing rate falls above a criterion value, against the same quantity calculated for null motion trials, for each possible criterion from zero to the maximum observed firing rate. The area under the ROC curve represents the percent of trials in which an ideal observer could correctly identify the direction of motion at this contrast (Green & Swets, 1966). The resulting data set—proportion correct as a function of stimulus contrast—was then fit with a sigmoid function, and the threshold and slope of this function was extracted using the same method as for the psychophysical data (Figure 11B).

*choice probability*(Britten, Newsome, Shadlen, Celebrini, & Movshon, 1996).

*SD*= 0.08), which indicates that MT neural responses to S-cone contrast are, on average, 10% of that to L,M-cone contrast. It may be noted that the response to the low S-cone contrast was sometimes quite small and that the profile of responses to the range of S-cone contrasts was not necessarily linear (e.g., Figure 4C). If small responses represent a neural response that has not yet crossed a threshold of activation, including these values may skew our estimates of the average response to cone contrast. We therefore also computed our response-per-RMS ratios using only responses that exceeded 10% of the spontaneous firing rate. The overall result was very similar (mean = 0.11,

*SD*= 0.07).

*b*is the critical parameter for evaluating whether the cone interactions show any opponency. This is necessarily the case, because in the model, the only free parameter that can mediate any opponent interaction between cone signals before rectification is

*b*. Asymmetries in the neural response profile, which are evident in many of the neurons we recorded (e.g., Figures 2 and 5), must arise from interactions prior to rectification. Figure 8A shows that for the majority of cells,

*b*is greater than zero, indicating summation of S- and L,M-cone signals. However, parameter

*b*is negative for a significant minority of cells, indicating subtraction (i.e., color opponency). The 95% confidence interval for each estimate of

*b*is shown in Figure 8B; few of the confidence intervals include zero. Thus, our sample of MT cells contained units that reliably summed as well as cells that reliably differenced S- and L,M-cone signals.

*b*measures the relative contribution of the L,M- and S-cone contrasts to the response component represented by the first term. The distribution of the magnitude of

*b*is plotted in Figure 8C separately for positive and negative values. Regardless of whether the cone inputs add or subtract, the magnitude of

*b*centers near a value of 10 (10.26 for additive signals,

*b*> 0; 13.55 for differencing signals,

*b*< 0).

*c*

_{1}and

*c*

_{2}(Figure 9A and 9B) were quite broad, the confidence intervals were, in general, relatively narrow. Each parameter controls the curvature of the saturating contrast response function for each cone signal, and our neurons exhibited a wide range of profiles, both in terms of degree of curvature, concavity or convexity, and how quickly they saturated. This led to a large range of values for the parameters

*c*

_{1}and

*c*

_{2}.

*b*is relatively small, the S-cone contribution is roughly

*m*

_{1}; the L,M-cone contribution is given by

*m*

_{1}(&z.sfnc;

*b*&z.sfnc;) +

*m*

_{2}. (Note that this is only an approximation; Equation 5 does not yield a simple term that allows us to compare

*s*and

*k*weightings in terms of all five coefficients.) The ratio of these two quantities,

*m*

_{1}/(

*m*

_{1}(&z.sfnc;

*b*&z.sfnc;) +

*m*

_{2}), is shown for each cell in Figure 10A. The distribution centers on a mean of 0.076, which, interestingly, is comparable to the ratio of S-cone to L- and M-cone photoreceptors in the retina (Curcio et al., 1991; Wandell, 1995).

*m*

_{1}/(

*m*

_{1}+

*m*

_{2}) against the parameter

*b*. The parameter

*b*tells us the relative weight of S to L,M within the term in which they interact, and

*m*

_{1}/(

*m*

_{1}+

*m*

_{2}) measures the relative weight of that term with respect to the whole response. We then note where the different color channels, measured at the level of the retinal ganglion cells and the lateral geniculate nucleus (LGN), fall within this plot. Koniocellular LGN neurons have opposed S- and L,M-cone inputs, weighted roughly equally (

*b*approximately −1), and little if any luminance input (

*m*

_{1}/(

*m*

_{1}+

*m*

_{2}) is large). Hence, koniocellular neurons fall in the upper center part of the parameter space, indicated by a small gray ellipse. We found no cells in the region of the space that would correspond most closely to koniocellular responses. Classically described luminance (magnocellular) neurons, with no S-cone signal (

*m*

_{1}approximately 0), would fall near a line at the bottom of this space. (The value of

*b*is undefined, so all values of

*b*are included.) Magnocellular neurons with a small S-cone signal, as described by Chatterjee and Callaway (2002), fall in the region where

*b*is approximately 10, given the relative weight of cone signals they observed within those cells (large gray ellipse). Few of our neurons fall within the “classical” luminance region, but many fall within the ellipse that describes the magnocellular data of Chatterjee and Callaway (2002).

*r*= .62,

*p*< .001; slope,

*r*= .32,

*p*< .05). Thus, we see no evidence for separate populations of cells with high sensitivity to only luminance or chromatic stimuli.

*z*scores (see Methods). Figure 12C displays the CP distributions for all neurons in this data set for S-cone stimuli, and Figure 12D displays the same distribution for luminance stimuli. The average CP for S-cone stimuli was .525, and this mean was significantly higher than .5 (t-test,

*p*< .001). Individual neurons for which CP was significantly higher than .50 are shaded (permutation test,

*p*< .5).

*p*< .001). Britten et al. (1996) obtained a similar average CP (mean = .548) using a different luminance stimulus. Most importantly for our purposes here, the means of the distributions for S-cone and luminance Gabors were not statistically different (

*p*> .05).

*p*> .05).