Enhanced luminance sensitivity on color and luminance pedestals: Threshold measurements and a model of parvocellular luminance processing

Abstract

Psychophysical interactions between chromatic and achromatic stimuli may inform our understanding of the cortical processing of signals of parvocellular origin, which carry both luminance and color information. We measured observers’ sensitivity in discriminating the luminance of circular patch stimuli with a range of baseline (“pedestal”) luminance and chromaticity. Pedestal stimuli were defined along vectors in cone-contrast space in a plane spanned by the red-green cone-opponent (L-M) and achromatic (L + M + S) axes. For a range of pedestal directions and intensities within this plane, we measured thresholds for discriminating increments from decrements along the achromatic axis. Low-contrast pedestals lowered luminance thresholds for every pedestal type. Thresholds began to increase with higher pedestal contrasts, forming a “dipper”-shaped function. Dipper functions varied systematically with pedestal chromaticity: Compared to the achromatic case, chromatic pedestals were effective at lower contrast. We suggest that the enhancement of luminance sensitivity caused by both achromatic and chromatic pedestals stems from a single mechanism, which is maximally sensitive to chromatic stimuli. We fit our data with a computational model of such a mechanism, in which luminance is computed from the rectified output of cone-opponent mechanisms similar to parvocellular neurons.

Introduction

The extent to which luminance and color are processed independently or conjointly early in the visual system is an important open question. Psychophysical experiments that reveal interactions between chromatic and achromatic stimuli provide unique evidence on this subject and can constrain models of how visual cortex extracts luminance and color information from its thalamic inputs. 

The detectability of simple isolated stimuli of any chromaticity can be modeled as resulting from three separate detection mechanisms, two chromatic and one luminance-based. These three mechanisms appear to act independently at detection threshold, showing minimal subthreshold summation (Chaparro, Stromeyer, Kronauer, & Eskew, 1994; Cole, Hine, & McIlhagga, 1993; Eskew, McLellan, Giulianini, Gegenfurtner, & Sharpe, 1999; Mullen, Cropper, & Losada, 1997; Mullen & Sankeralli, 1998; Sankeralli & Mullen, 1996; Stromeyer, Cole, & Kronauer, 1985). Above threshold, however, this independence no longer holds. Masking experiments determining the effect of a suprathreshold mask on the detection of a test stimulus have revealed complex interactions in which the detectability of luminance or color stimuli may be modulated by the presence of a pedestal of the opposite type. High-contrast pedestals of any type tend to reduce the detectability of any test stimulus, an effect attributable to gain-control mechanisms processing both color and luminance contrast (Chen, Foley, & Brainard, 2000b; Mullen & Losada, 1994; Switkes, Bradley, & DeValois, 1988). At lower contrasts, a range of studies have shown that achromatic pedestals enhance chromatic sensitivity, using sinusoidal gratings (Chen, Foley, & Brainard, 2000a; Mullen & Losada, 1994; Switkes et al., 1988), circular patch stimuli (Chaparro et al., 1994; Cole, Stromeyer, & Kronauer, 1990), and other spatial configurations (Hilz & Cavonius, 1970; Hilz, Huppmann, & Cavonius, 1974; Montag, 1997). 

Tests of luminance sensitivity on chromatic pedestals, however, have shown less consistent results. Cole, Stromeyer, and Kronauer (1990), using small spot stimuli, showed significant enhancement in the detectability of luminance increments when presented on red or green chromatic pedestals. Several studies using grating stimuli have shown no facilitation in this direction (Chen et al., 2000a; DeValois & Switkes, 1983; Switkes et al., 1988). The results of Mullen and Losada (1994), however, suggest that a form of facilitation does occur in this stimulus configuration, but its measurement depends on whether the experimental design allows observers to judge the brightness of individual subregions of the grating or forces them to integrate over subregions to detect spatial luminance contrast. These inconsistencies warrant further investigation but may show that the pathway through which luminance supports the perception of surface brightness is more closely linked to chromatic processes, compared to a more “colorblind” pathway concerned only with spatial luminance contrast. 

Color-luminance interactions are of particular interest in our effort to map psychophysical observations onto early neural mechanisms, since chromatic signals are multiplexed with achromatic in the subcortical and early cortical stages of visual processing. Parvocellular neurons of the visual thalamus respond to both chromatic and achromatic stimuli (Creutzfeldt, Lee, & Elepfandt, 1979; Derrington, Krauskopf, & Lennie, 1984; Hicks, Lee, & Vidyasagar, 1983; Lee, Virsu, & Elepfandt, 1983; Wiesel & Hubel, 1966). These cells comprise the majority of the feedforward input to visual cortex (Perry, Oehler, & Cowey, 1984), yet our understanding of the cortical processes using this input to support luminance and color vision remains incomplete. This question has inspired computational models for decades. Early work showed that linear filtering and recombination of multiplexed color-luminance signals could recover independent achromatic and chromatic channels, with spatial contrast sensitivities resembling those of human observers (Billock, 1995; DeValois & DeValois, 1993; Ingling & Martinez-Uriegas, 1983; Kingdom & Mullen, 1995). More recent work, quantitatively linked to experimental data, suggests that luminance and chromatic channels are both influenced by early, nonlinear transformations of parvocellular signals (Stockman, Henning, & Rider, 2017; Stockman, Petrova, & Henning, 2014). 

In this article, we aim to characterize how the mechanisms of luminance processing are influenced by chromatic context. We measured observers’ sensitivity to luminance increments and decrements added to chromatic patch stimuli, similarly to the experiment of Cole et al. (1990). We expanded this approach to include a wider range of pedestals with mixed luminance and red-green content in order to measure the chromatic tuning of pedestal effects. Luminance sensitivity was enhanced by every pedestal we tested, and the form of this enhancement varied systematically with pedestal chromaticity. We find that a model in which luminance is computed from a nonlinear combination of parvocellular signals closely replicates our psychophysical results. 

Methods

Participants

Three observers (two female) participated in the experiment: one volunteer and the two authors. All observers were experienced with psychophysical methods and aware of the hypotheses being tested. All had normal or corrected-to-normal acuity and had normal color vision as assessed with Ishihara pseudoisochromatic plates and the Farnsworth-Munsell 100-Hue Test. All procedures conformed to the Declaration of Helsinki and were approved by McGill University's institutional review board. 

Visual stimuli

Stimuli were circular patches presented foveally in a 4-degree aperture. Patch intensity was constant in the central 3.2 degrees, falling smoothly to zero in the outer 0.4 degrees with a raised-cosine profile. The color of each patch was defined in the three-dimensional space of cone contrast, as a triplet {c_L c_M c_S}, with \({c_L} = \;( {L - \bar L} )/\bar L\) representing the fractional change in long-wavelength-sensitive cone excitation from baseline (\(\bar L\)). The intensity of a stimulus was defined as the vector length \(c\; = \;\sqrt {c_L^2 + c_M^2 + c_S^2} \), with units of cone contrast. 

As illustrated in Figure 1, we used a pedestal + increment paradigm to measure sensitivity to differences in luminance about pedestals of various luminance and chromaticity. All stimuli were confined to the plane spanned by the red-green axis lms = {1 –1 0} and the achromatic axis lms = {1 1 1}. We refer to pedestal “direction” as the angle in this plane, with 0 and 90 degrees representing the red-green and achromatic axes, respectively. The red-green direction is optimal for stimulating the L/M cone opponent mechanism. This is not an isoluminant direction, since we are interested in the combination of color and luminance contrasts. The achromatic direction is meant to stimulate only luminance mechanisms and silence both L/M cone opponent and S cone opponent chromatic mechanisms. For each of eight directions (–67.5, –45, …, 90 degrees), we separately tested pedestals with positive and negative polarity and a range of contrasts (0.1875% to 96% cone contrast, in one-octave steps). Specifically, for a pedestal direction defined by the vector p  = {l_p m_p s_p}  of unit length, two stimuli were constructed by the addition and subtraction of a test increment in the achromatic direction a = {1 1 1}/\(\sqrt 3 \). The two stimuli to be discriminated were thus defined as \({c_p}p \pm \frac{{{c_I}}}{2}\;a\), containing a luminance increment + decrement of total contrast c_I about a pedestal of contrast c_p. In a control experiment, we also tested a subset of pedestals using only luminance increments as test stimuli. 

Figure 1.

 

(A) Stimuli were defined in cone-contrast space, in the plane spanned by the L-M (red-green) and L + M + S (achromatic) axes. (B) Pedestals were defined along eight axes in the red-green/achromatic plane, with a range of contrasts. For each pedestal, we measured thresholds for discriminating an increment from a decrement added to the pedestal, along the achromatic axis. (C) Increment/decrement pairs were presented together in a two-interval trial. The observer reported which interval appeared brighter or more white. Stimulus onset was smooth with a raised-cosine envelope; each stimulus was followed by a broadband noise stimulus to mask its offset.

Temporally, contrast increased over 100 ms with a raised-cosine ramp, then remained constant for 200 ms. Stimuli were immediately followed by 100 ms of a noise mask, described below. We used a two-interval design; the second interval began 300 ms after the offset of the first noise mask. To reduce uncertainty in the location of low-visibility stimuli, we presented a thin, low-contrast ring prior to each trial and removed it 300 ms before the onset of the first interval. The ring was 4 degrees in diameter, 0.065 degrees (3 pixels) thick, and 3% positive contrast. 

Poststimulus mask

Each stimulus presentation was followed immediately by 100 ms of broadband, chromoluminance noise. The purpose of this mask was only to eliminate after-images, forcing observers to make judgments based on the percept driven by stimulus onset. We do not expect the noise to have masked this onset response, and the role of noise masking does not factor into our interpretation of experimental results. Each sample of noise was created by frequency-domain filtering: An array of random values was drawn from a Gaussian distribution with zero mean and unit variance to create a space-time “movie” of white noise. A three-dimensional discrete Fourier transform and cartesian-to-polar change of variables mapped this to the space of orientation, spatial frequency, and temporal frequency. This was multiplied by the product of two frequency-domain filters (Butterworth, order 10): a lowpass spatial-frequency filter, with a cutoff frequency of 2 cycles/deg, and a bandpass temporal-frequency filter with passband between 1 and 6 Hz. The resulting filter was isotropic in orientation. An inverse Fourier transform converted the sample back to a space-time movie. For each sample, two noise movies were generated to create independent achromatic and red-green noise components, which were then summed. Specifically, one noise movie, a, was converted to cone-contrast space by the mapping \(l\; = \;m\; = \;s\; = \;a/\sqrt 3 \), and a second movie, rg, by the mapping \(l\; = \;rg/\sqrt 2 \), \(m\; = - rg/\sqrt 2 \;,\) s  =  0. These samples were separately scaled to have postfilter root-mean-square cone contrasts of 15% and 5%, respectively, to compensate for the different sensitivity of luminance and chromatic mechanisms, then summed. Noise masks were presented with the same spatial contrast envelope as patch stimuli. For programmatic reasons, we generated a single noise movie of a 1-s duration for each block and presented a randomly selected 100-ms portion of this on each trial. This windowing altered the frequency content of the noise slightly but did not introduce a large zero-frequency component: the space-time mean chromaticity of shortened noise samples differed from the background by less than 1% cone contrast. 

Threshold measurements

We used a two-interval forced-choice (2IFC) procedure to measure luminance discrimination thresholds. On each trial, a luminance-increment and a luminance-decrement stimulus were presented sequentially in random order, and the observer reported which interval contained the increment (which appeared brighter, or “more white”) via a button press. Auditory feedback was given. One block of trials tested a single pedestal direction and absolute pedestal contrast, with positive and negative pedestal polarities randomly interleaved. The interleaving of two pedestal polarities within a block ensured that the mean luminance and chromaticity of the display were constant across conditions, reducing receptor-level adaptation effects. We used an adaptive staircase procedure (two-down, one-up) to choose the increment contrast level tested on each trial. Two independent staircases per condition were randomly interleaved and terminated after five reversals. At least six staircases were run for every condition. Trials from all staircases were combined and fit with a logistic psychometric function, from which we obtained a threshold contrast yielding 75% correct. In separate blocks, we also measured thresholds for detection of the pedestal stimuli, using a 2IFC procedure in which the pedestal stimulus appeared in one interval and no stimulus appeared in the other. 

Control experiment using a single stimulus polarity

Our bipolar test stimulus, combined with mixed-polarity pedestals, could lead to a form of uncertainty not found in other designs. To address this, we performed a control experiment with the more common method of testing only luminance increments, with observers still reporting the brighter alternative. Here we blocked trials by pedestal polarity in addition to contrast, so that a single pedestal was used for all trials in a block. The two methods led to very similar results, as shown below. 

Apparatus

Stimuli were generated in MATLAB (The MathWorks, Natick, MA, USA) using the Psychophysics Toolbox (Brainard, 1997; Kleiner, Brainard, & Pelli, 2007; Pelli, 1997) and displayed on a CRT monitor (Sony Triniton, Sony Electronics, New York, NY) with a mean luminance of 53 cd/m² and chromaticity xy = [0.29, 0.30]. Observers viewed the screen from a distance of 80 cm, at which it subtended 22 × 27 degrees of visual angle. A Bits# visual stimulus generator (Cambridge Research Systems, Kent, UK) was used to control the amplitude of each color channel with 14-bit precision. Nonlinearity in the output of each color channel was characterized using a SpectroCAL spectroradiometer (Cambridge Research Systems) and corrected in software. The measured emission spectra of the monitor phosphors were integrated with psychophysically derived cone fundamentals (Smith & Pokorny, 1975) to create a linear transformation specifying the RGB values required to elicit any target triplet of cone excitation levels. 

Results

Luminance discrimination thresholds

We measured observers’ ability to discriminate between an increment and decrement in the luminance of circular patch stimuli that varied in baseline (“pedestal”) luminance and chromaticity. On purely achromatic pedestals, luminance-discrimination thresholds varied with pedestal luminance with a typical “dipper” shape: Thresholds were reduced on low-contrast pedestals, including those slightly below detection threshold, and elevated at higher pedestal levels (Legge & Foley, 1980; Legge & Kersten, 1983; Nachmias & Sansbury, 1974). Figure 2A shows an example of this in the threshold-versus-contrast (TvC) curve for positive-luminance pedestals, using average data from three observers. 

Figure 2.

 

Luminance discrimination thresholds are plotted with respect to pedestal contrast, both in units of absolute cone contrast. The horizontal line in each plot represents the luminance threshold obtained with no pedestal. Observer-averaged results are shown for three example pedestal directions in cone-contrast space: (A) achromatic white (L + M + S), (B) the intermediate axes angled 45 degrees in the red-green/achromatic plane, and (C) red (L-M). The short vertical bars above each curve represent thresholds for detecting the pedestal stimuli. Chromatic pedestals influenced luminance sensitivity at lower absolute pedestal contrast compared to achromatic pedestals. Solid curves show model fits, described in the text.

Luminance thresholds were also reduced on chromatic pedestals, as reported previously (Cole et al., 1990; Gowdy, Stromeyer, & Kronauer, 1999), and on mixed pedestals containing luminance and chromatic components. The shape of TvC functions varied systematically with pedestal chromaticity: Compared to the achromatic case, pedestals with chromatic content were effective at lower pedestal contrast. The minimum red pedestal (L-M) contrast needed to reduce threshold was roughly three times lower than in the achromatic case. The point at which thresholds began to increase (the upturn of the dipper) was also at lower pedestal contrast. This can be seen in the example data of Figure 2. Figure 2C shows a TvC curve obtained with red pedestals defined along the vector L-M in cone-contrast space. Figure 2B shows results for mixed pedestals, on a vector angled 45 degrees in the achromatic/red-green plane (see Figure 1). 

Thresholds for detecting pedestal stimuli presented in isolation (represented by vertical bars in Figure 2) were significantly lower for chromatic versus achromatic pedestals, as expected (Mullen, 1985). The effectiveness of a pedestal to enhance luminance sensitivity therefore changed in the same direction as its visibility, but to a lesser extent, such that facilitation by chromatic pedestals occurred at or above detection threshold, unlike the subthreshold facilitation caused by achromatic pedestals. 

Our full data set is represented in Figure 3, each panel showing results from one pedestal direction and one observer, with pedestals of opposite polarity plotted together. The pattern described above was found for each observer, across the full range of pedestals tested. Pedestals with chromatic content were visible at lower contrast and affected luminance sensitivity at lower contrast. Pedestals along the same cone-contrast axis but of opposite polarity had nearly identical effects in most cases, as reported previously (Cole et al., 1990). An exception is found in the data of Observer 2, in which luminance thresholds were higher on light-red pedestals than their dark-green counterparts (Figure 3, second column, rows 2–4). Interestingly, this asymmetry is not seen in the achromatic or the most strongly chromatic (L-M) conditions, implying it cannot be explained by a global white-black or red-green asymmetry. 

Figure 3.

 

Threshold-versus-contrast functions for luminance discrimination are plotted as in Figure 2, separately for three observers (in columns) and for the eight axes in cone-contrast space along which pedestals were tested (rows). Pedestal axes are referred to by their angle in the red-green/achromatic plane, with L-M defined as 0 degrees and L + M + S as 90 degrees. Opposite-polarity pedestals on the same cone-contrast axis are plotted together. Horizontal bars represent luminance-discrimination thresholds obtained with no pedestal. Vertical ticks above each curve represent thresholds for detection of pedestal stimuli. Solid curves show model fits, described in the text.

Figure 4.

 

Model luminance mechanism: The rectified outputs of partially cone-opponent units, similar to parvocellular neurons, are combined to create luminance-increment and decrement channels. Each luminance channel is separately processed by a second-stage nonlinearity consisting of an expansive transducer coupled with divisive gain control. A measure of total stimulus energy for purposes of gain control is computed by pooling over parvo channels, each raised to an exponent. See the main text for mathematical details.

Overall, the dipper functions of Figure 3 appear more similar than different, with significant luminance facilitation occurring regardless of pedestal chromaticity. Our interpretation of this consistency is that all pedestals may influence luminance processing through a common mechanism, distinct from that determining pedestal visibility, but more sensitive to chromatic than achromatic stimuli. We next consider a candidate mechanism meeting these criteria. 

Model of a parvocellular luminance channel

Our observations are closely replicated by a model in which luminance is computed from the rectified output of partially cone-opponent mechanisms, functionally similar to subcortical neurons of the parvocellular pathway (see Figure 4). In the central visual field, these neurons receive excitatory input from one cone type (L or M) and inhibitory input from both L and M cones via the receptive field surround (Boycott, Hopkins, & Sperling, 1987; Calkins, Schein, Tsukamoto, & Sterling, 1994; Kolb & Dekorver, 1991). Both L- and M-center neurons may be of the ON or OFF cell type, signaling increments or decrements in cone excitation, respectively (Dacey, 2000). The degree of cone opponency imparted by a single neuron's surround depends on the spatial profile of the stimulus relative to that of its receptive field, as well as the cone weighting and overall strength of surround suppression. Our experiment, using stimuli of a single spatial form, does not allow modeling of these factors separately. We instead describe the combined output of a population of cells of one type as proportional to a weighted subtraction of L and M cone-contrast signals. For the L-ON channel, this is expressed as  

\begin{eqnarray*}{r_{L\;On}}\;\sim\;{c_L} - w \cdot {c_M},\end{eqnarray*}

with the relative weighting parameter w representing the effective cone opponency for our particular stimulus configuration. 

The outputs of the first stage of the model are rectified, allowing only positive signals to pass to later stages. The rectification operator, ⌊x⌋ =  max(0, x), is applied to each cell-type channel separately. This results in four signals providing the input for later processing:  

\begin{equation*}\begin{array}{@{}lcl@{}} {r_{L\;On}}\; &=& \;k \cdot \left\lfloor {{c_L} - w \cdot {c_M}} \right\rfloor \;\\[-6pt]\\ {r_{L\;Off}}\; &=& \;k \cdot \left\lfloor { - ({c_L} - w \cdot {c_M})} \right\rfloor \;\\[-6pt]\\ {r_{M\;On}}\; &=& \;k \cdot \left\lfloor {{c_M} - w \cdot {c_L}} \right\rfloor \;\\[-6pt]\\ {r_{M\;Off}}\; &=& \;k \cdot \left\lfloor {\; - \left( {{c_M} - w \cdot {c_L}} \right)} \right\rfloor \;. \end{array}\end{equation*}

 

The second stage of the model computes luminance by summing parvo inputs of the same sign, with L-ON and M-ON contributing to a channel signaling luminance increments and OFF cells to a luminance-decrement channel. Each channel is then subject to a nonlinear transformation similar to that found in many previous models of achromatic contrast processing: nonlinear amplification (raising signals to an exponent greater than 1) coupled with divisive gain control (Legge & Foley, 1980; Meese & Summers, 2007). We compute a measure of overall stimulus energy, for purposes of gain control, by pooling over the four parvo-like input signals, each raised to an exponent:  

\begin{equation*}E\; = \;h \cdot \left( {r_{L\;on}^q\; + \;r_{L\;off}^q\; + \;r_{M\;on}^q\; + \;r_{M\;off}^q} \right).\end{equation*}

 

An alternative structure, pooling responses of “second-stage” luminance and chromatic channels, could be used to compute a similar measure (Chen et al., 2000b). In preliminary testing, we found that these two gain-control models led to similar predictions, as expected from two computational components designed to compute the same thing (total stimulus energy). Our masking data do not support a meaningful model comparison in this case. We favor the formulation given above for its simplicity and for eliminating the need to model an explicitly chromatic cortical channel (one differencing L- and M-center parvo cells). Interestingly, without a second-stage red-green mechanism, the model predicts stronger masking for chromatic than achromatic pedestals (consistent with our data), only on the basis of the chromatic tuning of parvo inputs. 

The outputs of luminance-increment and decrement channels are given by  

\begin{equation*}{r_{Lum\;On}}\; = \;\frac{{{{\left( {{r_{L\;On}}\; + \;{r_{M\;On}}} \right)}^p}}}{{1 + \;E\;}}\end{equation*}

and  

\begin{equation*}{r_{Lum\;Off}}\; = \;\frac{{{{\left( {{r_{L\;Off}}\; + \;{r_{M\;Off}}} \right)}^p}}}{{1 + \;E\;}},\end{equation*}

and these channels are combined to create a single, signed output representing stimulus luminance:  

\begin{equation*}L\; = \;{r_{Lum\;On}} - \;{r_{Lum\;Off}}.\end{equation*}

 

This signal serves as the decision variable for luminance discrimination: From any pair of stimuli presented in one trial, the model observer chooses the stimulus leading to the larger output. In practice, we did not simulate or model single trials. Instead, we define the discrimination threshold of the model as the minimum amplitude increment/decrement leading to output values differing by a value of 1. This is equivalent to assuming that constant-variance, Gaussian noise is added at the output stage, and the units of model output are multiples of the noise standard deviation. Criterion performance is then achieved when the responses to two stimuli differ with a d′ of 1. 

We fit the model separately to each observer's data and also to observer-averaged data by minimizing the sum of squared differences between predicted and observed log thresholds. The model fits the data well (solid black curves in Figures 2 and 3), with only five parameters, and replicates the main experimental observations described above. Fitted parameters are given in Table 1. The model predicts identical effects from pedestals of opposite sign, not accounting for the (infrequent) asymmetries in the data of Observer 2. Other places where data deviate systematically from the model are mostly at higher pedestal intensity, where model predictions are determined by its gain-control stage. As mentioned above, the structure of this stage of the model is less constrained by these data. This gain-control computation is also not our primary interest. The purpose of our modeling analysis is instead to show that the facilitatory effects of both chromatic and achromatic pedestals can be explained by a model with this structure. 

Table 1.

 

Fitted values of the five model parameters are given for each observer separately, as well as values from fitting observer-average data.

 

The numerical value of model parameters is of less interest than the model structure, yet to evaluate the model overall, it is useful to know how the quality of fit depends on each parameter. We performed an additional fitting analysis in which each parameter in turn was fixed at 1 in a range of values. For each fixed value, the fit was repeated with all other parameters free to vary, and the resulting model error (summed squared difference between predicted and observed log thresholds) was compared to that of the original fit. Figure 5 plots this relative error as a ratio for Observer 1 only. From each error curve, we computed the range of values for which the fit error was within 10% of the best fit. The lower row of Figure 5 shows this range for each observer and each parameter. Because the exponent parameters p and q tend to have equal and opposite effect (in numerator and denominator), it is preferable to use the difference between them (p – q) as a parameter rather than q itself. 

Figure 5.

 

We refit the data with one parameter fixed at a single value and other varying freely. A range of fixed values was tested for each parameter, and the resulting model errors were compared to the best-fit error as a ratio. The top row shows this error ratio for Observer 1, for each parameter. The gray bars in the second row represent the range of fixed parameter values yielding fit errors within 10% of the best fit. This range is shown for Observers 2 and 3 as well.

This analysis shows that the cone-weighting parameter w is constrained to the range of 0.5 to 0.8, representing strong but incomplete L-M opponency. The scaling parameter k is of the right order of magnitude to convert between units of contrast and those of d′ (the model output). The values of exponential parameters p and q are in the range commonly found in similar models, between 2 and 3, with q slightly less than p (Foley, 1994; Gheiratmand, Meese, & Mullen, 2013; Meese, 2004). 

Chromatic tuning of color-luminance facilitation

We summarized the chromatic tuning of luminance enhancement by computing, for each pedestal type, the minimum pedestal intensity (in cone contrast units) required to lower luminance threshold by 20%. The numerical choice for this criterion level of facilitation is arbitrary and does not affect our conclusion. This “threshold for facilitation”, derived from model fits, is plotted in Figure 6, along with pedestal-detection thresholds, measured directly (not modeled). As described above and in previous studies (Cole et al., 1990; Mullen & Losada, 1994), achromatic pedestals facilitate just below threshold and chromatic pedestals only above their threshold. The size of this difference varied across our three observers, however. For Observer 1, detection and facilitation thresholds were more closely matched, differing by a factor of 2 or less. For Observers 2 and 3, this difference was larger (up to a factor of 4). The small number of observers does not allow a strong conclusion, but this pattern of individual differences is consistent with proposal that the mechanism responsible for detection of color stimuli is distinct from that through which these stimuli can influence luminance processing. We discuss this proposal further below. 

Figure 6.

 

We used model fits to compute the minimum pedestal intensity needed to reduce luminance thresholds by 20%, separately for each pedestal. This “threshold for facilitation” is plotted with respect to pedestal angle, along with thresholds for detecting pedestal stimuli.

The chromatic tuning of the pedestal effect (in the L-M direction) is indirectly caused by the tuning of the model parvo units, which themselves show less complete cone opponency (e.g., L – wM, w < 1). The connection between the two relies on the fact that most pedestals influence multiple parvo cell types. An L-M pedestal, for example, will excite both L-ON and M-OFF cells, thereby driving both the ON and OFF luminance pathways into regions of steeper slope, increasing sensitivity to any further change. The most effective pedestal is therefore one that drives the population as a whole furthest away from baseline; due to the symmetry across cone type and polarity, this is the L-M case. 

We have not modeled a red-green mechanism parallel to the luminance mechanism, and the model predicts only achromatic luminance thresholds. Extending the model in this direction would require a description of how parvo inputs are differently combined to extract a red-green signal. As in previous models (Billock, 1995; DeValois & DeValois, 1993; Ingling & Martinez-Uriegas, 1983; Kingdom & Mullen, 1995), this would logically entail differencing, rather than summing, L- and M-center cells of the same polarity. In this case, as above, the imbalanced cone weighting seen in input units would combine to impart a balanced chromatic tuning in the L-M direction, consistent with the known tuning of psychophysically measured detection mechanisms (e.g., Chaparro et al., 1994). 

Validating with luminance-increment thresholds

Our method of presenting an increment and decrement around a common pedestal, within a single trial, differs from the more common paradigm in which a fixed pedestal is presented as one alternative and a pedestal + increment as another, the observer reporting which alternative contains the increment. In principle, the two methods measure the same thing: which pairs of contrasts are visibly different from each other and which are not. However, our method of using two-polarity tests while also interleaving pedestals of opposite polarity could possibly have led to greater uncertainty for the observer, especially when one or both stimulus intervals were below threshold. To address this, we repeated part of our experiment using positive-polarity achromatic increments as tests and without interleaving pedestals of opposite polarity. We tested achromatic and red-green (L-M) pedestals of both polarities in two observers. To reduce the adaptation effects that come with blocks of single-polarity stimuli, we tested only low-contrast pedestals (up to 3% cone contrast, which includes the site of facilitation effects). 

In the case of achromatic pedestals, results from the two methods can be compared directly. The two methods use conflicting definitions of “pedestal”: the midpoint of two discriminated stimuli in the first case, or the stimulus of the pair having lower contrast in the second. This terminology can be circumvented by analyzing the absolute contrasts of each pair of just-discriminable stimuli (the two alternatives presented at threshold). In Figure 7A, we have plotted data from both experiments in this format. Threshold data viewed this way partition the space of all contrast pairs into those that are perceptually indistinguishable (shaded gray area) and those that can be discriminated. A “dipper” shape can be recognized if the plot is rotated 45 degrees to consider just-discriminable contrast differences (distances from the diagonal) as they depend on mean contrast (proportional to projections onto the diagonal). 

Figure 7.

 

A control experiment measured thresholds for detecting positive luminance increments on fixed pedestals. The results are compared to increment/decrement discrimination thresholds from the main experiment. (A) Data from achromatic pedestals in both experiments are plotted for one observer with a second observer's data in the inset. Axes represent the contrasts (in cone-contrast units) of just-discriminable pairs, the two stimuli presented at threshold. The shaded region represents pairs of contrast expected to be indistinguishable. (B) Model fits to data from the main experiment were used to predict thresholds in the control experiment (not used in fits). Results from achromatic and red-green (L-M) pedestals are plotted for two observers. Prediction error did not show a strong dependence on pedestal type or contrast.

Achromatic-pedestal results were in good agreement between the two methods, eliminating our concern that the increment + decrement method of the main experiment introduced any peculiar uncertainty for the observers. Such a direct comparison cannot be made for chromatic pedestals: Luminance-increment thresholds on a chromatic pedestal measure discrimination about a midpoint with positive luminance contrast, as well as chromatic contrast, a condition not tested in our main experiment, in which chromatic-pedestal tests had zero mean luminance. The significance of this small luminance offset can be inferred from Figure 7A. 

We asked if model fits, derived only from our main experiment, could predict the new results of our control experiment. The results of this comparison are shown in Figure 7B. The model predicted new thresholds to within 1% cone contrast. More importantly, deviations of measurements from predictions did not appear to depend on pedestal chromaticity or contrast. Successful prediction of new measurements offers a validation of the model, as well as evidence that the patterns in the data with which we are concerned do not depend on this aspect of our experimental method. 

Discussion

We found that sensitivity in discriminating the luminance of circular patch stimuli was enhanced on chromatic, achromatic, and mixed pedestals. This facilitation was chromatically tuned, with chromatic pedestals having effect at lowest pedestal contrast. The effectiveness of a pedestal, however, was not predictable from its visibility: Chromatic pedestals facilitated only at or above their own detection threshold, unlike the subthreshold facilitation found in the achromatic case. 

Interpreting pedestal facilitation

Facilitation by a subthreshold pedestal when pedestal and test are identical is strong evidence of a low-level interaction. In this case, pedestal and test are detected through a common pathway, and the site of facilitation within this pathway must precede the site limiting detectability (Legge & Foley, 1980). This logic does not apply if pedestal and test are detected through parallel pathways. Facilitation by color pedestals occurring only above threshold does not imply interaction at a later or higher-level site, unless we assume a serial relationship between the mechanism limiting color sensitivity and the mechanism by which color influences luminance. Alternatively, if parallel luminance and color pathways share a common, color-sensitive input, we can reasonably conclude that the strength of chromatic stimuli required to affect the luminance pathway is more than enough to allow detection by a separate chromatic pathway. 

Our data do not exclude the possibility that color-luminance facilitation is mediated by a top-down mechanism, through which only suprathreshold color signals are fed back to modulate luminance processing. This could explain why chromatic pedestals only influence luminance above their own detection threshold. There is no reason to imagine a similar route for luminance-luminance facilitation, however, given the well-accepted explanation for this effect, occurring at an early expansive nonlinearity (Legge & Foley, 1980). This explanation then invokes two different mechanisms operating at different levels to explain what otherwise appears to be a single effect—similar facilitation of luminance by any chromoluminance pedestal. We consider an explanation involving a single feedforward mechanism more parsimonious. Alternative models involving higher-level color-luminance interaction should be considered, however, and tested with further experiments. It could be proposed that a chromatic pedestal supports target processing by making the combined stimulus more visible, reducing uncertainty in the target's location and extent (but see Eskew et al., 1991). This idea might be tested by manipulating spatiotemporal parameters to alter the detectability of chromatic pedestals and analyzing how the strength of pedestal facilitation varies with pedestal visibility. 

Comparison to previous experiments

Our findings are consistent with those of Cole et al. (1990), who with a similar experiment found strong facilitation of luminance increments on chromatic and luminance pedestals when pedestal and test were small spot stimuli. We have expanded this experiment with intermediate pedestals in order to measure the chromatic tuning of facilitation. Related studies using grating stimuli as pedestal and test have had mixed results. Switkes et al. (1988) found no facilitation of luminance-contrast detection by isoluminant red-green pedestals. Chen et al. (2000a) tested a wider range of mixed chromoluminance pedestals and found no luminance facilitation by any isoluminant pedestal. 

Mullen and Losada (1994) did observe facilitation by chromatic gratings, but only when the relative polarity of pedestal and test was constant across trials. Facilitation in this condition was also found by Gowdy et al. (1999). This polarity relationship affects which stimulus cues are informative for the task. When achromatic increments predictably lighten the red bars of the pedestal grating and darken the green, accurate judgments can be made based on the luminance of any single subregion of the grating. With relative polarity randomized across trials, the observer must make a more generalized judgment of spatially integrated luminance contrast. 

Further investigation is needed to define the conditions under which color does or does not influence luminance sensitivity. The various results reported thus far raise the possibility that luminance is processed by multiple pathways, with different relationships to color. Pathways devoted to processing spatial luminance contrast may be farther removed from color signals than those that support perception of surface brightness. 

Modeling parvocellular luminance computation

It has long been recognized that luminance might be computed cortically by summing the outputs of L- and M-center parvocellular neurons (Billock, 1995; DeValois & DeValois, 1993; Ingling & Martinez-Uriegas, 1983; Kingdom & Mullen, 1995; Lennie & D'Zmura, 1988). When this is modeled as a linear operation, it implies the cancellation of L and M signals of opposite sign (those carrying red-green information), thereby creating a purely achromatic luminance channel. More recent work, however, has highlighted the potential role of nonlinear processing of parvo signals prior to summation (Petrova, Henning, & Stockman, 2013; Stockman et al., 2014; Stockman et al., 2017). Our model is similar to that of Stockman et al. (2014), in which early rectification of parvocellular signals helps account for experimentally measured shifts in the perceived hue and brightness of flickering monochromatic lights. 

In our model, rectification plays the essential role of preventing linear cancellation of chromatic signals at the site of L + M summation, thus allowing chromatic pedestals to have effect at later stages in the luminance pathway. Pedestal facilitation occurs at the stage of nonlinear transduction, after L + M summation. Signals driven by a pedestal push the mechanism response into a range where this transducer has a steeper positive slope; a smaller luminance input is required to elicit a criterion change in output, so thresholds are reduced (Legge & Foley, 1980). 

Our model aims to describe a computation and not the details of its neural implementation. Several constraints are worth considering, however. Parvocellular neurons of the visual thalamus are not fully rectified and respond with firing rate increases and decreases to their preferred and antipreferred stimuli, respectively (De Valois, Abramov, & Jacobs, 1966; Derrington et al., 1984; Wiesel & Hubel, 1966). The rectification required by the model must then occur at an early stage of cortical processing. The simplest schematic for the excitatory portion of the model would then include two cortical stages. Neurons at the first stage would have functional properties similar to their thalamic inputs and would be categorized as single-opponent or “Type 1” color-sensitive cells (Wiesel & Hubel, 1966). The output of these would be summed by neurons at the second stage to compute luminance. Neurons at the second stage may be “luminance-preferring” or “color-luminance” cells, depending on the relative weight given to their L and M inputs (Johnson, Hawken, & Shapley, 2004). They also may be single-opponent luminance (“Type 3”) or double-opponent cells, depending on the spatial sampling of their inputs (Johnson, Hawken, & Shapley, 2001; Livingstone & Hubel, 1988; Shapley & Hawken, 2011). The results of our test with simple stimuli do not allow us to distinguish between these possibilities. Additional pedestal-type experiments aimed at this question would be of interest, given the connections that have been made between other perceptual color-luminance interactions and neural double opponency (Shapley, Nunez, & Gordon, 2019). 

The model does not include any cone-type specific wiring, beyond the exclusion of S cones. The L-M opponency generated at the first stage of the model is assumed to arise from receptive field surrounds with mixed L and M input, opposing receptive field centers driven by a single L or M cone (Calkins et al., 1994). Thereafter, signals are grouped by polarity (on-center cells are summed separately from off-center cells), but L- and M-center cells are treated identically. The model structure could therefore describe a system that evolved with a single cone type with sensitivity in the medium-long wavelength range (Mollon, 1989). If single-cone receptive fields had evolved in this system to allow fine spatial resolution of luminance, a subsequent split of the single cone type into L and M types would lead immediately to red-green opponency at the retinal level. As the behavior of the model shows, chromatic signals would then have significant downstream effects in a pathway concerned only with achromatic luminance. 

Modeling luminance perception with an equal weighting of L and M cone signals is a simplification, which works in this case but is likely to fail in modeling other experiments. Individual differences in the relative number of L and M cones (Roorda & Williams, 1999) are thought to underly the known differences in effective cone weighting between individuals’ luminance mechanisms (Pokorny et al., 1991; Rushton & Baker, 1964). In preliminary modeling of our data, introducing a free parameter describing L/M cone ratios did not lead to significantly better fits, but this likely reflects our choice of measurements. Some deviations of our data from model fits may be due to L/M imbalance, such as the asymmetry between light-red and dark-green pedestals noted in the data from Observer 2, which cannot be explained by asymmetry between light and dark or red and green. 

Parallel pathways to luminance perception

Neurons of the parvocellular and magnocellular layers of the LGN both respond strongly to achromatic stimuli, with different but highly overlapped spatial and temporal tuning profiles (Derrington & Lennie, 1984; Kaplan & Shapley, 1982; Levitt, Schumer, Sherman, Spear, & Movshon, 2001; Spear, Moore, Kim, Xue, & Tumosa, 1994). For many luminance-based tasks, informative signals may therefore be carried by both pathways. We understand little of how parallel luminance signals are integrated at later stages, partly due to the difficulty in dissociating the two streams psychophysically using achromatic stimuli. A stronger dissociation can be achieved with red-green chromatic stimuli, which drive strong parvocellular responses but little or no magnocellular response (Derrington et al., 1984; Wiesel & Hubel, 1966). The usefulness of this fact for the study of luminance processing depends on the extent to which the color and luminance components of parvo signals remain overlapped in cortex. If early cortical filtering separates these components completely (Billock, 1995; Ingling & Martinez-Uriegas, 1983), the resulting parvo luminance signal would be difficult to distinguish from its magnocellular counterpart. Nonlinearities early in a parvocellular-cortical pathway (Stockman et al., 2014) may make such a perfect demultiplexing of luminance and color impossible, leaving parvocellular-based luminance signals susceptible to influence by chromatic context in ways that magnocellular signals are not. Further study of color-luminance interactions may then prove a useful tool for dissociating these two streams and offer insight into the purpose of this parallel architecture. 

Acknowledgments

We thank our volunteer participant for assistance with psychophysical data collection. 

Supported by Canadian Institutes of Health Research (CIHR) grant 153277 to Kathy T. Mullen. 

Commercial relationships: none. 

Corresponding author: Kathy T. Mullen. 

Email: [e-mail]. 

Address: McGill Vision Research, Department of Ophthalmology & Visual Sciences, Montreal, Quebec, Canada.