The temporal characteristics of the early and late stages of the L- and M-cone pathways that signal color

Abstract

Abstract

Abstract:

Abstract  Flickering long-wavelength light appears more yellow than steady light of the same average intensity. The hue change is consistent with distortion of the visual signal at some nonlinear site (or sites) that produces temporal components not present in the original stimulus (known as distortion products). We extracted the temporal attenuation characteristics of the early (prenonlinearity) and late (post-nonlinearity) filter stages in the L- and M-cone chromatic pathway by varying the input stimulus to manipulate the distortion products and the measuring of the observers' sensitivity to them. The early, linear, filter stage acts like a band-pass filter peaking at 10–15 Hz with substantial sensitivity losses at both lower and higher frequencies. Its characteristics are consistent with nonlinearity being early in the visual pathway but following surround inhibition. The late stage, in contrast, acts like a low-pass filter with a cutoff frequency around 3 Hz. The response of the early stage speeds up with radiance, but the late stage does not. A plausible site for the nonlinearity, which modelling suggests may be smoothly compressive but with a hard limit at high input levels, is after surround inhibition from the horizontal cells.

Introduction

Flickering a light of constant time-averaged intensity can produce shifts in its brightness (e.g., Bartley, 1938, 1939, 1951a, 1951b; Brewster, 1838; Brücke, 1848) or hue (e.g., Ball, 1964; Ball & Bartley, 1966, 1971; Bartley & Nelson, 1960; Bleck & Craig, 1965; Stewart, 1887; van der Horst & Muis, 1969). Such shifts arise because of nonlinearities within the visual pathway that distort the representation of the visual input and so change the appearance of the flickering light. New signals produced by the nonlinearity that are not present at the visual input are called distortion products. By varying the characteristics of the input stimulus, the distortion products generated by the nonlinearity can be manipulated experimentally and measured psychophysically to reveal the internal workings of the visual system. Analogous to neurophysiological recordings, the nonlinearity can be thought of as a psychophysical “electrode” at which the experimenter injects visual signals into the observer's visual pathway. 

In this paper we are principally interested in distortion products that are perceived as shifts in hue. We generate hue shifts of a particular temporal frequency by using contrast-modulated stimuli made up of a sinusoidal carrier (of frequency f_c Hz), the contrast of which is sinusoidally modulated at a much lower modulation frequency (f_m Hz). Such stimuli have no component at f_m Hz but produce a hue change at that frequency. We explore the notion that this hue change arises because of nonlinear distortion at some site within the visual system. Contrast-modulated stimuli, therefore, provide a particularly useful tool for studying nonlinearities and for distinguishing processes and stimulus representations in the visual system that precede the nonlinearity from ones that follow it (e.g., Stockman & Plummer, 1998). 

The conception of the visual system as a linear-nonlinear-linear “sandwich” of stages (e.g., Burns, Elsner, & Kreitz, 1992; Burton, 1973; Chen, Makous, & Williams, 1993; MacLeod, Williams, & Makous, 1992; Marmarelis & Marmarelis, 1978; Spekreijse & Reits, 1982; Stockman, MacLeod, & Lebrun, 1993; Stockman & Plummer, 1998; Trimble & Phillips, 1978; Victor & Shapley, 1980; Victor, Shapley, & Knight, 1977; Wu, Burns, Reeves, & Elsner, 1996) is illustrated in Figure 1. 

Figure 1

 

An example of a compressive nonlinearity (N) separating early (F1) and a late (F2) linear filters operating on a contrast-modulated sinusoidally flickering stimulus. (One modulation cycle of the stimulus is shown in Panel S1.) The amplitude spectrum of the stimulus is shown in Panel C1. Amplitude spectra of the effective waveform after it has passed through successive stages of the linear-nonlinear-linear sandwich are shown in Panels C2–C4 and as corresponding temporal waveforms in Panels S2–S4. The nonlinearity (N) introduces new components at harmonics and sum and difference frequencies of the Fourier components of the stimulus. The colors of the discs above S1–S4 represent the approximate color perception that might be expected at the peaks and troughs of the contrast-modulated flicker. (The Fourier components are illustrative; they are not drawn to scale.)

The left-hand column of Figure 1 shows one cycle of the contrast-modulated visual input (Panel S1) and then its effective alteration (Panels S2 to S4) by successive stages of the linear-nonlinear-linear sandwich. The right-hand column (Panels C1 to C4) shows the corresponding amplitude spectra of the continuous waveforms (i.e., the amplitudes of the waveforms' constituent sinusoidal components). The input stimulus (Panel S1) is a light flickering sinusoidally at f_c Hz, the contrast of which is sinusoidally modulated to a depth of 100% at f_m Hz (with f_m ≫ f_c). Panel C1 at the top right illustrates the amplitude spectrum of the stimulus, which comprises three high-frequency sinusoidal components near f_c (in addition to the component at 0 Hz that corresponds to the mean or direct current (DC) level around which the light is flickered). The three flickering components have frequencies at f_c Hz, the carrier frequency, and at two side-bands: f_c − f_m and f_c + f_m Hz; there is no component at f_m Hz. Given that the visual system behaves linearly at this stage, we should therefore expect no perceptible variation in hue at a frequency corresponding to f_m (illustrated by the unchanging hue of the icons above Panel S1). All an observer should perceive at this stage is flicker at f_c waxing and waning in strength at a rate determined by f_m. 

The temporal characteristics of the linear stages before the nonlinearity are summarized, in this example, by a low-pass filter (Panel F1). This early filter attenuates the components in Panel C1 to produce the intermediate spectrum (Panel C2), which has the same frequency components as the input signal but with reduced amplitude. Because f_m is small relative to f_c, the side bands at f_c ± f_m are close to f_c and so the three components are similarly altered by the filter—their phases and relative amplitudes are virtually unaffected—thus producing a temporal waveform (Panel S2) similar to the stimulus (Panel S1). Again, since there is no component at f_m, we expect no perceptible change in hue at f_m at this stage (illustrated by the icons above Panel S2). One goal of this work is to determine the characteristics of the process preceding the nonlinearity that we characterize as the early filter. 

The linear early stage is followed in the sandwich model by a nonlinear stage (N). The form of nonlinearity illustrated in Figure 1 is compressive in that it selectively attenuates high signal intensities, thus clipping the peaks of the waveform (Panel S3). Other nonlinear forms are considered in the Discussion. In general, a temporal signal passing through any nonlinearity has its frequency spectrum altered by the nonlinearity so that new frequency components (distortion products) are produced at harmonics of the original frequencies and at combinations (weighted sums and differences) of the input frequencies, the amplitudes of which depend on the exact form of the nonlinearity (Bedrosian & Rice, 1971). In the case of our contrast-modulated flicker, among the components produced are those at f_m and 2f_m Hz as well as higher harmonics of the carrier and the side bands (Panel C3). None of the distortion products were present in the early linear stage (C1 or C2). 

We assume that it is the distortion product at f_m Hz that produces a hue change at that frequency (illustrated by the icons above Panel S3). For simplicity, we assume initially that the nonlinearity is monotonic, static, in that its input-output relation depends on the instantaneous value of the input and not on previous inputs and that the form of the nonlinearity is independent of temporal frequency. 

The collective temporal characteristics of the linear processes after the nonlinearity are represented by a second filter, which, in the example of Figure 1, is the low-pass filter shown in Panel F2. This late filter, like the early one, selectively attenuates frequencies near f_c but not enough for them to be below the threshold for detection (Panel C4). 

Versions of the sandwich model have been explored in analyzing the nonlinear interactions between stimuli of two or more sinusoidal frequencies or interactions within white noise for the study of spatial vision (Burton, 1973; Henning, Hertz, & Broadbent, 1975; MacLeod et al., 1992; Williams, 1985), temporal vision (Burns et al., 1992; MacLeod & He, 1993), color vision (Chang, Kreitz, & Burns, 1993; Stockman & MacLeod, 1992; Stockman et al., 1993; Stockman & Plummer, 1998), and brightness enhancement (Krauskopf, Wu, & Farell, 1996; Wu et al., 1996). Wiener analysis has also been used to distinguish the linear and nonlinear stages of spatial vision (e.g., Abel & Quick, 1978). The work reported here is an extension of measurements by Stockman and Plummer (1998) in which they used the sandwich model to dissect the S-cone pathway. They used a contrast-modulated 440-nm flickering target superimposed on an intense steady 620-nm background (to isolate the S-cone response) and from their results concluded that the nonlinearity causing the color change for S-cone stimuli was a hard saturating nonlinearity (i.e., one that was effectively linear at low contrast and limited by a response ceiling at high contrast). (The shape of the early filter was similar to physiologically measured cone responses [e.g., Schneeweis & Schnapf, 1999], which suggests that the nonlinearity was likely to be early in the S-cone pathway close to the photoreceptors [and before significant surround inhibition].) 

The shifts in hue for flicker detected primarily by the L- and M-cones depend upon the wavelength of the flickering light. Near 560 nm the shifts are relatively small, while at wavelengths down to about 520 nm and at wavelengths longer than 560 nm, the hue shifts are towards that of the invariant hue of the 560-nm light (Ball & Bartley, 1971; van der Horst & Muis, 1969). As well as changing in hue, most lights appear more desaturated when flickered (e.g., Ball, 1964). For details, see Ball (1964), van der Horst and Muis (1969), and Ball and Bartley (1971). 

In a subsequent paper, we focus on the brightness shifts. In this paper, we focus on hue shifts. The principal stimulus is a contrast-modulated sinusoidally-flickering 650-nm light that appears more yellow when the flicker contrast is high. Flicker-induced hue shifts from red or green towards yellow seems consistent with a decelerating compressive nonlinearity, such as a logarithmic nonlinearity, separately affecting the red and green inputs into a red-green chromatic pathway. Such a nonlinearity will decrease the mean signal more at high-flicker amplitudes than at low. Thus, a 650-nm flickering light, which produces a larger red signal than a green one, should appear on average less red. A compressive nonlinearity was implicitly assumed by van der Horst and Muis (1969) in their data analysis, and a comparable model (with logarithmic compression) was proposed in 1877 to explain the Bezold-Brücke effect (Peirce, 1877). 

Our results are also broadly consistent with the predominant nonlinearity under these conditions being compressive, but our results suggest that the nonlinearity follows a stage of surround inhibition. 

General methods

This research adhered to the tenets of the Declaration of Helsinki. 

Apparatus

A Maxwellian-view optical system illuminated by a 900-W Xenon arc lamp with a 2-mm entrance pupil was used to produce the stimuli. Infrared and ultraviolet radiation was reduced by glass filters (Schott). Wavelengths were selected by interference filters with full width at half-maximum bandwidths of between 7 and 11 nm (Ealing or Oriel). The radiance of each channel was controlled by the insertion of fixed neutral density filters (Ealing, Oriel, or Melles Griot) and by the rotation of circular, variable 3-log₁₀ unit neutral density filters (Rolyn Optics). The flickering waveforms were generated by pulse-width modulation of fast liquid-crystal light shutters running at 400 Hz with rise and fall times faster than 50 μs (Displaytech) thus producing effectively rectangular pulses of variable width at a fixed frequency of 400 Hz. The pulse width was varied under computer control using programmable timers (Data Translation, DT2819) to produce sinusoidal components at the desired frequencies and at signal modulations up to 92%. Frequencies at the 400-Hz rectangular-pulse frequency and above were much too high to be resolved so that observers saw only the temporally-varying stimuli produced by variation of the pulse width. Consequently, only the form of the pulse width modulation needs to be considered. 

The position of the observer's head was maintained by a dental wax impression fixed to a milling-machine head that could be translated in three dimensions to align the observer's pupil in the optical system. 

Observers

One male (GBH) and one female observer (DP), both authors, participated. They were experienced psychophysical observers with normal color vision and normal (DP) or corrected to normal (GBH) spatial acuity. 

Stimuli

Visual stimuli were centrally-fixated target discs of 4° diameter. The flickering target was a 650-nm light set to one of four time-averaged radiance levels (9.10, 9.70, 10.33, or 10.93 log₁₀ quanta s⁻¹ deg⁻²) in the two main experiments. The background was dark. The target was sinusoidally flickered at f_c Hz and its contrast was sinusoidally-modulated at f_m Hz to produce contrast-modulated sinusoidal flicker. In neither case did the flicker alter the time-averaged radiance. 

Since the waveforms we used are somewhat atypical, we have adopted several conventions to describe them: We refer to the maximum amplitude of the flicker waveform relative to the mean radiance as the overall modulation, m, which is defined as the conventional Michelson contrast: 

 

where I_max and I_min are the maximum and minimum radiances of the stimulus, respectively. Thus, for simple sinusoidal flicker, the waveform, A(t), would be given by: 

 

where R¯ is the mean radiance and f_c is the rate of flicker (Hertz). 

In the main experiments, where contrast-modulated flicker was used, the temporal waveform, A_m(t), was: 

 

where f_c is the carrier frequency, f_m is the modulation frequency (both in Hertz), and m is the overall modulation.¹ We call this stimulus the contrast-modulated flicker. The factor in square brackets is sometimes called the amplitude modulation. The amplitude modulation in our experiments always varied sinusoidally between one and zero at a rate of f_m Hz; i.e., it was always 100% amplitude modulation. 

The flickering component of Equation 3 can be expanded to show that it comprises three sinusoidally-flickering terms: 

 

where the components at f_c Hz, with amplitude R¯m/2 and two side bands at f_c − f_m and f_c + f_m Hz with half the amplitude of the carrier are made explicit. (Note that in our experiments the amplitude modulation was always 100% and the overall modulation of the entire waveform, m, was varied with the result that the side bands always had half the contrast of the carrier.) This type of waveform has been used before (e.g., Stockman & MacLeod, 1992; Stockman & Plummer, 1998; Wu et al., 1996). 

Calibration

Nonlinear distortion of the contrast-modulated stimulus within the generating equipment would produce unwanted distortion products at harmonics of its three components as well as intermodulation distortion at the modulation frequency, f_m, and its harmonics and would thus seriously compromise our attempts to measure distortion generated in the visual system. Consequently, it was vital for this work that the experimental apparatus be linear. The linearity of the system was checked in two ways: first, by plotting the achieved radiance (measured in the plane of the pupil using a United Detector Technology (UDT) Pin-10 photodiode) against the required radiance and noting the linearity of the relation and second, by measuring nonlinear distortion in a contrast-modulated optical stimulus. 

In the second check, the voltage produced by a photometer (Gamma Scientific 2020–31) imaging the flickering stimulus was connected to a spectrum analyzer (Hewlett-Packard 3580A) and the amplitudes of components at various frequencies in the linear range of the photometer were measured. The measurements showed that the amplitude of any unwanted low-frequency components in the stimulus were at least 60 dB lower than that of the carrier and could therefore be ignored. 

The radiant fluxes of test and background fields were measured daily in the plane of the pupil using a calibrated UDT S370 optometer (UDT Instruments, San Diego). 

Procedures

The visual stimulus, focused in the plane of the pupil, was the only visible light source for the observers in an otherwise darkened room. They used their right eye for observation; their left eye was covered. The image of the source in the plane of the observer's pupil was 2.5 mm diameter, always less than the diameter of the natural pupil. The method of adjustment was used to measure visual responses in the experiments. The observers responded using four buttons to vary the overall modulation in small (0.02 log₁₀ unit) or large (0.1 log₁₀ unit) modulation steps up or down to find the particular threshold being measured. A fifth button signaled that the modulation was at threshold. When contrast-modulated stimuli were used, a computer-generated audio signal indicated the moment of the peak contrast at the maximum of the modulation. 

In experiments using contrast-modulated stimuli, the observers were asked to find: (a) the modulation depth at which all flicker disappeared and (b) the modulation depth at which hue changes at the modulation frequency, f_m, disappeared. The results for the three runs were averaged and the mean and standard error for each condition determined. 

Preliminary experiment: Fusion frequencies for detecting flicker and hue change

Introduction

In a preliminary experiment, we determined how the fusion frequency for detecting either flicker at f_c or the hue shift at f_m depended on target radiance. One of the reasons for this experiment was to discover whether the Talbot-Plateau fails at any target radiance as it does with S-cone isolating stimuli (Stockman & Plummer, 1998). 

Methods

A 650-nm target was contrast-modulated with f_m fixed (0.5 Hz) and the overall modulation set to the system maximum of 92%. The observer varied f_c to find the frequencies at which either the flicker, near f_c, or the hue change, at f_m, just disappeared. The settings were made for a range of target radiances. In separate blocks of trials, the time-averaged radiance of the contrast-modulated 650-nm target was varied in steps of approximately 0.45 log₁₀ unit from 6.53 to 10.93 log₁₀ quanta s⁻¹ deg⁻². At each radiance, the observers varied f_c in 0.5 Hz steps to find the highest frequency at which either the flicker or the hue change was just perceptible. Three flicker settings were made first followed by three hue settings because the observers could see flicker at higher f_c frequencies than they could see hue changes. 

Results

Figure 2 shows the threshold frequencies (linear scale) for flicker (open circles) and hue change (solid yellow squares) as a function of log radiance. Data for each observer are shown separately and the error bars indicate ± one standard error. 

Figure 2

 

Critical fusion frequencies, f_c, for the carrier of a contrast-modulated flicker as a function of the time-averaged radiance (log₁₀ quanta s⁻¹ deg⁻²) of a 650-nm, 4° target. The sinusoidal flicker at f_c was contrast-modulated at f_m = 0.5 Hz with overall modulation fixed at 92%. The frequencies for detecting flicker at f_c are shown as white circles and for detecting hue change at f_m as yellow squares. The frequency scale is linear. The error bars indicate ±1 standard error of the mean (SEM). The four downward pointing colored triangles just above the abscissa indicate the four radiance levels used in Experiments 1 and 2.

Curves for both observers grow approximately in parallel and approximately linearly with log radiance from about 8.0 to 11.3 log₁₀ quanta s⁻¹ deg⁻² consistent with the Ferry-Porter law (Ferry, 1892; Porter, 1902). The slope over this range for both observers and for both the flicker and hue-change thresholds is about 10 Hz per log₁₀ unit of radiance. 

Discussion

Although the separation between the hue and the flicker fusion frequencies is smaller for GBH than for DP, the fusion frequencies for seeing hue change are always lower than those for seeing flicker. Consequently, the results for 650-nm flicker are consistent with the Talbot-Plateau law. 

The presence of a consistent gap in frequency between first seeing flicker and first seeing the hue change suggests that, over the range of radiances in Figure 2, flicker that is near-threshold is relatively unaffected by the nonlinearity. This result further suggests that at any frequency there is a range of modulations, near flicker threshold, within which flicker is visible and the distortion product is too small to be seen. 

The observers recorded the hue changes that they saw. These were: (a) a brightness change from dark red to bright red below about 8.0 log₁₀ quanta s⁻¹ deg⁻²; (b) a hue change from red to orange between about 8.0 and 9.5 log₁₀ quanta s⁻¹ deg⁻²; (c) a hue change from red to yellow between about 9.5 and 10.5 log₁₀ quanta s⁻¹ deg⁻²; and (d) a hue and saturation change from red to pale yellow above 10.5 log₁₀ quanta s⁻¹ deg⁻². The hue changes above 8.0 log₁₀ quanta s⁻¹ deg⁻² are consistent with a compressive nonlinearity that reduces the L-cone signal relative to the M-cone signal. The increases in brightness below 8.0 log₁₀ quanta s⁻¹ deg⁻², however, seem to be consistent with an expansive nonlinearity that increases the L-cone signal (the M-cone signal produced by the 650-nm target is small or subthreshold at low 650-nm radiances). 

Previous results for S-cone detected contrast-modulated flicker differ in two important respects: First, above some radiance level, the fusion frequency for seeing S-cone flicker decreases with increasing radiance, and, second, the decrease corresponds to the fusion frequency for seeing flicker falling below that for seeing the hue change, so that the Talbot-Plateau law fails (Stockman & Plummer, 1998). These differences suggest that the origins of the S-cone and M/L-cone nonlinearities may be different, with the former being associated with the well-known saturation of the S-cone signal that occurs under long-wavelength adaptation (e.g., Stromeyer, Kronauer, & Madsen, 1979). For 650-nm flicker below the critical flicker frequency, the hue change is apparent at all radiances above 8.0 log₁₀ quanta s⁻¹ deg⁻². At modulations high enough for the distortion of the 650-nm flicker to be visible, the nonlinearity presumably must also affect modulation at the carrier frequency, f_c, since it is by redistributing energy from the carrier (and sidebands) that the nonlinearity produces the distortion product that produces the hue change at f_m. 

Experiment 1: TCSFs for hue-change, flicker, and chromatic flicker as a function of fc

Introduction

In Experiment 1, we measured contrast-sensitivity functions for 650-nm targets at the four radiances indicated by the colored triangles indicated just above the abscissa of Figure 2. The levels are within the radiance region in which the distortion is seen as a hue change. 

Contrast-sensitivity for detecting the hue change at f_m as a function of f_c provides an estimate of temporal characteristics of the early filter before the nonlinearity. We make this claim because the component at f_m is present in the system only after the nonlinearity. Further, because f_m is fixed at 0.5 Hz, the observers, in finding the hue-change thresholds at each f_c, are adjusting the overall modulation of the contrast-modulated stimulus to produce the same threshold f_m-signal at the input to the late filter. That threshold will be determined by the late filter's response to the constant f_m frequency of 0.5 Hz and thus (apart from an unknown scaling factor) will be unaffected by the attenuation characteristics of the late filter. Moreover, because f_m ≫ f_c the early filter's effect on the contrast-modulated stimulus will be simply to reduce the effective overall modulation. Thus the observers' adjustments of overall modulation compensate for the change produced by the early filter and provide a measure of the attenuation characteristics of the early filter. 

In this experiment we compare the hue-change thresholds with conventional temporal contrast sensitivity functions (TCSFs) measured with monochromatic and chromatic flicker. 

Methods

Three experiments were carried out at mean radiances of 9.10, 9.70, 10.33, and 10.93 log₁₀ quanta s⁻¹ deg⁻². 

Hue change temporal sensitivity measurements

In the first experiment, the primary target was the 4° diameter, contrast-modulated, 650-nm light of Equation 3 with the carrier at f_c and contrast modulation at f_m. The observers varied the overall modulation (m in Equation 3) in either 0.02 or 0.10 log₁₀ unit steps to find the threshold for just detecting a hue change at f_m. We varied f_c from 5 to 60 Hz with a fixed f_m of 0.5 Hz. (Below an f_c of 5 Hz, flicker at f_c, like the distortion at f_m, also appeared chromatic, as a result of which it was difficult to separate the two.) 

Conventional temporal sensitivity (flicker) measurements

In addition to measuring the observers' thresholds for detecting the hue change, we also measured conventional temporal contrast sensitivity functions (TCSFs) with simple sinusoidally flickering stimuli (Equation 2). A sinusoidally-flickering, 4°-diameter, 650-nm target was presented at frequencies from 0.5 to 60 Hz and at one of the four mean target radiances. At each f_c, subjects varied the overall modulation (m in Equation 2) in either 0.02 or 0.10 log₁₀ unit steps to set the modulation of the target at which they could just detect its flicker. We refer to these measurements as monochromatic TCSF measurements. 

For the monochromatic measurements, the target was sinusoidally-flickered at f_c Hz (i.e., f_m = 0 Hz). Contrast-modulation was not used, because when f_c was low, the frequency components near f_c Hz and the distortion products near f_m were similar in frequency and appearance, so that reliable threshold settings could not be made. At higher f_c, the thresholds for contrast-modulated flicker with f_m fixed at 0.5 Hz (not shown) were very similar to the thresholds obtained with continuous flicker. 

Temporal contrast sensitivity for detecting chromatic flicker

In this experiment, we first determined a series of equiluminant or chromatic stimuli with which to measure the chromatic TCSFs by superimposing a 4°-560-nm target on the 4°-650-nm target, both flickering sinusoidally at 30 Hz but 180° out of phase. The 560-nm and 650-nm targets were luminance equated individually for each observer by asking them to cancel the flicker of the superimposed targets photometrically. To do this, two 30-Hz targets were superimposed in opposite phase. Both were set to 92% modulation with the radiance of the 650-nm light fixed. The observers then varied the radiance of the 560-nm light to null the perception of flicker. (Observers were also able to make small phase adjustments, in steps of 2° or 10°, to perfect the null, but the optimum phase for the null was always near 180° at 30 Hz.) 

The mean time-averaged luminances of the combined luminance-equated 650- and 560-nm targets were then set to four different time-averaged luminances of the 650-nm target alone using the Sharpe, Stockman, Jagla, and Jägle (2005, 2011) luminous efficiency estimates (now adopted as a standard by the Commission Internationale de l'Éclairage or CIE [2006]). The chromatic TCSFs were then measured by jointly adjusting the modulation (Equation 2) of the out-of-phase, luminance-equated, 650- and 560-nm targets while maintaining the appropriate equiluminant ratio. Adjustments were made in steps of either 0.02 or 0.10 log₁₀ unit to find the modulation at which the observers could just detect flicker. As for the monochromatic measurements, the target for chromatic flicker was sinusoidally-flickered at f_c Hz (i.e., f_m = 0 Hz). 

The principle of this type of measurement is that flicker photometric cancellation, when effective, silences the luminance pathway so that flicker detection must be by some chromatic mechanism or pathway (for review, see Stockman & Brainard, 2009). Thus, we refer to these measurements as chromatic TCSF measurements. 

Results

Each panel of Figure 3 shows the logarithm of temporal contrast sensitivity for detecting either hue change at f_m (colored symbols) or monochromatic (equichromatic) or chromatic (equiluminant) flicker at f_c (open and grey symbols, respectively), all plotted as a function of f_c (Hertz, logarithmic axis). Data for DP are shown in the left-hand column and those for GBH in the right. Time-averaged radiance for each row (indicated at the top right of each panel in the right-hand column) increases from 9.10 at the top to 10.93 log₁₀ quanta s⁻¹ deg⁻² at the bottom. 

Figure 3

 

The log₁₀ modulation sensitivities for DP (left-hand column) and GBH (right-hand column) for detecting either monochromatic (equichromatic) or chromatic (equiluminant) flicker at f_c (open and grey symbols, respectively), or for detecting the hue change at f_m (colored symbols) both plotted as a function of f_c (logarithmic axis). Each row shows results for the different time-averaged radiances shown in the top corner of the panels in the right-hand column: 9.10- (inverted brown triangles), 9.70- (red circles), 10.33- (orange diamonds), and 10.93- (yellow squares) log₁₀ quanta s⁻¹ deg⁻². Unmodulated sinusoidal flicker at f_c was used for the flicker measurements. For the hue-change measurements, the sinusoidal flicker at f_c was contrast-modulated at f_m = 0.5 Hz. Error bars indicate ±1 SEM. The chromatic flicker was made up of luminance-equated 650-nm and 560-nm targets flickering in opposite phase. The modulations for the chromatic flicker shown in the figure are referenced to the modulation of the 650-nm component. At each level, the overall time-averaged luminance of the combined 650- and 560-nm lights was set to be the same as the time-averaged luminance of the 650-nm targets used in the monochromatic measurements using the Sharpe et al. (2005, 2011) luminous efficiency estimates. In increasing order of radiance, the 560-nm equiluminant quantal radiances were 1.09, 1.13, 1.30, and 1.03 less than the 650-nm radiances for DP and 1.05, 1.04, 1.09, and 1.18 less for GBH, respectively.

The TCSFs for detecting the hue change near f_m (colored symbols in Figure 3) are restricted to frequencies ≥ 5 Hz. Below 7.5 or 10 Hz, the functions fall off slightly in sensitivity so that the functions are slightly band pass. The high frequency falloffs in sensitivity reach a slope of about 3 log₁₀ units per decade, which are shallower than the slopes for both the monochromatic 650-nm and the chromatic flicker TCSFs. 

The TCSFs for monochromatic sinusoidal flicker (open symbols in Figure 3) are band pass in form. They peak between about 5 and 10 Hz and fall off at low frequencies with slopes of about 0.8 log₁₀ unit per decade and at high frequencies with slopes of more than 4 log₁₀ units per decade. The low and high frequency losses are consistent with other achromatic and monochromatic flicker TCSFs (e.g., De Lange, 1958; Green, 1969; Kelly, 1961, 1973; Varner, Jameson, & Hurvich, 1984). The high-frequency slopes of the functions in Figure 3 become slightly shallower as the mean target radiance increases. These changes are consistent with a shortening of the temporal response of the visual system (see, for review, Stockman, Langendörfer, Smithson, & Sharpe, 2006). 

The TCSFs for chromatic sinusoidal flicker (grey symbols in Figure 3), unlike the monochromatic ones, are low pass in form and only falloff in sensitivity above 5 to 10 Hz. Moreover, where they overlap in frequency the falloff in sensitivity is typically steeper than for the monochromatic TCSFs. These data are similar in shape to other TCSFs measured with equiluminant chromatic flicker (e.g., Kelly, 1975; Kelly & van Norren, 1977; Varner et al., 1984) and to some measured under L- or M-cone-isolating conditions (e.g., Estévez & Spekreijse, 1974; Smith, Pokorny, Davis, & Yeh, 1995). 

Discussion

The hue change at f_m produced by contrast-modulated flicker is assumed to be generated by distortion at a nonlinear site within the visual pathway (see above). At threshold in the hue-change measurements, this site generates the same threshold 0.5-Hz hue shift for all f_c, so that, in principle, the effects of the stages after the nonlinear site (the temporal characteristics of which are estimated next) can be discounted. The contrast sensitivity for detecting the hue shift at f_m as a function of f_c (colored symbols, Figure 3) is therefore an estimate of the temporal characteristics of the early filter before the nonlinearity (aside from an unknown scaling constant). 

By contrast, the monochromatic and chromatic TCSFs reflect the temporal characteristics of both the early and the late filters and therefore should be steeper, as we find. We discuss the relation between the monochromatic and chromatic TCSFs and which of them is more likely to depend on the same early filter as the hue-change measurements in more detail below. The monochromatic and chromatic TCSFs will also be used to extend our estimates of the characteristics of the early and late filters to frequencies at which they could not be determined from the hue-distortion measurements (see Figures 7 and 8 below). 

Experiment 2: TCSFs for hue-change as a function of fm

Introduction

Contrast sensitivity for detecting the hue shift at f_m as a function of f_m for a fixed f_c allows us to extract an estimate of temporal characteristics of the late filter after the nonlinearity. 

We make this claim because, with f_c fixed at 30 Hz and f_m ≪ f_c, the only effect of the early filter is to scale the overall modulation, m, by an unknown factor that may depend on the mean radiance but does not depend on f_m. Further, the component producing the hue change at f_m arises after the nonlinearity. Thus, in finding hue-change thresholds as a function of f_m, the observers are adjusting the overall modulation at the input to the nonlinearity in order to produce a threshold signal at f_m at the output of the late filter. The overall modulation necessary to produce this threshold signal will depend both on the form of the nonlinearity and the attenuation characteristics of the late filter. A supplementary experiment described in the next section is required to separate the attenuation characteristics of the late filter from the effects of the nonlinearity. 

Methods

Observers in this experiment were asked to set thresholds for hue change as a function of the modulation frequency, f_m. The carrier frequency, f_c, was held constant at 30 Hz. The experiment was carried out with the same mean radiances as Experiment 1. Measurements could be made only up to about 5 Hz, because at higher modulation frequencies observers could not detect a hue change varying at f_m. 

Results

Figure 4 shows the TCSFs for detecting hue change. In each panel, the logarithm of modulation sensitivity is plotted against log f_m. Data for DP are shown in the upper panel; for GBH, in the lower. Functions for four radiance levels (log₁₀ quanta s⁻¹ deg⁻²) are shown: 9.10 (brown triangles), 9.70 (red circles), 10.33 (orange diamonds), and 10.93 (yellow squares). The TCSFs are mainly low pass in form falling at about 0.5 log₁₀ units per decade. 

Figure 4

 

The log₁₀ modulation sensitivities for DP (upper panel) and GBH (lower panel) for detecting hue change at f_m as a function of f_m (logarithmic axis) at four time-averaged target radiances: 9.10- (brown triangles), 9.70- (red circles), 10.33- (orange diamonds), and 10.93- (yellow squares) log₁₀ quanta s⁻¹ deg⁻². The sinusoidal flicker at f_c = 30 Hz was contrast-modulated at f_m Hz. Error bars indicate ±1 SEM.

Sensitivity appears to increase between the two lowest radiances, but thereafter there are no clear systematic changes with radiance. 

Discussion

In this experiment, observers varied the overall modulation of the contrast-modulated stimulus to set the threshold for seeing the hue change at f_m. Their thresholds depend on the characteristics of the (late) filter after the nonlinearity and on how the distortion product generated by the nonlinearity varies with input modulation. Since f_c is fixed at 30 Hz (and f_c ≫ f_m), the filter before the nonlinearity affects the curves in Figure 4 only by effectively attenuating the optical input, which is equivalent to vertically displacing the logarithmic TCSF curves with no change in shape. 

In the next experiment, we investigate the relation between the distortion product and the input modulation. 

Supplementary experiment: Dependence of the hue change on modulation

Introduction

In this supplementary experiment, we determine how the contrast of the f_m-distortion product at the output of the nonlinearity depends on the overall modulation, m, of the contrast-modulated stimulus at the input to the nonlinearity. Knowledge of this relation allows us to extract the late filter characteristics (F2 in Figure 1) from the hue-change TCSF measurements made as a function of f_m shown in Figure 4. To gauge the size of the chromatic distortion product, we use a matching procedure. 

Methods

In a series of preliminary studies, we evaluated various methods of nulling or matching the chromatic distortion. We finally adopted a side-by-side matching technique by splitting the 4° target into test (left) and matching (right) hemi-fields. In the test hemi-field, the observer was presented with the contrast-modulated 650-nm target stimulus (with f_c = 30 Hz and f_m = 0.5 Hz). In the matching hemi-field, the observer was presented with a hue-matching stimulus. The matching stimulus comprised two superimposed components: (a) a steady 650-nm pedestal that had the same time-averaged radiance as the 650-nm test stimulus and (b) a 560-nm light that was sinusoidally flickered at 0.5 Hz and set to 92% modulation with the same phase as the 0.5-Hz contrast-modulation (f_m) of the test stimulus. The combined matching stimulus produced hue and saturation changes that could be compared to those produced by the contrast modulation of the test stimulus. The observers adjusted the time-average radiance of the 560-nm component (keeping the 92% modulation constant) to match the hue and saturation changes of the test stimulus. The arrangement is illustrated in Figure 5. Wu et al. (1996) used a comparable method for matching flicker-induced changes in brightness, but their stimuli were separated in time rather than in space. 

Figure 5

 

The arrangement used to match the hue change and desaturation of a 650-nm contrast-modulated flicker (left hemi-field and left stimulus) by varying the time-averaged radiance and amplitude of 92% modulated sinusoidal stimuli superimposed on a 650-nm pedestal of the same time-averaged radiance as the contrast-modulated target (right hemi-field and right stimuli). The sinusoid was usually 560 nm. The small discs at the bottom of the figure indicate the approximate appearances of the hemi-fields when the matching contrast modulation (left) and sinusoidal flicker amplitude (right) are at their minima and maxima.

We tried various other techniques, such as varying the modulation of the 560-nm matching light (which would have allowed us to maintain a constant time-averaged radiance in the matching half-field), but they produced much poorer matches. 

The modulation of the contrast-modulated, 650-nm left hemi-field was set to values between 2% and 92%. At each modulation, the observers varied the mean radiance of the sinusoidally flickering stimulus in the right hemi-field (keeping m constant) until the appearance of the maximum color change in the contrast-modulated stimulus on the left (towards yellow) was matched by the appearance of the maximum hue change of the sinusoidal modulation on the right (also towards yellow).This technique produced an acceptable match between the two half-fields at the maximum of the contrast-modulation. 

The observers were also able to vary the phase between the contrast-modulation flicker and the sinusoidal flicker of the two hemi-fields in either 2° or 10° steps until the maximum change in appearance of the two fields coincided. In practice, the phase adjustments away from in-phase were relatively small. However, we retained the adjustment as part of the procedure because the ability to adjust the phase seemed to help the observers make more consistent matches. 

The method was modified slightly at the highest radiance level for GBH for whom an additional desaturating 492-nm light was presented in-phase with the matching stimulus. His results with and without the 492-nm light were in agreement, so that only the results with the single flickering component are shown. 

Results

The contrast of the 0.5-Hz sinusoidal stimulus that matches the hue of the contrast-modulated test stimulus at the peak of its 0.5-Hz contrast modulation is plotted as L-cone contrast in the left-hand panels of Figure 6 for DP (upper) and GBH (lower) as a function of the overall modulation, m, of the contrast-modulated test hemi-field—both with linear scales. Extensive functions for three radiance levels (log₁₀ quanta s⁻¹ deg⁻²) are shown: 9.70 (red circles), 10.33 (orange diamonds), and 10.93 (yellow squares). Only two points could be measured at the lowest radiance, brown triangles. 

Figure 6

 

L-cone contrasts of the 0.5-Hz sinusoidally flickering hemi-field that matches the appearance of 0.5-Hz contrast-modulated 30-Hz flicker in an adjacent hemi-field are shown as a function of the overall modulation (m) of the contrast-modulated stimuli (left-hand column). The matches are plotted in cone contrast space in the right-hand column with L-cone contrast plotted along the abscissa and M-cone contrast along the ordinate. The upper row shows matches for DP and the lower row those for GBH. The time-averaged radiances were: 9.10- (brown triangles), 9.70- (red circles), 10.33- (orange diamonds), and 10.93- (yellow squares) log₁₀ quanta s⁻¹ deg⁻². Error bars indicate ±1 SEM. (Only two points could be measured for the 9.10 radiance level.) The red arrows shown in the right panels are the luminance vectors in cone contrast space that corresponds to pure monochromatic flicker (650-nm flicker in our experiment). Error bars indicate ±1 SEM.

Figure 7

 

Estimates of the logarithmic sensitivities corresponding to the attenuation characteristics of the late filter for DP (left-hand panel) and GBH (right-hand panel) at each of the four time-averaged radiance levels of 9.10 (triangles), 9.70 (circles), 10.33 (diamonds), and 10.93 (squares) log₁₀ quanta s⁻¹ deg⁻². Error bars indicate ±1 SEM. The colored symbols show the logarithmic sensitivities for detecting the hue change at f_m as a function of f_m from Figure 4 rescaled according to the input versus output contrast functions shown in Figure 6. Each set of data has been vertically aligned so that it has the same maximum amplitude sensitivity in log₁₀ quanta s⁻¹ deg⁻² before and after rescaling. The grey symbols are the sensitivity differences between the early filter estimates (colored symbols, Figure 3) and the chromatic TCSF measurements (grey symbols, Figure 3). The alignment of the grey symbols with the colored ones was determined by the fit of a two-stage low-pass filter model (red lines), the details of which are described in the text. Error bars indicate ±1 SEM.

Figure 8

 

Estimates of the logarithmic sensitivities corresponding to the attenuation characteristics of the early filter for DP (left-hand panel) and GBH (right-hand panel) at each of the four time-averaged radiance levels of 9.10 (triangles), 9.70 (circles), 10.33 (diamonds), and 10.93 (squares) log₁₀ quanta s⁻¹ deg⁻². Error bars indicate ±1 SEM. The colored symbols are the logarithmic sensitivities for detecting the hue change at 0.5 Hz as a function of f_c from Figure 3 (also shown there as colored symbols). The sensitivities are the logarithmic amplitude sensitivities in log₁₀ quanta s⁻¹ deg⁻². The open symbols are the sensitivity differences between the late filter estimates (colored symbols, Figure 7) and the monochromatic TCSF measurements (open symbols, Figure 3). The alignment of the open symbols with the colored ones was determined by the fit of a model (black lines), the details of which are described in the text. Error bars indicate ±1 SEM.

The right-hand panels for DP (upper) and GBH (lower) plot the larger M-cone contrast for each match against the L-cone contrast. (M- and L-cone-contrasts were calculated using the Stockman & Sharpe, 2000, cone fundamentals and are for the combined matching stimulus, i.e., they include both the 560-nm sinusoidal flicker and the 650-nm pedestal.) 

In cone-contrast space (the positive quadrant of which is shown in the right-hand panels of Figure 6), vectors at 45°, along which the L-cone and M-cone contrasts increase or decrease by the same amount together, are pure equichromatic vectors, whereas those at 135°, along which one cone contrast increases and the other decreases, are pure chromatic vectors. Other vectors have both an equichromatic and chromatic component. The matching stimuli in our experiment vary along a vector close to the M-cone axis—a vector with both luminance and chromatic components. Note also that with increasing mean radiances, the matches first move out along the matching vector and then, with further increases in mean radiance, particularly for DP, move back toward the origin, which suggests a reduction in the size of the distortion product at the highest radiances or perhaps a change in filter shape (see Figure 12, below). 

For DP, the contrasts of the matching stimuli are less than about 4% and 25% for L- and M-cones, respectively, and less than 9% and 50%, respectively, for GBH. The large M-cone contrasts, particularly for GBH, raise the concern that the 0.5-Hz sinusoidal matching stimulus may itself undergo some distortion at the nonlinearity. However, the 0.5 Hz matching stimulus is attenuated by the early filter, roughly indicated at the low-frequency end by the model fits in Figure 8. Thus, for the worst case (10.33 log₁₀ quanta s⁻¹ deg⁻² for both observers) the 0.5-Hz sinusoid at the input to the nonlinearity will be smaller by a factor of about 1.5 log₁₀ unit for both observers. Consequently the highest matching contrast at the input to the nonlinearity for GBH is about 1.6% and for DP, about 0.8% and most matching contrasts will be much lower. Together with the data shown in Figure 2, these results suggest that the matching sinusoid is effectively undistorted. Thus, for each observer, there is an unknown scaling factor at each mean radiance (determined by the slope of the linear approximation at that radiance) that converts the input modulation, m, of the contrast-modulated waveform to the contrast of the distortion product at the input to the late filter. 

Discussion

Given the reasonable approximation of small-signal linearity at all mean radiances, we can simply replace the overall contrast of the contrast-modulated waveform (abscissae of the left-hand panels in Figure 4) by scaled values of the corresponding ordinates of the left-hand panel of Figure 6. The rescaled TCSFs for detecting the hue change at f_m are plotted as the colored symbols for the two observers in Figure 7 (data for DP on the left, for GBH on the right). Values for four radiance levels (log₁₀ quanta s⁻¹ deg⁻²) are again shown: 9.10 (brown triangles), 9.70 (red circles), 10.33 (orange diamonds), and 10.93 (yellow squares). (Rescaling was not attempted at 9.10 log₁₀ quanta s⁻¹ deg⁻² because of the lack of matching data at that level, see Figure 6). The pattern as a function of f_m is appropriate for each mean radiance. They have been vertically shifted so that the peak sensitivity at each level is the same as it was before scaling. Other details from this figure are discussed below. 

Considering only their shape, we find that the relative rescaling of the TCSF curves has steepened the loss above 1 Hz slightly more for DP than for GBH because the convexities for DP in Figure 6 are greater. But the effects of the rescaling are relatively small. The colored symbols in Figure 7 constitute our estimates for the low-frequency end (f < 5 Hz) of the post-nonlinearity (late) filter. 

General discussion

Combining the results at low and high frequencies

In Experiment 1, we were only able to characterize the attenuation characteristics of the early filter above 5 Hz; below carrier frequencies of 5 Hz, it was too difficult to distinguish the modulation and carrier frequencies and so that we were unable to characterize the low-frequency end of the early filter. In Experiment 2, we could only measure hue changes below 5 Hz; at modulation frequencies above 5 Hz, there was no apparent hue change at f_m, so that we were unable to characterize the high-frequency end of the late filter. Although limited in frequency range, these estimates represent our strong conclusions. Since they depend on a single type of judgment, the threshold for detecting a hue shift at the modulation frequency, they can be reasonably assumed to depend on a single process. The strong conclusions are highlighted by the colored symbols in Figures 7 and 8. 

To estimate the characteristics of the early filter below 5 Hz and the late filter above 5 Hz (i.e., at frequencies where we cannot measure directly), we explored two possibilities: we assumed that the multiplicative combination of the early and the late filters was reflected either in the shapes of the TCSFs measured with equiluminant chromatic flicker or in the shapes of the TCSFs measured with monochromatic flicker. Because these estimates are based on different judgments of flicker detection and hue change, they are less likely to depend on a single process. Consequently, these estimates are inevitably more speculative and represent our weaker conclusions. The weaker conclusions are indicated by the achromatic symbols in Figures 7 and 8. 

We now discuss why the first of these possibilities seems to be the appropriate one for the late filter and the second of them for the early filter. The estimates for the late filter are shown in Figure 7 and for the early filter in Figure 8. In each panel of Figures 7 and 8, the results are plotted separately for DP (left-hand panels) and GBH (right-hand panels) for four radiance levels (log₁₀ quanta s⁻¹ deg⁻²): 9.10 (triangles), 9.70 (circles), 10.33 (diamonds), and 10.93 (squares). 

We assume that the inferred shapes of the late filters above 5 Hz are the logarithmic differences between the chromatic TCSFs (grey symbols, Figure 3) and the corresponding early filter characteristics (colored symbols, Figure 3 or 8). These estimates are shown as the grey symbols in Figure 7, where they have been scaled to align with the scaled estimates of the late filters for each observer—the colored symbols in each panel. We used the chromatic TCSFs to derive this estimate on the grounds that the distortion product was seen primarily as a change in hue. (Had we used the monochromatic TCSFs, the high frequency sensitivity loss would have a logarithmic slope of about −1 rather than −2, see Figures 10 and 11 below). 

Figure 9

 

Fitting parameters of the models describing the early (left panels) and late (right panels) filter characteristics. Overall, the model accounts for the data well with an R² value of 0.989. The upper panel of Figure 9 shows the corner frequencies for DP (yellow squares) and GBH (filled squares) and the lower panel the scaling or sensitivity losses for DP (yellow circles) and GBH (filled circles) that were required in addition to the effects of changes in corner frequency. The red line shows the sensitivity losses that would be expected if Weber's Law is obeyed. The parameters for DP and GBH are very similar.

Figure 10

 

Estimates of the early filter (left-hand panel) from Figure 8 and late filter (right panel) from Figure 7 for DP replotted as colored symbols with the final model fits (red lines). In the left panel, the open symbols show the differences between the late filter estimates at low frequencies (colored symbols, Figure 7) and the chromatic TCSFs (grey symbols, Figure 3). The curves fitted to the open symbols are versions of the red curves scaled by 60% but with the same peak sensitivity. The scaling and alignment are best fitting, but they are largely illustrative. The initial alignment was based on a model of the form of Equation 7. In the right panel, the open symbols show the differences between the early filter estimates at higher frequencies (colored symbols, Figure 8) and the monochromatic TCSFs (open symbols, Figure 3). Each set of data have been fitted by a line with a logarithmic slope of −1 (black lines), and the lines and data have been vertically shifted together so that the lines align with the chromatic curves at 2.5 Hz. Error bars indicate ±1 SEM. The vertical positions of the top set of data in each panel are the same as in Figures 7 and 8, but for clarity the second, third, and fourth sets have been shifted downwards by an extra one, two, and three log₁₀ units, respectively.

Figure 11

 

As Figure 10, but for GBH. The curves fitted to the open symbols in the left panel are versions of the red curves scaled by 33% but with the same peak sensitivity.

Figure 12

 

Effect of the compressive nonlinearity illustrated in the upper panel on the contrast of the distortion product at f_m. The upper panel shows the input/output function of Equation 8 and the lower panel the contrast of the distortion product at f_m resulting from a contrast-modulated input plotted as a function of the input modulation and DC offset. For details, see text.

We assume that the shapes of the early filters below 5 Hz are the logarithmic differences between the monochromatic TCSFs (open symbols, Figure 3) and the late filter characteristics (colored symbols, Figure 7). The resulting low-frequency estimates are shown as the open symbols in Figure 8, where they have been scaled to align with the directly measured characteristics of the early filters (colored symbols, Figure 8). We used the monochromatic TCSFs to derive this estimate on the grounds that the contrast-modulated flicker used to generate the distortion product is also monochromatic and therefore subject to comparable low-frequency attenuation. (Had we used the chromatic TCSFs, the low-frequency attenuation would be much less, see Figures 10 and 11 below and the related discussion). 

In making these estimates, we ignore the effects of the nonlinearity intervening between the early and late filters on the grounds of the obedience to the Talbot-Plateau law (see Figure 2) and small-signal linearity (the flicker thresholds are less than 10%). 

It will be remembered that our measurements determined early and late filter characteristics only up to an arbitrary scalar; i.e., the shape was determined but not the vertical displacement on our double logarithmic coordinates. That allows us some freedom in the alignments illustrated in Figures 7 and 8—the alignments of the measured and estimated filter characteristics are not fixed by our data. To achieve the alignments shown in Figures 7 and 8, we developed simple models of the early and late filters. These models not only helped to align the data, but also yield insights into the underlying physiological mechanisms that give rise to the data. 

Modelling the early and late filters

We take the classic approach of modeling the data by using filters composed of cascading (n) leaky integrating stages (or buffered resistor-capacitor circuits) (see Watson, 1986). The amplitude response, A(f), of n cascaded, identical, leaky integrators is: 

 

where, in this formulation, τ (seconds) is a time constant common to all stages, n is the number of stages, and f is frequency (Hertz) The phase response, ϕ(f), is: 

 

In the frequency domain, each filter and the cascade of n filters, is low pass in form. 

It can be helpful to reformulate parts of Equations 5a and 6a in terms of cutoff or corner frequency f₀ (Hertz), where f₀ = 1/(2πτ). Thus reformulated, Equation 5a becomes: 

 

and Equation 6a: 

 

 

The corner frequency, f₀, can be thought of as the frequency at which frequency-dependent sensitivity losses become significant (when f = f₀ the loss factor for a single filter stage is 1/ $\sqrt{2}$ ). Phase changes begin at much lower frequencies and asymptote at n π/2 as f becomes large; however, we shall not be concerned with phase responses. Inspection of Equation 5b also shows that when f > f₀ the rate of sensitivity losses becomes independent of f₀ (or τ) and depends solely on n. 

This linear systems approach remains relevant in the modern context of cascaded molecular processes, because a first-order biochemical reaction behaves like one of these leaky integration stages. 

The early filter

The early filter estimates (Figure 8) are clearly band pass in shape. To account for the low-frequency attenuation, which is usually related to some form of surround inhibition (see below), we divide a center temporal response of a cascade of filters by a surround temporal response of another cascade of filters, thus: 

 

where the subscript c refers to the parameters of the central cascade of filters, and the subscript s refers to the parameters of the surround cascade. In the spatial-frequency literature, this treatment is called divisive inhibition (e.g., Foley, 1994). The assignment of the filters to a center and a surround is of course speculative (since the two could be spatially coextensive), but plausible given known retinal physiology and surround feedback circuits (e.g., Dacey et al., 2000). 

Best-fitting versions of the model described by Equation 7 were obtained using a standard, nonlinear, least-squares, curve-fitting algorithm (SigmaPlot, SPSS) to account for the data shown in Figure 8 obtained for each observer. Fits were made using the logarithm of Equation 7 and the data. A logarithmic scaling constant, k, that could vary with target radiance level was allowed. In terms of the model, k represents a frequency-independent sensitivity loss that is in addition to any losses resulting from the changing corner frequencies of the filters. In addition, an extra arbitrary alignment constant, v, was added to the low-frequency data for each level, the value of which was individually optimized in the fit to determine the best-fitting vertical alignment of the low- and high-frequency data, thus: 

 

 

We fitted Equations 7 and 8 simultaneously to the estimates of the functions for both DP and GBH and simplified the model by fixing those parameters that did not vary systematically with target radiances or across observers. Our aim was to derive a plausible descriptive model with as few parameters as could reasonably account for the totality of the data. 

One simplification, embodied in the use of Equation 7, is the assumption that at any target radiance the cutoff frequencies of all the center stages (f_0c) are the same and similarly that the cutoff frequencies of all the surround stages (f_0s) are the same. Initially, we allowed both f_0c and f_0s to vary with target radiance level but found that we could reasonably simplify the model by fixing f_0s across levels. (In preliminary fits, we allowed the number of stages in the center and surround, n_c and n_s, to take on noninteger values, but in the final fits we constrained them to the nearest integer values, n_c = 6 and n_s = 2). When allowed to take on noninteger values, the fitted values and their standard errors were n_c = 5.55 ± 0.33 and n_s = 1.77 ± 0.21. Scaling (i.e., the vertical logarithmic shift, k, in Equation 8) and the center cutoff frequencies were allowed to vary separately for each observer. 

The results of the final fit of the model for the early filters are shown in Figure 8 as the continuous dark lines. The parameters from the fit are tabulated in Table 1 and plotted for both observers in the left-hand panels of Figure 9. The values of v, which are arbitrary, are not given. The upper panel of Figure 9 shows the corner frequencies for DP (yellow squares) and GBH (filled squares) plotted as a function of target radiance. The lower panel shows the scaling or sensitivity losses for DP (yellow circles) and GBH (filled circles) that were required in addition to the losses caused by changes to the corner frequency. Overall, the model accounts for the data well with an R² value of 0.989. The parameters for the fits for DP and GBH are very similar. 

Table 1

 

Best-fitting parameters for the early filter model. See text for details.

Parameter	DP	GBH
nc (fixed)	6
ns (fixed)	2
f0s (fixed)	1.18 ± 0.07
f0c 9.10	12.69 ± 0.45	11.82 ± 0.45
f0c 9.70	15.37 ± 0.57	14.83 ± 0.50
f0c 10.33	18.19 ± 0.60	17.68 ± 0.58
f0c 10.93	18.77 ± 0.58	18.95 ± 0.63
k 9.10	0.00 ± 0.06	0.00 ± 0.07
k 9.70	0.34 ± 0.07	0.18 ± 0.06
k 10.33	0.62 ± 0.06	0.44 ± 0.06
k 10.93	1.20 ± 0.05	0.99 ± 0.06
R2	0.989

Readers should note that stages of n_c = 6 and n_s = 2, since they are divisive, imply that the logarithmic limiting high-frequency slope is −4. 

The late filter

The late filter for both observers is broadly consistent with the simple low-pass filter of Equation 5b. We fitted this equation simultaneously to the estimates of the functions for both DP and GBH, and, again, we simplified the model by fixing some parameters that did not systematically vary across either target radiances or observers. In preliminary fits, we found that we could fix the cutoff frequencies across radiance levels. We also allowed the number of stages to take on noninteger values, but in the final fit we constrained them to the nearest integer values of n = 2. Again, an additional arbitrary alignment constant, v, was added to the low-frequency data of each set and individually optimized to determine the best-fitting vertical alignment of the low- and high-frequency (see Equation 8). 

The results of the final fit of the model for the late filters are shown in Figure 7 as the continuous red lines. The parameters from the final fit are tabulated in Table 2 and plotted in the right-hand panels of Figure 9. Again, the values of v, which are arbitrary, are not given. The upper panel shows the fixed corner frequencies for DP (yellow squares) and GBH (filled squares) plotted as a function of target radiance, the lower panel shows the scaling or sensitivity losses for DP (yellow circles) and GBH (filled circles). As indicated by the red line with a slope of one, the increases in the sensitivity losses with target radiance are consistent with Weber's Law. 

Table 2

 

Best-fitting parameters for the late filter model. See text for details.

Parameter	DP	GBH
n (fixed)	2
f0 (fixed)	2.79 ± 0.34	3.08 ± 0.39
k 9.10	0.00 ± 0.09	0.00 ± 0.09
k 9.70	0.51 ± 0.08	0.54 ± 0.08
k 10.33	1.23 ± 0.08	1.22 ± 0.09
k 10.93	1.81 ± 0.08	1.95 ± 0.09
R2	0.986

The parameters from the fit are tabulated in Table 2 and plotted in the right panels of Figure 9. The upper panel shows the corner frequencies for DP (yellow squares) and GBH (filled squares) plotted as a function of target radiance. The lower panel shows the scaling or sensitivity losses for DP (yellow circles) and GBH (filled circles) that were required in addition to the losses caused by changes to the corner frequency. Overall, the model accounts for the data well with an R² value of 0.986. 

Site of the nonlinearity

Previous studies have suggested that nonlinearities comparable to the one investigated here lie at an early level of the visual system close to the photoreceptors (MacLeod et al., 1992; Stockman & Plummer, 1998; Wu et al., 1996) but perhaps after an early stage of surround inhibition (Chen et al., 1993). Our results, which show a substantial loss of low-frequency sensitivity for monochromatic flicker before the nonlinearity, also suggest that the nonlinearity occurs after or coincident with the action of surround inhibition. 

Surround inhibition within the retina comes mainly from horizontal cells, which provide a mixed M- and L-cone inhibitory surround (Dacey, Lee, Stafford, Pokorny, & Smith, 1996; Dacheux & Raviola, 1990). For midget bipolar and midget ganglion cells in the fovea, these mixed surrounds are effectively chromatically-opponent (Boycott, Hopkins, & Sperling, 1987; Lennie, Haake, & Williams, 1991; Paulus & Kröger-Paulus, 1983) simply because the centers of midget cells are fed by single M- or L-cones (Calkins, Schein, Tsukamoto, & Sterling, 1994; Kolb & Dekorver, 1991). 

Consequently, the compressive nonlinearity is not acting on pure L- or M-cone signals but on postreceptoral signals that have been shaped by chromatic and spatial opponency. We can further segregate these postreceptoral signals into four chromatic types depending on the type of cone feeding the center and whether the signals are ON or OFF (e.g., Dacey et al., 2000; Dacey & Packer, 2003; Schiller, 1992; Werblin & Dowling, 1969). Thus, the signals, and the percepts with which they might be loosely associated are RED: [L − (LM)]_ON and [M − (LM)]_OFF and GREEN: [M − (LM)]_ON and [L − (LM)]_OFF. See Eskew (2008, 2009) for a recent consideration of unipolar and bipolar chromatic mechanisms. 

We have argued that the hue shift from red towards yellow is consistent with a compressive nonlinearity decreasing the mean red signal. Accordingly, the compression could be affecting the [L − (LM)]_ON signal, the [M − (LM)]_OFF signal, or both. Alternatively, the distortion could be at a still higher level where the four postreceptoral signals are somehow combined or compared to generate the red-green chromatic percept. We do know that the signals at f_c that reach the nonlinear site can be as high as 40 Hz (see Figure 3), which means that the site must be at a stage where the representation of high frequency flicker is still present. 

Our argument that the change in hue is consistent with the primary nonlinearity being in a chromatic channel is complicated by the results of the matching experiment plotted in the right-hand panels of Figure 6, which show that the matching stimuli have both a chromatic and a luminance component in cone-contrast space. The chromatic component, which is in the chromatic direction that increases M-cone contrast and decreases L-cone contrast, is consistent with a compressive nonlinearity that differentially reduces L-cone contrast. However, the luminance component, which is in the direction that increases both L- and M-cone contrast, is consistent with an expansive nonlinearity. In the next paper, we focus specifically on the brightness enhancement that accompanies flicker and is probably due to an expansive nonlinearity (Petrova, Henning, & Stockman, submitted). 

Our results suggest that the nonlinearity is followed by a more central two-stage filter with a cutoff frequency of about 3 Hz. Such a filter is comparable to the chromatic TCSFs measured by Wisowaty (1981), who asked observers to set the threshold for perceiving the chromatic red-green alternation produced by alternating 546- and 656-nm lights (rather than setting the threshold for seeing flicker of any type). It may be that access to the chromatic information required to perform this task (as opposed to simple flicker detection) is available only after the late filter. Studies investigating the temporal properties of the brightness and color changes induced into a steady light by a flickering surround show that the induction effects are restricted to frequencies below 3 Hz (D'Antona & Shevell, 2006; De Valois, Webster, & De Valois, 1986). This may suggest that the lateral interactions that induce brightness and color changes occur central to the late filter identified in our experiments. 

Chromatic and achromatic pathways

Given our assumption that the primary nonlinearity resides within a chromatic pathway, and given our choice of stimuli, the attenuation of the early filter derived using the sandwich model should correspond to the response of the chromatic pathway to monochromatic flicker. Because monochromatic flicker generates signals in the center and surround of chromatic mechanisms that are antagonistic, we should expect the filter estimate to exhibit strong low-frequency attenuation—as indeed we find. Had we used chromatic flicker, which generates more synergistic signals between the chromatic center and surround, the low-frequency attenuation should have been less. We can estimate the low-frequency attenuation of chromatic stimuli by the early filter by log differencing the late filter estimates (colored symbols, Figure 7) from the chromatic TCSFs (grey symbols, Figure 3). The differences are shown as open symbols in the left panels of Figures 10 (DP) and 11 (GBH), where they are compared with the previous monochromatic estimates (colored symbols, continuous red lines) from Figure 8. The alignments of the open symbols and the curves fitted to them (black lines) have been chosen to characterize the differences between the low-frequency attenuation of the monochromatic and chromatic stimuli. We have assumed that the logarithmic low-frequency attenuation of the chromatic stimuli (black lines) is the same as the attenuation of the monochromatic stimuli (red lines) but reduced by a scaling factor. For DP the attenuation of chromatic stimuli is reduced by 40% (to 60%) for GBH by 67% (to 33%). 

Given again our assumption that the primary nonlinearity resides within a chromatic pathway, our estimate of the high-frequency attenuation of the late filter should be based on the differences between the early filter and the chromatic TCSFs, as we argued above. These differences, which were shown previously in Figure 7, have been replotted in the right-hand panels of Figures 10 (DP) and 11 (GBH) as colored symbols fitted again with the best-fitting two-stage low-pass filters (red lines). We can gain some insight into the late filter in the pathway that signals monochromatic or equichromatic flicker by log differencing the early filter and the monochromatic TCSFs. Those differences are shown as open symbols in the right panels of Figures 10 (DP) and 11 (GBH). The lines fitted to the open symbols all have a slope on the logarithmic coordinates of −1 (black lines) and the lines and data have been vertically shifted so that the curves align with the chromatic curves at 2.5 Hz. A slope of −1 is consistent with the late filter in the monochromatic pathway's being a one-stage low-pass filter in contrast to the filter in the chromatic pathway, which is two stage. 

It is interesting to note that even though the overall chromatic TCSFs (Figure 3, grey symbols) are approximately low pass in shape, the early filters for chromatic flicker are actually band pass. The change from an early band-pass function to an overall low-pass function is due to the late filter, the cutoff frequency of which is such that it produces a relative increase in low-frequency sensitivity in a way that offsets and balances the early low-frequency attenuation. We note that the band pass nature of the early filter for chromatic flicker is consistent with models of chromatic opponency in which the center mechanism is fed by a single L- or M-cone but the surround is fed by a mixture of L- and M-cones (e.g., Boycott et al., 1987; Calkins et al., 1994; Kolb & Dekorver, 1991; Paulus & Kröger-Paulus, 1983). A mixed surround means that there should, as we find, still be some low-frequency attenuation even for chromatic flicker—consistent with the low-frequency chromatic attenuation implied by the impulse response functions measured by Eskew, Stromeyer, and Kronauer (1994). It is interesting to note that the differences between the chromatic and monochromatic TCSFs in Figure 3, which reflect the differences between the open and colored symbols in Figures 10 and 11, are approximately independent of the mean target radiance, which suggests that, if the properties of the chromatic and monochromatic pathways change with level, they change together. The primary differences between the two pathways in terms of the TCSFs are (a) increased low-frequency attenuation in the achromatic pathway and (b) an additional central filter in the chromatic pathway. 

Characteristics of the nonlinearity

We cannot determine the shape of the underlying nonlinearity precisely from our observations and data, but we can infer its general characteristics. From the subjective observations that contrast-modulated, sinusoidally-flickering, 650-nm light changes in hue from red to yellow when the contrast modulation is high, we can infer that the primary nonlinearity in our range of measurements is more likely to be compressive than expansive. We can also use the data relating the output contrast after the nonlinearity to the input contrast before it (see Figure 6) to test the feasibility of different nonlinear forms. We generated a range of nonlinear forms that included smooth expansive nonlinearities, smooth compressive nonlinearities, expansive and compressive nonlinearities that became hard saturating nonlinearities (i.e., functions limited by an abrupt ceiling), and nonlinearities that were smoothly expansive at low input levels and smoothly compressive at high levels such as that exhibited by the Naka-Rushton equation. We then used each form to generate predictions for the data of Figure 6 using MatLab and Simulink (MathSoft) for all calculations and to generate the Fourier transform of the input and output signals before and after the nonlinearity. 

To generate the predictions, we started with a particular nonlinear input-output function, such as the example shown in the upper panel of Figure 12. This particular example has the form: 

 

where x is the input to the nonlinearity and y is the output. The units are arbitrary. We assumed that as the time-averaged 650-nm target radiance increases, the DC (mean) input level (which we call the “offset”) also rises, thus moving up the nonlinear input-output function. In the simulation, we varied the DC offset from 0.0 to 2.5 and at each level varied the modulation of the contrast-modulated input from 0% to 100%, where 100% was the maximum modulation possible at that DC level (for example, at a DC level of one, the minimum and maximum of the contrast-modulated input signal are zero and two, respectively). At each combination of DC level and modulation, we calculated the contrast of the distortion product at f_m; that is, the ratio of the amplitude of the distortion product at f_m to the total DC after the nonlinearity. The simulated contrast of the signal at f_m is plotted as a function of the DC offset and input modulation in the lower panel of Figure 12. 

It is important to recognize that the nonlinearity in these simulations affects the contrast-modulated flicker in two ways: first, it introduces the f_m-intermodulation distortion product in which we are interested and second, it introduces a steady-state (DC) term that changes the mean output level around which the stimulus flickers. The contrast of the f_m distortion product, which we can reasonably assume determines its detectability and is plotted in Figure 12, depends on both of these products. If the nonlinearity is expansive, the amplitude of the f_m term has a positive coefficient, whereas if it is compressive, it has a negative coefficient. Similarly, the mean (DC) level is increased by the expansive nonlinearity and decreased by the compressive one. Thus, the contrast and therefore the relative detectability of the f_m-component depend on the interplay between these components. 

Any candidate nonlinearity must account for the features of Figure 6. The contrast of the distortion product must grow with input modulation and the functions must get steeper as the 650-nm mean radiance increases from 9.10 to 10.33 log quanta s⁻¹ deg⁻². However, at the highest levels, between 10.33 and 10.93 log quanta s⁻¹ deg⁻², the function describing the nonlinearity, certainly for DP and possibly for GBH, must get shallower again. We found that most of the tested nonlinear forms could account for some but not all features of the input contrast to output contrast functions. The simplest nonlinearity that could easily account for the functions (including the falloff for DP) was one that was compressive at low input levels but that reached a ceiling at high levels (like the example shown in the upper panel of Figure 12). The changes in the simulated modulation versus f_m contrast functions with DC offset in the lower panel of Figure 12 mirror the changes in the output contrast with input modulation in Figure 6. These similarities suggest that the nonlinearity of Figure 12 is a reasonable model of the underlying nonlinearity. 

These simulations and modelling are largely descriptive, but they allow us to draw some general conclusions about the nonlinearity. In order to simulate the decrease in the input versus output contrast functions between the two highest levels for DP, we think that the nonlinearity must have an abrupt ceiling. A purely logarithmic nonlinearity (without a ceiling), for example, generates contrast functions that approximate to an asymptotic level without decreasing. 

We note that the inferred nonlinearity of Figure 12 relates only to the four 650-nm mean radiance levels from 9.10 to 10.33 log quanta s⁻¹ deg⁻² used in the experiments. At lower levels, the nonlinearity may have a different form or may be missing. However, the critical-fusion-frequency data of Figure 2 indicate that distortion can still be detected down to 650-nm radiances as low as 6.5 log quanta s⁻¹ deg⁻². Interestingly, both subjects reported that below about 8.0 log₁₀ quanta s⁻¹ deg⁻², the distortion is seen solely as a brightness change from dark red to bright red rather than as a hue and saturation change. Thus, at low light levels the effective nonlinearity might be expansive. 

Summary

A nonlinearity in the L-M chromatic pathway that causes a red-appearing 650-nm light to look more yellow when flickered has been used as an internal landmark at which to dissect the pathway into early (prenonlinearity) and late (post-nonlinearity) stages. The early temporal stage acts like a band-pass filter with sensitivity losses at higher temporal frequencies that are consistent with the nonlinearity occurring relatively early in the visual pathway. The losses at lower frequencies, however, suggest that the nonlinearity must be after the action of surround inhibition. The late stage acts like a simple two-stage low-pass filter with cutoff frequencies around 3 Hz. Modelling suggests that the form of the nonlinearity is smoothly compressive but with a hard saturating limit at high input levels. The form of the nonlinearity may be expansive at low input levels. 

Acknowledgments

This work was supported by the BBSRC. Preliminary work was supported by NIH. We thank Rhea Eskew and two anonymous reviewers for helpful comments. 

Commercial relationships: none. 

Corresponding author: Andrew Stockman. 

Email: [email protected] 

Address: UCL Institute of Ophthalmology, University College London, London, England. 

Footnotes

¹   Strictly m in Equation 3 (and 4) is not exactly the modulation of Equation 1 but it is very close to it in our stimuli when f_m ≪ f_c.