The temporal characteristics of the early and late stages of L- and M-cone pathways that signal brightness

Abstract

Abstract

Abstract:

Abstract  Flickering 560-nm light appears brighter and less saturated than steady light of the same average intensity. The changes in appearance are consistent with the visual signal's being distorted at some nonlinear site (or sites) within the visual pathway at which new temporal components, not part of the original waveform, are produced. By varying the input stimulus to manipulate these new temporal components—called distortion products—and measuring our observers' sensitivity in detecting them, we derived the temporal attenuation characteristics of the early (prenonlinearity) and late (post-nonlinearity) stages of the L- and M-cone pathway that signals brightness. We found that the early stage acts like a band-pass filter peaking at 10–15 Hz with sensitivity losses at both lower and higher frequencies, whereas the late stage acts like a two-stage low-pass filter with a corner frequency near 3 Hz. Although brightness is often associated with the fast achromatic or luminance pathway, these filter characteristics, and particularly those of the late filter, are consistent with comparable features of the L-M chromatic pathway that produce mainly chromatic distortion products (Petrova, Henning, & Stockman, 2013). A plausible site for the nonlinearity is after surround antagonism from horizontal cells. Modeling suggested the form of the nonlinearity to be initially expansive but possibly with a hard limit at the highest input levels.

Introduction

Flickering a light of constant time-averaged intensity may enhance its brightness (e.g., Bartley, 1938, 1939, 1951a, 1951b; Brewster, 1838; Brücke, 1848) or change its hue (e.g., Ball, 1964; Ball & Bartley, 1966, 1971; Bartley & Nelson, 1960; Bleck & Craig, 1965; Petrova, Henning, & Stockman, 2013; Stewart, 1887; van der Horst & Muis, 1969). Such effects seem to arise from nonlinearities that distort the representation of the visual input within the visual pathway. 

Flicker-induced changes in brightness differ from flicker-induced changes in hue in several ways: First, though the two effects can occur together, they can also occur separately. For instance, a deep-red flickering target looks yellower and less saturated than a steady field of the same time-averaged intensity but with little apparent change in brightness, whereas a flickering light near 560 nm looks brighter than a steady field with little or no change in hue (e.g., Ball, 1964; van der Horst & Muis, 1969). Second, the two effects have very different spectral sensitivities. For luminance-equated lights, the hue change is strongly dependent on wavelength whereas the brightness enhancement is not (van der Horst & Muis, 1969; Walters & Harwerth, 1978). In fact, the threshold spectral sensitivity for brightness enhancement approximately follows the V(λ) photopic spectral sensitivity (Walters & Harwerth, 1978). Third, the brightness enhancement is consistent with an expansive nonlinearity (e.g., Wu, Burns, Reeves, & Elsner, 1996), whereas the hue change in the red-green range is consistent with a compressive nonlinearity (Petrova et al., 2013; van der Horst & Muis, 1969) thus the two effects are probably generated by different nonlinearities. These distinctions raise the intriguing possibility that the two nonlinearities lie in different postreceptoral pathways, one, perhaps, in a chromatic pathway and the other, perhaps, in an achromatic or luminance pathway. In our previous paper, we investigated the hue change induced by 650-nm flicker (Petrova et al., 2013). In this paper, we focus on brightness enhancement, which is sometimes referred to as the Brücke-Bartley effect. 

The principal stimulus used in these experiments was a contrast-modulated sinusoidally flickering 560-nm light that, when the flicker contrast is high, appears brighter than steady light of the same radiance and wavelength. A wavelength of 560 nm was chosen because, as our own observations confirm, flicker near that wavelength induces little change in hue (Ball, 1964; van der Horst & Muis, 1969). The contrast-modulated 560-nm stimulus consists of a sinusoidal carrier (of frequency f_c Hz) the contrast of which is sinusoidally modulated at a much lower modulation frequency (f_m Hz). This stimulus has no component at f_m Hz but produces brightness changes at that frequency. We assume that the enhancement in brightness is caused by the distortion of the visual signal at some internal nonlinear site (or sites) at which new components, called distortion products, are generated. In the contrast-modulated stimulus, components at the frequencies of the distortion products do not exist until after the nonlinearity. Consequently, distortion products can be used to distinguish characteristics of the “early” visual processes that precede the nonlinearity from “late” processes that follow it. 

In this context, the visual system is treated as a linear-nonlinear-linear “sandwich” of stages (e.g., Burns, Elsner, & Kreitz, 1992; Burton, 1973; Chen & Makous, 1990; Chen, Makous, & Williams, 1993; MacLeod, Williams, & Makous, 1992; Marmarelis & Marmarelis, 1978; Petrova et al., 2013; Spekreijse & Reits, 1982; Stockman & MacLeod, 1986; Stockman, MacLeod, & Lebrun, 1993; Stockman & Plummer, 1998; Trimble & Phillips, 1978; Victor & Shapley, 1980; Victor, Shapley, & Knight, 1977; Wu et al., 1996). Several aspects of our work are directly comparable with the previous work of Wu et al. (1996), as noted below. 

Figure 1 illustrates the application of the sandwich model to the case of brightness enhancement. The left-hand column of Figure 1 illustrates one cycle of the temporal waveform of the visual input (Panel S1) and the presumed representation of the waveform after it has been modified by successive stages of the linear-nonlinear-linear sandwich (subsequent panels). The right-hand column (Panels C1 to C4) shows the amplitude spectra corresponding to the temporal waveforms shown in the left-hand column (i.e., the amplitudes of their constituent sinusoidal components). The waveforms and their spectra are intended merely to indicate the information present at each stage and should not be taken as the form in which the information is transmitted. 

Figure 1

 

The figure illustrates an example of an expansive nonlinearity (N) separating early (F1) and a late (F2) linear filters operating on a contrast-modulated sinusoidally flickering stimulus. (One modulation cycle of the stimulus is shown in Panel S1.) The amplitude spectrum of the stimulus is shown in Panel C1. Amplitude spectra at successive stages are shown in Panels C2–C4 and as corresponding time “waveforms” in Panels S2–S4. The nonlinearity (N) introduces new components at harmonics and at sum and difference frequencies of the Fourier components of the stimulus. The colors of the icons above S1–S4 represent the approximate brightness and saturation changes that might be expected at the peaks and troughs of the contrast-modulated flicker. (The Fourier components are illustrative; they are not drawn to scale.) This figure can be compared with Figure 1 of Petrova, Henning, and Stockman (2013), which illustrated the compressive case.

Panel S1 shows one modulation cycle of the contrast-modulated stimulus, which is produced by sinusoidally flickering a light at f_c Hz and sinusoidally modulating its contrast at a lower frequency of f_m Hz. Panel C1 at the top right illustrates the amplitude spectrum of the stimulus, which comprises three high-frequency sinusoidal components: the “carrier” frequency at f_c Hz and two “side-bands” at f_c − f_m and f_c + f_m Hz. The component at 0 Hz corresponds to the mean (“DC”) level around which the light is modulated. At this stage there is no sinusoidal component at f_m Hz. Consequently, all an observer should perceive is flicker at f_c varying in strength at f_m. 

The temporal characteristics of the early linear stages before the visual nonlinearity are exemplified in Figure 1 by a low-pass filter (Panel F1). The filter attenuates the components of the visual stimulus (Panel C1) to produce the intermediate spectrum (Panel C2). The intermediate stimulus, when f_m ≪ f_c, is approximately an attenuated version of the input stimulus (Panel S2). In the continued absence of a component at f_m, no perceptible change in brightness at f_m should be expected (as illustrated by the unchanging icons above Panel S2). 

The linear early stage is followed in the sandwich model by a nonlinearity (N). The example in Figure 1 shows an expansive nonlinearity that selectively expands high signal intensities relative to low ones, thus selectively enhancing the peaks of the waveform (Panel S3). The contrast-modulated flicker signal passing through the nonlinearity has its frequency spectrum altered by the nonlinearity (Bedrosian & Rice, 1971). Components at higher harmonics of the carrier and side bands are produced as well as components called intermodulation distortion products. Among the latter, there is one at f_m and one at 2 f_m Hz (Panel C3). It is the component at f_m Hz that we wish to consider as the distortion product that produces the brightness changes illustrated by the icons above Panel S3. For simplicity, we assume that the nonlinearity is static, in that its input-output relation depends on the instantaneous value of the input, not on previous inputs, and that the form of the nonlinearity is independent of temporal frequency. We further assume that it is the distortion product at the frequency of f_m Hz that produces the brightness change that the observers see at that frequency. 

The collective temporal characteristics of the linear processes after the nonlinearity are represented in Figure 1 by a second low-pass filter (Panel F2). This late filter, like the early one, selectively attenuates frequencies near f_c but not enough for them to fall below the flicker threshold (Panel C4). Thus, brightness enhancement at f_m is accompanied by flicker near f_c, as we find (see Figure 2, below). The goal of this work is to determine the forms of both the early and the late filters. We emphasize that these filters are hypothetical and depend on the applicability of the sandwich model to the phenomenon of flicker-induced brightness enhancement. 

Figure 2

 

Critical fusion frequencies, f_c, for the carrier of a contrast-modulated flicker as a function of the time-averaged radiance (log₁₀ quanta s⁻¹deg⁻²) of a 560-nm target. The sinusoidal flicker at f_c was 100% contrast modulated at f_m = 0.5 Hz and with the overall modulation fixed at 92%. The frequencies for detecting flicker at f_c are shown as white dotted circles and for detecting brightness change at f_m as green dotted diamonds. The frequency scale is linear. The error bars indicate ±1 standard error of the mean (SEM). The four downward pointing colored triangles just above the abscissae indicate the four radiance levels used in Experiments 1 and 2.

Versions of the sandwich model have been explored in analyzing the nonlinear interactions that produce hue changes (Petrova et al., 2013) and our techniques have been described there in full. The work by Wu et al. (1996) is also directly relevant. They estimated the magnitude of brightness enhancement at a series of carrier frequencies and stimulus modulations by matching one cycle of the amplitude-modulated flicker against one cycle of a sinusoidally varying matching light, from which they inferred that the early part of the visual pathway signaling brightness is consistent with an accelerating nonlinearity preceded by a broadly tuned band-pass filter peaking at about 16 Hz. Their technique, however, did not yield direct estimates of the characteristics of a late filter. 

Given that we see mainly an increase in brightness, it seems reasonable to suppose that we are dissecting a brightness pathway. Whether brightness is mediated by the so-called achromatic or luminance pathway or by some other pathway, however, is a question to which we will return in the General discussion. 

General methods

This research adhered to the tenets of the Declaration of Helsinki. Many of the experimental details can be found in our companion paper (Petrova et al., 2013). We describe here the essential characteristics of the methods. 

Apparatus

A Maxwellian-view optical system with a 2.5-mm entrance pupil illuminated by a 900-W Xe arc lamp was used to produce the stimuli. Wavelengths were selected by interference filters with full width at half-maximum bandwidths of between 7 and 11 nm (variously manufactured by Ealing or Oriel). The radiance of each channel was controlled by the insertion of fixed neutral density filters (manufactured by Ealing or Oriel or Melles Griot) and by the rotation of circular variable 3-log₁₀ unit neutral density filters (Rolyn Optics). The flickering waveforms were generated by pulse-width modulation of fast liquid-crystal light shutters running at 400 Hz with rise and fall times faster than 50 μs (Displaytech) thus effectively producing variable width rectangular pulses at a fixed rate of 400 Hz. The pulse width was varied under computer control using programmable timers (Data Translation, DT2819) to produce sinusoidal components at the desired frequencies and at signal modulations up to 92%. 

Observers

The same observers participated as in the experiments of Petrova et al. (2013). Both were authors, one male (GBH) and one female (DP), and both were experienced psychophysical observers with normal color vision and normal (DP) or corrected to normal (GBH) spatial acuity. 

Stimuli

Visual stimuli were centrally fixated target discs of 4° diameter. The flickering target was a 560-nm light that was set, in the main experiments, to one of four time-averaged radiance levels (8.26, 8.86, 9.51, or 10.11 log₁₀ quanta s⁻¹deg⁻², corresponding to 2.16, 2.76, 3.41, and 4.01 log₁₀ trolands). The background was dark. The target was sinusoidally flickered at f_c Hz and its contrast was sinusoidally modulated at f_m Hz to produce “contrast-modulated sinusoidal flicker” (Panel S1, Figure 1). [Occasionally unmodulated sinusoidal flicker (f_m = 0 Hz) was used.] Flicker in both cases was around a given time-averaged radiance. 

Since the waveforms we used are somewhat atypical, we again describe them and the naming conventions we adopted: We refer to the amplitude of the flicker waveform relative to the mean radiance as the “overall modulation,” m, which is defined as the conventional Michelson contrast:  

 

where I_max and I_min are the maximum and minimum radiances of the stimulus, respectively. Thus, for simple sinusoidal flicker the waveform, A(t), is given by:  

 

where R̄ is the mean radiance and f_c is the rate of flicker (Hz). 

In experiments in which contrast-modulated flicker was used, the temporal waveform, A_m(t), is:  

 

where f_c is the carrier frequency, f_m is the modulation frequency (both in Hz), and m is the overall modulation. We call this stimulus “contrast-modulated flicker.” The factor in square brackets is sometimes called the “amplitude modulation” and the amplitude modulation in our experiments always varied sinusoidally between one and zero at f_m Hz; i.e., it was 100% amplitude modulation. 

The flickering component of Equation 3 can be expanded to show that it comprises three sinusoidally flickering terms:  

 

where the components, at f_c Hz, with amplitude R̄m/2 and two sidebands at f_c − f_m and f_c + f_m Hz with half the amplitude of the carrier, are made explicit. (Note that in our experiments the amplitude modulation was always 100% and the overall modulation of the entire waveform, m, was varied with the result that the sidebands always had half the contrast of the carrier.) For further details, see Petrova et al. (2013). 

Calibration

The radiant fluxes of test and background fields were measured daily in the plane of pupil using a calibrated UDT S370 optometer (UDT Instruments, San Diego). The means by which we checked for system linearity is described in our earlier paper (Petrova et al., 2013). 

Procedures

The visual stimulus, focused in the plane of the pupil, was the only visible light source for the observers in an otherwise darkened room. They used their right eye for observation; their left eye was covered. The image of the source in the plane of the observer's pupil was 2.5 mm diameter—always less than the diameter of the natural pupil. The method of adjustment was used to measure visual responses in the experiments. 

In experiments using contrast-modulated stimuli, the observers were asked to find: (a) the modulation depth at which all flicker disappeared and (b) the modulation depth at which brightness changes at the modulation frequency, f_m, disappeared. The results for three runs of each sort were averaged and the mean and standard error for each condition determined. The procedures were similar to those described in Petrova et al. (2013). 

Preliminary experiment: Fusion frequencies for detecting flicker or brightness enhancement

Introduction

In a preliminary experiment, we determined the highest carrier frequency (f_c) at which either flicker at f_c or brightness change at f_m can be detected as a function of target radiance. These are known as critical fusion frequencies. Our goal was to determine the separation between the two fusion frequencies, in part to see if the Talbot-Plateau Law failed at any target radiances. A failure of the Talbot-Plateau law (Plateau, 1835; Talbot, 1834), which holds that a light flickering above the fusion frequency should appear identical to a steady light of the same time-averaged intensity, would be found if the fusion frequency for detecting flicker near f_c was lower than that for detecting the brightness change at f_m. 

Methods

We used a contrast-modulated stimulus and fixed the modulation frequency, f_m, at 0.5 Hz. In separate blocks of trials, the time-averaged radiance of the contrast-modulated 560-nm target was varied in steps of approximately 0.45 log₁₀ unit from 5.81 to 10.11 log₁₀ quanta s⁻¹deg⁻². At each radiance, observers varied f_c in 0.5 Hz steps to find the highest frequency at which either flicker or brightness change was just perceptible. Three flicker settings were made first followed by three brightness settings. 

Results

Figure 2 shows the threshold frequencies (linear scale) for flicker (white dotted circles) and brightness change (green dotted diamonds) as a function of log₁₀ radiance. Data for each observer are shown separately and the error bars indicate ± one standard error. 

Curves for both observers increase steadily with increasing radiance from about 6.0 to 9.0 log₁₀ quanta s⁻¹deg⁻². The curves for detecting a brightness change begin to flatten with further increases in radiance at a slightly lower radiance than those for detecting flicker but both appear to decrease slightly at the very highest radiance used. 

Discussion

One reason for this experiment was to discover whether the Talbot-Plateau fails at any target radiance as it does with S-cone isolating stimuli (Stockman & Plummer, 1998). Although the separation between the brightness and the flicker fusion frequencies is smaller for GBH than for DP, the fusion frequencies for seeing a brightness change are always below those for seeing flicker. Consequently, the results are consistent with the Talbot-Plateau law. The gap in frequency between reporting flicker and reporting brightness change suggests that near-threshold flicker is relatively unaffected by the nonlinearity. This result indicates that at any frequency, there will be a range of near-threshold modulations within which flicker will be visible but the distortion product that produces brightness change will be undetectable and that we may safely assume small-signal linearity near the threshold for flicker. 

It may be helpful to outline what observers typically see in these experiments as the carrier frequency at f_c is increased. Below the fusion frequency for detecting both flicker near f_c and the brightness change at f_m, observers see mainly a brightness change at f_m Hz (as illustrated by the icons above Panel S4 in Figure 1) superimposed on a slow waxing and waning of the strength of the flicker at the same low frequency, with the increased brightness coinciding with the increased amplitude of flicker. Below the fusion frequency for flicker detection but above that for detecting the brightness change, observers see only the waxing and waning of the flicker strength at f_m Hz with no brightness variation. Lastly, above the fusion frequency for detecting both the flicker and the brightness change, the observer sees only a steady non-flickering target. Experienced observers can dissociate the brightness change from flicker under most conditions, except when f_c and f_m are both low in frequency where they become difficult to distinguish (see below). 

Experiment 1: TCSFs for brightness change, flicker, and chromatic flicker as a function of f_c

Introduction

In Experiment 1, we measured temporal contrast-sensitivity functions (TCSFs) using contrast-modulated stimuli with f_m fixed at 0.5 Hz and variable f_c Hz. Observers varied the overall modulation to find thresholds for detecting the 0.5-Hz brightness change at each f_c. 

Given that the 0.5-Hz brightness component is present in the system only after the nonlinearity and that observers vary the modulation to produce the same threshold 0.5-Hz brightness signal at the input to the late filter for all f_c Hz, the effect of the late filter on the brightness-change thresholds will be a constant factor, possibly different for different radiances but independent of f_c. The TCSF, therefore, provides a direct measure of the attenuation characteristics of the early filter (F1 in Figure 1). For further details see our earlier paper (Petrova et al., 2013). 

In this experiment, we also compare the brightness-change thresholds with conventional TCSFs measured with sinusoidal monochromatic (560-nm) and chromatic flicker. The conventional TCSFs are also used later in the General discussion to estimate missing parts of the early and late filter characteristics (see below). 

Methods

The three experiments were carried out at mean radiances of 8.26, 8.86, 9.51, and 10.11 log₁₀ quanta s⁻¹deg⁻² indicated by the colored triangles just above the abscissae of Figure 2. 

Brightness enhancement temporal sensitivity measurements

The primary target was the 4° diameter, contrast-modulated, 560-nm light of Equation 3 with the carrier at f_c and contrast modulation, f_m, fixed at 0.5 Hz. Observers varied the overall modulation (m in Equation 3 or 4) in either 0.02 or 0.10 log₁₀ unit steps to find the threshold for just detecting a brightness change at f_m. The carrier frequency, f_c, was varied from 5 to 60 Hz. Below an f_c of 5 Hz, flicker at f_c, like the brightness change at f_m, appeared as changing brightness, so that it was difficult to separate the two. 

Conventional temporal sensitivity (flicker) measurements

We measured conventional TCSFs with simple sinusoidally flickering stimuli (Equation 2) with the same 4° 560-nm target and the same observers. At each f_c, observers varied the overall modulation (m in Equation 2) in either 0.02 or 0.10 log₁₀ unit steps to set the modulation of the target at which they could just detect its flicker. Contrast modulation was not used for these measurements, because when f_c was low, the frequency components near f_c Hz and the distortion products near f_m were similar in frequency and appearance and threshold settings consequently difficult. At higher f_c, the fusion thresholds for contrast-modulated flicker with f_m fixed at 0.5 Hz (not shown) were very similar to the thresholds with f_m fixed at 0 Hz shown as open symbols in Figure 3. 

Figure 3

 

The log₁₀ modulation sensitivities for DP (left-hand column) and GBH (right-hand column) for detecting either monochromatic (equichromatic) (open symbols) or chromatic (equiluminant) (gray symbols) flicker at f_c, or for detecting the brightness enhancement at f_m (colored symbols) all plotted as a function of f_c (logarithmic axis). Each row shows results for the different time-averaged radiances shown in the top corner of the panels in the right-hand column: 8.26 (inverted dark brown triangles), 8.86 (dark green circles), 9.51 (green diamonds), and 10.11 (light green squares) log₁₀ quanta s⁻¹deg⁻². Unmodulated sinusoidal flicker at f_c was used for the monochromatic and chromatic flicker measurements. For the brightness-change measurements, the sinusoidal flicker at f_c was contrast modulated at f_m = 0.5 Hz. Error bars indicate ±1 SEM. The chromatic flicker was made up of luminance-equated 560-nm and 650-nm targets flickering in opposite phase. At each level, the overall time-averaged luminance of the combined 560- and 650-nm lights was set to be the same as the time-averaged luminance of the 560-nm targets used in the monochromatic measurements using the Sharpe et al. luminous efficiency estimates (Sharpe et al., 2005, 2011). (At the 10.11 log₁₀ quanta s⁻¹deg⁻² level we had to use a 630-nm target to cancel 560-nm flicker because the brightest 650-nm target was too dim.) In increasing order of the radiance of the 560-nm target for the three lowest radiances, the 650-nm equiluminant quantal radiances were 1.16, 1.22, and 1.37 more than the 560-nm radiances for DP and 0.98, 0.98, and 1.08 more for GBH. At the highest radiance of the 560-nm target, the 630-nm equiluminant quantal radiances were 0.74 more than the 560-nm radiance for DP and 0.65 more for GBH. The modulations for the chromatic flicker are those of 560-nm component (or 650-nm component) alone.

Temporal contrast sensitivity for detecting equiluminant chromatic flicker

In these experiments, we first determined a series of equiluminant or “chromatic” stimuli with which to measure the chromatic TCSFs by superimposing a 4° 650-nm target on the 4° 560-nm target, both flickering sinusoidally but temporally 180° out of phase. The 560-nm and 650-nm components were luminance-equated individually for each observer by asking the observers to cancel the flicker of the superimposed targets. To do this, two 30-Hz targets were superimposed in opposite phase. Both were set to the maximum 92% modulation, and the radiance of the 560-nm light was fixed. The observers then varied the radiance of the 650-nm light to null the perception of flicker. [Observers were also able to make small phase adjustments (in steps of 2 or 10°) to perfect the null, but the optimum phase for the null was always near 180°.] At the highest 560-nm radiance of 10.11 log₁₀ quanta s⁻¹deg⁻² we had to use a 630-nm target to cancel 560-nm flicker because our most intense 650-nm target was insufficient. The mean time-averaged luminances of the combined luminance-equated 560-nm and 650-nm (or 630-nm) stimuli were then set to four different time-averaged luminances of the 560-nm target alone using the Sharpe, Stockman, Jagla, and Jägle luminous efficiency estimates (2005, 2011) now adopted as a standard by the CIE (2006). Note that the addition of the 650-nm component changed the appearance of the steady target from yellow-green to an orange-yellow. As for the monochromatic measurements, the target for chromatic flicker was sinusoidally flickered at f_c Hz (i.e., f_m = 0 Hz). The assumption behind this type of measurement is that flicker photometric cancellation silences the luminance pathway, so that flicker detection with the luminance-equated stimuli must be by some “chromatic” pathway (for review, see Stockman & Brainard, 2009). Hence, we term them chromatic TCSF measurements. 

The chromatic TCSFs were then measured by jointly adjusting the modulation (Equation 2) of the out-of-phase luminance-equated 560- and 650-nm stimuli while maintaining the appropriate equiluminant ratio. Adjustments were made in steps of 0.02 or 0.10 log₁₀ unit to find the modulation at which the observers could just detect flicker. 

Results

Figure 3 shows the log modulation sensitivity (the reciprocal of threshold) for detecting the brightness change at f_m (filled colored symbols), for detecting monochromatic (equichromatic) flicker at f_c (open symbols), and for detecting chromatic (equiluminant) flicker at f_c (gray symbols)—all plotted as a function of f_c (Hz, logarithmic axis). Results for DP are shown in the left-hand column, those for GBH, in the right-hand column. The time-averaged radiance (log₁₀ quanta s⁻¹deg⁻²) for each row (indicated at the top right of each panel in the right-hand column) increases from 8.26 at the top to 10.11 at the bottom. 

The results in Figure 3 show that the brightness-change TCSFs (filled colored symbols) particularly for DP fall off slightly less steeply with increasing frequency than the monochromatic 560-nm flicker TCSFs (open symbols). (The differences are plotted as open symbols in Figure 10, below.) The chromatic TCSFs (gray symbols) have a limited range at high frequencies, but where they overlap with the brightness change TCSFs, the chromatic TCSFs fall off much more steeply with increasing frequency. (The differences are plotted as gray symbols in Figure 7, below.) 

Discussion

We expected the brightness change TCSFs to be shallower than the chromatic flicker TCSFs, because the brightness change TCSFs depend on a fixed-frequency distortion product and thus are not affected by attenuation by the late filter. The relatively smaller differences in high-frequency slope found between the brightness-change TCSFs and the monochromatic TCSFs may suggest that the brightness change and monochromatic flicker are detected by different pathways (see General discussion). The monochromatic TCSFs (open symbols in Figure 3) are band pass in form. They peak between about 5 and 10 Hz and fall off at low frequencies with slopes of about 0.8 log₁₀ unit per decade and at high frequencies with slopes of more than 4 log₁₀ units per decade. The low- and high-frequency losses are consistent with other achromatic and monochromatic flicker TCSFs (e.g., De Lange, 1958; Green, 1969; Kelly, 1961, 1973; Petrova et al., 2013; Varner, Jameson, & Hurvich, 1984). By contrast, the chromatic TCSFs (gray symbols) are low pass in form and fall off in sensitivity above 5 to 10 Hz more steeply than the monochromatic TCSFs. These data are similar in shape to other TCSFs measured with equiluminant chromatic flicker (e.g., Kelly, 1975; Kelly & van Norren, 1977; Varner et al., 1984). Given the difficultly of perfectly equating the luminances of counter-phase flickering lights across the entire stimulus area, the chromatic TCSFs may contain some luminance-flicker artifacts at the highest temporal frequencies where the chromatic signal is so much smaller than the luminance signal (e.g., Kelly & van Norren, 1977; Varner et al., 1984). 

As in our companion paper, the TCSFs will be used to estimate the characteristics of the early and late filter at frequencies at which they could not be determined from the brightness-change measurements (see General discussion). 

Experiment 2: TCSFs for brightness change as a function of f_m

Introduction

The contrast sensitivity for detecting the brightness change at f_m as a function of f_m for a fixed f_c allows us to extract an estimate of temporal characteristics of the filter after the nonlinearity—the late filter. The contrast-modulated stimuli generate a distortion product at f_m Hz after the nonlinearity, which is seen as a brightness change. The observers' task, as before, was to detect the slow brightness change at f_m Hz. Since the stimuli have the same f_c and f_m ≪ f_c, they are similarly affected by the linear (early) filter that precedes the nonlinearity. Consequently, the changes in sensitivity with f_m are determined by the nonlinearity and the late filter. (A supplementary experiment described in the next section is required to estimate the temporal characteristics of the late filter independent of the effects of the nonlinearity.) 

Results

Figure 4 shows the TCSFs for detecting brightness change as a function of f_m (logarithmic axes). Data for DP are shown in the upper panel and those for GBH in the lower panel. Functions for four radiance levels (log₁₀ quanta s⁻¹deg⁻²) are shown: 8.26 (dark triangles), 8.86 (dark green circles), 9.51 (light green diamonds), and 10.11 (light green squares). For these measurements, the 560-nm target was contrast modulated at various f_m (Hz) with a fixed carrier frequency, f_c, of 30 Hz. 

Figure 4

 

The log₁₀ modulation sensitivities for DP (upper panel) and GBH (lower panel) for detecting brightness change at f_m as a function of the contrast modulation frequency, f_m (logarithmic axis) at four time-averaged target radiances: 8.26 (inverted dark triangles), 8.86 (dark green circles), 9.51 (green diamonds), and 10.11 (light green squares) log₁₀ quanta s⁻¹deg⁻². The sinusoidal flicker at f_c = 30 Hz was contrast modulated at f_m Hz. Error bars indicate ±1 SEM.

The results extend only up to 5 Hz because at higher modulation frequencies observers could not detect a brightness change varying at f_m. The TCSFs are mainly low pass in form falling at about 0.5 log₁₀ units per decade. Sensitivity increases between the two lowest radiances but then decreases with further increases in radiance. 

Discussion

As noted in our companion paper, the shapes of these functions reflect the characteristics of both the nonlinearity and the post-nonlinearity (late) filter. Because f_m ≪ f_c, and f_c is fixed, the vertical positions of the functions, but not their shapes, are affected by the early filter. Sensitivity at each radiance is thus represented on logarithmic coordinates up to an unknown constant vertical displacement in terms of the overall modulation of the stimulus at the input to the nonlinearity. In the next section, we estimate the relation between the contrast of the distortion product at the output of the nonlinearity and the overall modulation, m, of the contrast-modulated stimulus at the input to the nonlinearity, so that the late filter (F2 in Figure 1) can be characterized. 

To estimate the characteristics of the hypothesized late filter, we need to know how the amplitude of the distortion product at f_m Hz created by the nonlinearity is related to the overall modulation at the input to the nonlinearity, because it is the overall modulation at the input that we manipulate. In other words, we need to know, at each of the mean radiances we used, how changing the overall modulation at the input to the nonlinearity affects the contrast of the distortion product at f_m at its output. A technique for determining this is described in the following supplementary experiment. 

Supplementary experiment: Dependence of brightness change on modulation

Introduction

In this supplementary experiment, we determine how the contrast of the f_m distortion product at the output of the nonlinearity depends on the overall modulation, m, of the contrast-modulated stimulus at the input to the nonlinearity. Knowledge of this relation allows us to extract the late filter characteristics (F2 in Figure 1) from the brightness-change TCSF measurements made as a function of f_m and shown in Figure 4. To measure the size of the brightness enhancement, we use a matching procedure. 

Methods

In a series of preliminary studies, we evaluated various methods of nulling or matching the brightness enhancement. We eventually adopted a side-by-side matching technique in which the 4° target was split into left and right hemifields. The stimulus arrangement is illustrated in Figure 5. In the left hemifield, the observer was presented with the contrast-modulated 560-nm target (with f_c = 30 Hz and f_m = 0.5 Hz). In the right hemifield, the observer was presented with a brightness-matching stimulus. The matching stimulus comprised two superimposed components: (a) a steady 560-nm “pedestal” that had the same time-averaged radiance and appearance as a non-flickering 560-nm light in the left hemifield and (b) a 560-nm light that was sinusoidally flickered at 0.5 Hz and set to the maximum 92% modulation with the same phase as the 0.5-Hz contrast modulation (f_m) of the left hemifield. The combined stimulus in the right half field produced brightness changes that were comparable to those produced by the contrast modulation in the left hemifield. The observers adjusted the time-averaged radiance of the matching light (keeping the 92% modulation constant) to match the brightness of the left hemifield at the peak of its contrast modulation. This was done under computer control using response buttons. Subjects could vary the radiance of the matching light in steps of either 0.01 log₁₀ or 0.08 log₁₀ to complete the match (see Figure 5). (The observers were also able to vary the phase between the contrast-modulation flicker and the sinusoidal flicker of the two hemifields in 2° or 10° steps until the maximum change in appearance of the two fields coincided. Even though the phase adjustments away from in phase were relatively small, we retained them as part of the procedure because they helped the observers to make matches.) We tried various other techniques, such as varying the modulation of the 560-nm matching light, but they produced much poorer matches due to differences in the perceived brightness between the target and matching stimulus that could not be eliminated. 

Figure 5

 

The arrangement used to match the brightness change and desaturation of a 560-nm contrast-modulated flicker (left hemifield and left stimulus) by varying the time-averaged radiance and amplitude of 92% modulated 560-nm sinusoidal stimuli superimposed on a 560-nm pedestal of the same time-averaged radiance as the contrast-modulated target (right hemifield and right stimuli). The small half-discs at the bottom of the figure indicate the approximate appearances of the hemifields when the matching contrast modulation (left) and sinusoidal flicker amplitude (right) are at their minima and maxima.

Wu et al. (1996) used a comparable method for matching flicker-induced brightness changes, but their stimuli were separated in time rather than space. 

Results

Figure 6 shows the L- or M-cone contrasts (which are equal for monochromatic lights) of the 0.5-Hz sinusoidal stimulus that matched the brightness of the contrast-modulated test stimulus at the peak of its 0.5-Hz contrast modulation plotted as a function of the overall modulation, m, of the contrast-modulated test hemifield—both with linear scales. Data for DP are shown in the left-hand panel and those for GBH in the right-hand panel (note the change in scale on the ordinates for the two observers). The error bars indicate ± 1 SEM. Extensive functions for three radiance levels, all in log₁₀ quanta s⁻¹deg⁻², are shown: 8.86 (dark green circles), 9.51 (light green diamonds), and 10.11 (yellow squares), but only two points could be measured at the lowest level of 8.26 (dark triangles). 

Figure 6

 

Both panels show L or M-cone contrasts of the 0.5-Hz sinusoidally flickering hemifield that matches the appearance of 0.5-Hz contrast-modulated 30-Hz flicker in an adjacent hemifield as a function of the overall modulation (m) of the contrast-modulated stimuli (both axes linear). The left-hand panel shows matches for DP and the right-hand panel for GBH. The time-averaged radiances were: 8.26 (inverted dark triangles), 8.86 (dark green circles), 9.51 (light green diamonds), and 10.11 (yellow squares) log₁₀ quanta s⁻¹deg⁻². Error bars indicate ±1 SEM. (Only two points could be measured at the 8.26 radiance level.)

Figure 7

 

Estimates of the logarithmic sensitivities corresponding to the attenuation characteristics of the late filter for DP (left-hand panel) and GBH (right-hand panel) at each of the four time-averaged radiance levels of 8.26 (inverted triangles), 8.86 (circles), 9.51 (diamonds), and 10.11 (squares) log₁₀ quanta s⁻¹deg⁻². The frequency axis is logarithmic. Error bars indicate ±1 SEM. The colored symbols show the logarithmic sensitivities for detecting the brightness change at f_m as a function of f_m (from Figure 5 rescaled according to the input vs. output contrast functions shown in Figure 6). Each set of data has been vertically aligned so that it has the same maximum amplitude sensitivity in log₁₀ quanta s⁻¹deg⁻² before and after rescaling. The gray symbols are the sensitivity differences between the early filter estimates (colored symbols, Figure 4) and the chromatic TCSF measurements (gray symbols, Figure 4). The alignment of the gray symbols with the colored ones was determined by the fit of a two-stage low-pass filter model (red lines), the details of which are described in the text. Error bars indicate ±1 SEM.

With increasing modulation the brightness-matching data for DP become steeper at low modulations up to a modulation of about 0.3 but thereafter are roughly linear with modulation (except for the highest modulation at the 8.86 level), whereas those for GBH are roughly linear with modulation at low modulations but flatten slightly at modulations above about 0.7. Thus, there are qualitative differences between the functions for the two observers at low- and high-contrast modulations. Between 0.3 and 0.7 the shapes of the functions for the two observers are similar. 

One notable feature of the functions for DP is that the brightness matches first increase as radiance increases from 8.26 to 9.51 log₁₀ quanta s⁻¹deg⁻² but then decrease, whereas for GBH they increase monotonically with radiance. Assuming small-signal linearity, the matching contrasts at a given radiance are effectively determined by the slope of the linear approximation to the nonlinearity at that radiance. Thus the increase in matching contrast from 8.26 to 9.51 log₁₀ quanta s⁻¹deg⁻² suggests an increasingly steep (accelerating) nonlinearity while the decrease for DP at 10.11 log₁₀ quanta s⁻¹deg⁻² suggests a flattening (“hardening”) of the nonlinearity at high radiances as in the case of S-cone isolating stimuli (Stockman & Plummer, 1998). No such flattening is indicated in the data of GBH. Another important feature of the functions for both subjects is that the effects of mean target radiance can be approximately accounted for by a scaling factor. We will return to the potential significance of this scaling property in the next paper in this series (Stockman, Petrova, & Henning, submitted). 

Discussion

The contrasts of the matching stimuli are always less than about 43% for DP and less than 75% for GBH. The large contrasts, particularly for GBH, raise the concern that the 0.5-Hz sinusoidal matching stimulus may itself be distorted by the nonlinearity. However, the 0.5 Hz matching stimulus is substantially attenuated by the early filter, roughly indicated at the low-frequency end by the model fits in Figure 8. Thus, for the worst case (10.11 log₁₀ quanta s⁻¹deg⁻² for GBH; 9.51 log₁₀ quanta s⁻¹deg⁻² for DP) the 0.5-Hz sinusoid at the input to the nonlinearity will be smaller by a factor of more than 1.4 log₁₀ unit. Consequently the highest matching contrast at the input to the nonlinearity for GBH is about 3.0% and for DP about 1.7%, and most matching contrasts will be lower. These small contrasts at the input to the nonlinearity, together with the gap between flicker and brightness-change thresholds in Figure 2, suggest that, in the matching experiment, a linear approximation to the nonlinearity at each mean radiance is not unreasonable. Thus, there will be only an unknown scaling factor at each radiance that converts the input modulation, m, of the contrast-modulated waveform to the contrast of the distortion product at the output of the nonlinearity; i.e., at the input to the late filter. We therefore replaced the overall contrast of the contrast-modulated waveform (abscissae of the panels in Figure 4) by scaled values of the corresponding ordinates of the panels of Figure 6. The rescaled TCSFs for detecting the brightness change at f_m are plotted as the filled colored symbols for the two observers in Figure 7. The shapes of the functions are correct, but they have been vertically shifted to align with the predictions from the model described below. These rescaled data constitute our estimates for the low-frequency end (f ≤ 5 Hz) of the post-nonlinearity (late) filter. The qualitative differences seen between DP and GBH at low- and high-input modulations in Figure 6 seem to have relatively little effect on the shapes of rescaled functions, except perhaps at the highest modulations where the rescaled functions for GBH are shallower at the two highest mean radiances. 

Figure 8

 

Estimates of the logarithmic sensitivities corresponding to the attenuation characteristics of the early filter for DP (left-hand panel) and GBH (right-hand panel) at each of the four time-averaged radiance levels of 8.26 (inverted triangles), 8.86 (circles), 9.51 (diamonds), and 10.11 (squares) log₁₀ quanta s⁻¹deg⁻². The frequency axis is logarithmic. Error bars indicate ±1 SEM. The colored symbols are the logarithmic sensitivities for detecting the brightness change at 0.5 Hz as a function of f_c from Figure 4 (also shown there as colored symbols). The sensitivities are the logarithmic amplitude sensitivities in log₁₀ quanta s⁻¹deg⁻². The open symbols are the sensitivity differences between the late filter estimates (colored symbols, Figure 7) and the monochromatic TCSF measurements (open symbols, Figure 4). The alignment of the open symbols with the colored ones was determined by the fit of a model (black lines), the details of which are described in the text. Error bars indicate ±1 SEM.

The assumption of small-signal linearity is an approximation. If the matching stimuli are significantly distorted by an expansive nonlinearity—the form of the nonlinearity at low signal levels suggested by our analysis below—then we would be underestimating the size of the distortion product after the nonlinearity. 

General discussion

Combining the results at low and high frequencies

From the brightness-change thresholds at f_m Hz, we were able to characterize the attenuation characteristics of the early filter above 5 Hz and those of the late filter below 5 Hz. Given that both estimates depend upon a single type of threshold judgment, they can be reasonably assumed to depend on a single process. These estimates, therefore, represent our strong conclusions. They are highlighted by the colored symbols in Figures 7 and 8. 

To estimate the characteristics of the early filter below 5 Hz and the late filter above 5 Hz, where directly measured data could not be obtained, we assume that the shapes of the conventional TCSFs of Figure 3 reflect the multiplicative combination of the early and the late filters. Thus, the logarithmic differences between them and our early or late filter measurements should yield estimates of the shapes of the missing parts of each filter. Such estimates, however, depend on different types of threshold judgments, so that the assumption that they depend on a single process is less secure. Consequently, these estimates represent our weaker conclusions. They are indicated by the achromatic symbols in Figures 7 and 8. 

A remaining question, then, is which of the monochromatic and chromatic TCSF estimates should be used to estimate the missing parts of the early and late filters. On the grounds that the contrast-modulated flicker used to generate the distortion product is monochromatic, and therefore subject to the same low-frequency attenuation, we assumed that the inferred shapes of the early filters below 5 Hz should be the logarithmic differences between the monochromatic TCSFs (open symbols, Figure 3) and the late filter estimates (colored symbols, Figure 7). These estimates are shown as the open symbols in Figure 8, where they have been aligned with the direct estimates of the early filters (colored symbols). 

The choice of TCSFs for the derivation of the late filter above 5 Hz proved to be less straightforward. We initially supposed that the brightness variation is signaled by a luminance channel, which would be insensitive to chromatic flicker. Thus, the inferred shapes of the missing parts of the late filter should also be the logarithmic differences between the monochromatic TCSFs (open symbols, Figure 3) and the early filter estimates (colored symbols, Figure 8). However, these differences, which are shown below in Figure 10, are complex and do not lend themselves to simple modeling or interpretation. In contrast, the logarithmic differences between the chromatic TCSFs (gray symbols, Figure 3) and the early filter estimates (colored symbols, Figure 8) are consistent with a simple two-stage low-pass filter. These differences are shown as the gray symbols in Figure 7, where they have been aligned with the colored symbols in each panel to form the estimates of the late filters for each observer. As we discuss below, this simplicity lends weight to the hypothesis that brightness changes in these experiments are mediated by a pathway with temporal characteristics similar to chromatic pathways. 

In simply differencing the conventional TCSFs, we ignore the effects of the nonlinearity intervening between the early and late filters. This seems a reasonable simplification near flicker threshold where small-signal linearity and the separation between the flicker and brightness-change fusion frequencies (see Figure 2) suggest that the nonlinearity plays little role. 

In each panel of Figures 7 and 8, the filter estimates are plotted separately for DP (left-hand panels) and GBH (right-hand panels) for the four radiance levels (log₁₀ quanta s⁻¹deg⁻²): 8.26 (triangles), 8.86 (circles), 9.51 (diamonds), and 10.11 (squares). The alignments of the measured and estimated filter characteristics below and above 5 Hz are not fixed by our data. To fix the alignments shown in Figures 7 and 8, we developed simple models of the early and late filters. These models not only help to align the data but also yield insights into the underlying physiological mechanisms. 

Modeling the early and late filters

These models were developed in Petrova, Henning, and Stockman (2013), where further details can be found. Briefly, we take the classic approach of using filters composed of cascading leaky integrating stages (or buffered resistor-capacitor circuits) (see Watson, 1986). The amplitude response, A(f), of n cascaded, identical, leaky integrators is:  

 

where $f_0$ (Hz) is the “cut off” or “corner frequency.” For more details, see Petrova et al. (2013), for which this model was first developed. 

The early filter estimates (Figure 8) are clearly band pass in shape. In this they are similar to the early filter of Wu et al. (1996). To account for the low-frequency attenuation, we divide a “center” temporal response of a cascade of filters by a “surround” temporal response of another cascade of filters, thus:  

 

where the subscript c refers to the parameters of the central cascade of filters and the subscript s refers to the parameters of the surround cascade (e.g., Foley, 1994). 

Fits were made using the logarithm of Equation 6 and the data. A scaling constant, k, which could vary with target radiance level was allowed. In terms of the model, k represents a frequency-independent sensitivity loss that is in addition to any losses resulting from the changing corner frequencies of the filters. In addition, an extra arbitrary constant, v, was added to the low-frequency data for each level, the value of which was individually optimized for each set of data to determine the best-fitting vertical alignment of the low- and high-frequency data, thus:  

 

 

Best-fitting versions of the model were obtained using a standard, nonlinear, least-squares, curve-fitting algorithm (implemented in SigmaPlot, SPSS) to account for the data obtained for each observer at each of the four time-averaged radiances. Equations 6 and 7 were fitted simultaneously to the estimates of the functions for both DP and GBH. The model was simplified by fixing those parameters that did not vary systematically with target radiances or across observers. Our aim was to derive a plausible descriptive model with as few parameters as could reasonably account for the totality of the data. 

Implicit in the use of Equation 6 is the simplification that at any target radiance the cut-off frequencies of all center stages (f_0c) and all surround stages (f_0s) are the same. The number of stages in both center and surround (n_c and n_s) were allowed to take on noninteger values in preliminary fits, but in the final fits we fixed them at the nearest integer values (n_c = 8 and n_s = 2). A further simplification that could be made was that the surround corner frequency, f_0s, was fixed across levels. Both scaling (the vertical logarithmic shift, k) and the center cut-off frequencies were allowed to vary between observers and between levels. 

The results of the final fit of the model for the early filters are shown in Figure 8 as the continuous dark lines. The parameters from the fit are tabulated in Table 1 and plotted in the left-hand panels of Figure 9. The values of v, which are not experimentally constrained, are not given. These values are largely arbitrary because they depend on the relative sensitivities of the low- and high-frequency filter estimates, one of which was measured directly (and in the case of the late filter rescaled) and the other of which was obtained indirectly by logarithmically differencing other sensitivities measured in different units (achromatic vs. chromatic modulation). The upper left-hand panel of Figure 9 shows the corner frequencies for the center cascade for DP (yellow squares) and GBH (filled squares) plotted as a function of the logarithm of the target radiance (logarithmic). The gray symbols show the common corner frequency for the surround cascade that does not depend on radiance. The lower panel shows the scaling (or log sensitivity) losses for DP (yellow circles) and GBH (filled circles) that were required in addition to the losses caused by changes to the corner frequency. Overall, the model accounts for the data well with an R² value of 0.986. The corner frequencies and sensitivity losses are slightly larger for DP than for GBH. 

Figure 9

 

Fitting parameters of the models describing filter characteristics: early filter (left panels) and late filter (right panels). The upper left-hand panel shows the corner frequencies of the center cascade for DP (yellow squares) and GBH (black squares) and the lower panel the log scaling or sensitivity losses for DP (yellow circles) and GBH (black circles) that were required in addition to the effects of changes in corner frequency all as a function of log₁₀ radiance. The gray diamonds in the upper panel show the common corner frequency for the surround cascade for both observers; it does not depend on radiance. The upper right-hand panel shows the corner frequencies for the two-stage late filter; the lower panel the log scaling parameters: yellow squares (DP), dark green squares (GBH) both as a function of radiance. The red line in the bottom panel indicates the prediction of Weber's Law. Overall, for both observers, the model accounts for the data well with an R² value of 0.989.

Figure 10

 

Estimates of the late filter (colored symbols) and the model fits (red lines) for DP (left-hand panel) and GBH (right-hand panel) replotted from Figure 7. The open symbols in each panel show the differences between the early filter estimates at higher frequencies (colored symbols, Figure 8) and the monochromatic TCSFs (open symbols, Figure 3). The open symbols have been fitted by a line with a best-fitting slope across all four levels of −0.68 for DP and −0.50 for GBH (black lines), and the lines and data have been vertically shifted together so that the lines align with the chromatic curves at 2.5 Hz. Error bars indicate ±1 SEM. The vertical positions of the top set of data in each panel are the same as in Figure 8, but for clarity the second, third, and fourth sets have been shifted downwards by an extra 1, 2, and 3 log₁₀ units, respectively.

Table 1

 

Best-fitting parameters for the model of the early filter. See text for details.

Parameter	DP	GBH
nc (fixed)	8
ns (fixed)	2
f0s (fixed)	1.31 ± 0.08
f0c 8.26	19.80 ± 0.70	15.52 ± 0.56
f0c 8.86	22.81 ± 0.77	18.12 ± 0.55
f0c 9.51	25.95 ± 0.95	20.98 ± 0.62
f0c 10.11	23.25 ± 0.80	22.01 ± 0.66
k 8.26	0.00 ± 0.09	0.00 ± 0.09
k 8.86	0.25 ± 0.09	0.07 ± 0.07
k 9.51	0.60 ± 0.10	0.30 ± 0.07
k 10.11	1.53 ± 0.09	0.71 ± 0.07
R2	0.986

The late filter for both observers (solid red lines, Figure 7) is broadly consistent with the simple low-pass filter of Equation 5. We fitted this equation simultaneously to the estimates of the functions for both DP and GBH. Again, the model was simplified by fixing parameters that did not systematically vary across either target radiances or observers. In preliminary estimates, we found that we could fix the cut-off frequencies across radiance levels. We also allowed the number of stages to take on noninteger values but in the final fit constrained them to the nearest integer values of n = 2. Again, an additional alignment constant, v, was added to the low-frequency data of each set and individually optimized to determine the best fitting vertical alignment of the low and high frequency (see Equation 7). 

The parameters from the final fit are tabulated in Table 2 and plotted in the right-hand panels of Figure 9. The upper panel shows the fixed corner frequencies (linear ordinate) for DP (yellow squares) and GBH (filled squares) plotted as a function of target radiance (logarithmic), the lower panel shows the scaling or sensitivity losses for DP (yellow circles) and GBH (filled circles). As indicated by the red line with a slope of one, the increases in the sensitivity losses with target radiance are consistent with Weber's Law. 

Table 2

 

Best-fitting parameters for the model of the late filter. See text for details.

Parameter	DP	GBH
n (fixed)	2
f0 (fixed)	3.19 ± 0.33	3.39 ± 0.34
s 8.26	0.00 ± 0.07	0.00 ± 0.07
s 8.86	0.52 ± 0.07	0.62 ± 0.07
s 9.51	1.17 ± 0.07	1.23 ± 0.07
s 10.11	1.95 ± 0.07	1.83 ± 0.07
R2	0.988

Overall, the model accounts for the data well with an R² value of 0.988. 

Related psychophysical measurements

Our estimates of the form of the early filter can be usefully compared with the previous estimate of Wu et al. (1996). Both estimates depend on the nonlinear distortion that produces brightness enhancement, but there are some differences in methodology. First, Wu et al. measured the magnitude of brightness enhancement by suprathreshold matching, whereas we measured its threshold. Second, they used flicker of 594 nm, which would have generated a small hue change as well as a change in brightness (see above), whereas we used flicker of 560 nm. Third, their mean luminance of 4.25 log₁₀ photopic td was higher than our highest mean luminance of 4.01 log₁₀ photopic td (10.11 log₁₀ quanta s⁻¹deg⁻²). They obtained data from three observers. Their mean data, like ours, are band pass in form, but their estimate has a broader peak. The differences may be due to the differences in method. 

Site of the nonlinearity

Previous work on the distortion of high-spatial frequency gratings and of types of flicker has suggested that the relevant nonlinearities are at an early level of the visual system, perhaps close to the photoreceptors (MacLeod et al., 1992; Stockman & Plummer, 1998; Wu et al., 1996). Yet, even for unresolvable gratings produced by laser interference, the nonlinearity seems to follow an earlier stage of surround antagonism (Chen et al., 1993), which places the nonlinearity at least after, or coincident with, horizontal cell feedback. Our estimates of the early filter shape from the brightness enhancement of 560-nm flicker (Figure 8) show a substantial loss of low-frequency sensitivity, which suggests that this nonlinearity also follows a stage of surround antagonism. Comparable results obtained with 650-nm flicker, the distortion of which is seen as a hue change, show similar effects (Petrova et al., 2013). We conclude that the expansive nonlinearity that enhances brightness is acting on L- and M-cone signals that have been shaped in some way by opponency. 

But in which postreceptoral pathway is the brightness enhancement most likely to occur? Is the nonlinearity within the achromatic or luminance pathway sensitive to higher temporal frequencies that is often linked to the magnocellular stream of retinal processing? Or is it within the achromatic or luminance pathway sensitive to higher spatial frequencies and linked to the parvocellular stream of processing, which also encodes chromatic information? The dual nature of the luminance pathway has been discussed in several papers by Ingling et al. (Ingling & Drum, 1973; Ingling & Martinez, 1983; Ingling & Martinez-Uriegas, 1983, 1985; Ingling & Tsou, 1988) and has also been covered in several reviews (e.g., Lennie, Pokorny, & Smith, 1993; Stockman & Brainard, 2009). 

We initially supposed that brightness enhancement occurs within the luminance flicker pathway. Consequently, the TCSFs shown in Figure 3 for brightness enhancement and those for detecting monochromatic flicker (which at threshold should be mediated mainly by the luminance pathway, see Lennie et al., 1993) should be measures of the temporal properties of stages within a common pathway. On the assumption that the brightness-enhancement TCSFs correspond to the shapes of the early filter, while the monochromatic TCSFs correspond to the shapes of the combined early and late filters (see above), the logarithmic differences between them should produce a plausible estimate of the shape of late filter. Those differences are shown as open symbols in Figure 10 for DP (left-hand panel) and GBH (right-hand panel) where they have been vertically aligned with the other data near 2.5 Hz. Rather than being simple, the differences are complex: The estimates for DP, in particular, show clear evidence for a secondary peak near 15 Hz. If the inferred late filter is consistent with a cascade of low-pass filters, then its high-frequency logarithmic slope should decline with an integer logarithmic slope. Yet, the estimates for DP have a best-fitting linear slope across all four levels of −0.68 (black lines) and those for GBH have a slope of −0.50 (black lines). 

In contrast to the estimates based on the monochromatic TCSFs, those based on the chromatic TCSFs (which at threshold should be mediated mainly by the chromatic pathway, see above) fall off with a logarithmic slope of −2 (red lines, Figure 10), and are thus consistent with the simple interpretation that the late stage is a two-stage low-pass filter. This simplicity, and the finding that a nearly identical two-stage filter can also account for the late filter estimated using chromatic distortion (Petrova et al., 2013), suggests that the brightness signal after the nonlinearity is transmitted principally through a pathway that combines brightness and chromatic information. 

We conclude that the brightness enhancement of flicker is produced by distortion in pathways that can also carry chromatic signals. This is consistent with the double duty of the parvocellular pathway, which is both chromatically opponent and spatially opponent. Thus, it can encode chromatic information, which is dependent upon the difference between the spectral sensitivities of the center and surround, and it can also encode “achromatic” information, which is dependent on the sum of the center and surround spectral sensitivities. For color, the center and surround behave synergistically to produce a low-pass response to spatial variations in chromaticity, but for luminance they behave antagonistically to produce a more band-pass response to spatial variations in luminance (Ingling & Martinez, 1983; Ingling & Martinez-Uriegas, 1983, 1985). Thus, it is reasonable to assume that the parvocellular stream carries luminance and/or brightness information as well as color information (e.g., Ingling & Drum, 1973; Ingling & Martinez, 1983; Ingling & Martinez-Uriegas, 1985; Ingling & Tsou, 1988; Kelly, 1983; Lennie & D'Zmura, 1988; Merigan & Eskin, 1986; Schiller, Logothetis, & Charles, 1990). Simple mechanisms for decoding the luminance and chromatic signals from multiplexed signals have been proposed that difference or sum center-surround chromatically opponent neurons (e.g., Billock, 1991; Kingdom & Mullen, 1995; Lennie, 1984; Lennie & D'Zmura, 1988; Martinez-Uriegas, 1985), see Stockman and Brainard (2009) for a recent discussion. Note that our results are consistent with the behavior of a single process with a characteristic early and a late filter. We do not need to invoke multiple brightness components to explain the data (e.g., Drum, 1984). 

It is possible that at carrier frequencies low enough for individual flicker cycles to be resolved, subjects base their responses partly on the appearance of the peaks of the flicker rather than on the distortion signal varying at f_m Hz. Although still an inherently nonlinear process, the effective nonlinearity is likely to be at a very different stage of the processing stream than the one that produces brightness enhancement at f_m. This complication may be one of the reasons why reliable brightness threshold settings cannot be made at f_c < 5 Hz. 

The idea that brightness and chromatic signals are perceptually related is central to the substantial literature developed from Hering's (1878, 1920) opponent-color theory in which perceptions of red and green, blue and yellow, and black and white are opposed (e.g., De Valois & De Valois, 1993; Guth, 1991; Guth & Lodge, 1973; Guth, Massof, & Benzschawel, 1980; Hurvich & Jameson, 1955, 1956; Ingling & Tsou, 1977; Jameson & Hurvich, 1955, 1956; Krantz, 1975; Schrödinger, 1925). 

Characteristics of the nonlinearity

We cannot determine the shape of the underlying nonlinearity precisely from our observations and data, but we can infer its general characteristics. From the subjective observations that contrast-modulated, sinusoidally flickering, 560-nm light appears brighter when the contrast modulation is high, we can infer that the primary nonlinearity is likely to be expansive. We can also use the data relating the output contrast after the nonlinearity to the input contrast before it (see Figure 6) to test the feasibility of different nonlinear forms. We generated a range of nonlinear forms and then used each one to generate predictions for the data of Figure 6 using MatLab and Simulink (MathSoft) for all calculations and to generate the Fourier transform of the input and output signals before and after the nonlinearity. 

To generate the predictions, we started with a particular nonlinear input-output function, such as the example shown in the upper panel of Figure 11. This particular example has the form:  

 

where x is the input to the nonlinearity and y is the output. The units are arbitrary. We assumed that as the time-averaged 560-nm target radiance increases, the DC (mean) input level (which we call the “offset”) also rises, thus moving up the nonlinear input-output function. In the simulation, we varied the DC offset from 0.0 to 2.5 and at each level varied the modulation of the contrast-modulated input from 0 to 100%, where 100% was the maximum modulation possible at that DC level (for example, at a DC level of one, the minimum and maximum of the contrast-modulated input signal are zero and two, respectively). At each combination of DC level and modulation, we calculated the contrast of the distortion product at f_m; that is, the ratio of the amplitude of the distortion product at f_m to the total DC after the nonlinearity. The simulated contrast of the signal at f_m is plotted as a function of the DC offset and input modulation in the lower panel of Figure 11. 

Figure 11

 

Effect of the clipped expansive nonlinearity illustrated in the upper panel on the contrast of the distortion product at f_m. The upper panel shows the input/output function of Equation 8 and the lower panel the contrast of the distortion product at f_m resulting from a contrast-modulated input with the input modulations and DC offsets shown. For details, see text.

It is important to recognize that the nonlinearity in these simulations affects the contrast-modulated flicker by introducing the f_m-intermodulation distortion product in which we are interested and, in addition, a steady-state (DC) term that changes the mean radiance around which the stimulus flickers. The contrast of the f_m distortion product, which we can reasonably assume determines its detectability and is plotted in Figure 11, depends on both of these products. If the nonlinearity is expansive, the amplitude of the f_m term has a positive coefficient and the mean (DC) level is increased. The contrast and therefore the relative detectability of the f_m-component depend on the interplay between these components. 

Any candidate nonlinearity must account for the features of Figure 6. The contrast of the distortion product must grow with input modulation and the functions must increase as the mean radiance increases from 8.26 to 9.51 log quanta s⁻¹deg⁻². However, at the highest level, between 9.51 and 10.11 log quanta s⁻¹deg⁻², the function describing the nonlinearity, certainly for DP, must get shallower again. We found that most of the tested nonlinear forms could account for some but not all features of the input contrast to output contrast functions. The simplest nonlinearity that could easily account for the functions (including the fall off for DP) was one that was expansive at low input levels but that reached a ceiling at high levels (like the example shown in the upper panel of Figure 11). The changes in the simulated modulation versus f_m contrast functions with DC offset in the lower panel of Figure 11 mirror the changes in the output contrast with input modulation in Figure 6. These similarities suggest that the nonlinearity of Figure 11 is a reasonable model of the underlying nonlinearity. 

These simulations and modeling are largely descriptive, but they allow us to draw some general conclusions about the nonlinearity. In order to simulate the decrease in the input versus output contrast functions between the two highest levels for DP, we suggest that the nonlinearity must have an abrupt ceiling. We note that the inferred nonlinearity of Figure 11 relates only to the four 560-nm mean radiance levels from 8.26 to 10.11 log quanta s⁻¹deg⁻² used in the experiments. At lower levels, the nonlinearity may have a different form or may be missing. However, the critical-fusion frequency data of Figure 2 indicate that distortion can still be detected at 560 nm down to radiances as low as 5.81 log quanta s⁻¹deg⁻². 

Conclusions

A nonlinearity in an L- and M-cone pathway that causes a 560-nm light to look brighter when flickered has been used as an internal landmark at which to dissect the pathway into early (prenonlinearity) and late (post-nonlinearity) stages. The early temporal stage acts like a band-pass filter with sensitivity losses at higher temporal frequencies that are consistent with the nonlinearity occurring relatively early in the visual pathway. The losses at lower frequencies, however, suggest that the nonlinearity must be after the action of substantial surround antagonism. 

Modeling suggests that the form of the nonlinearity is smoothly expansive but perhaps with a hard saturating limit at high input levels. What advantage is there in having such a prominent expansive nonlinearity in this pathway? An obvious advantage is that the nonlinearity will enhance time-varying signals at middle to high temporal frequencies after reciprocal von Kries adaptation (1905) at low temporal frequencies (probably at the photoreceptor level) has removed the mean (DC) signal (see for review and model Stockman, Langendörfer, Smithson, & Sharpe, 2006). Thus, the early gain control at the photoreceptor helps to keep the mean output within the dynamic range of the later neural circuits, while the expansive nonlinearity enhances the remaining signals. Note that we minimize the effect of photoreceptor adaptation in our experiments by maintaining a constant time-averaged level about which stimuli are modulated. 

The late stage acts like a simple two-stage low-pass filter with cut-off frequencies around 3 Hz. The purpose of this filter may be to extract a slowly changing mean brightness signal from the input signal. 

Acknowledgments

This work was supported by BBSRC grant numbers BB/E016235/1 and BB/1003444/1. We thank Professor Rhea Eskew and two anonymous reviewers for helpful comments. 

Commercial relationships: none. 

Corresponding author: Andrew Stockman. 

Email: [email protected]. 

Address: UCL Institute of Ophthalmology, University College London, London, England, UK.