Same calculation efficiency but different internal noise for luminance- and contrast-modulated stimuli detection

Abstract

There is no consensus on whether luminance-modulated (LM) and contrast-modulated (CM) stimuli are processed by common or separate mechanisms. To investigate this, the sensitivity variations to these stimuli are generally compared as a function of different parameters (e.g., sensitivity as a function of the spatial or temporal window sizes) and similar properties have been observed. The present study targets the sensitivity difference between LM and CM stimuli processing. Therefore, instead of studying the variation of sensitivity in different conditions, we propose to decompose the sensitivities in internal equivalent noise (IEN) and calculation efficiency (CE) to evaluate at which processing level the two mechanisms differ. For each stimulus type, the IEN and CE of four observers were evaluated using three different carriers (plaid, checkerboard, and binary noise). No significant CE differences were noted in all six conditions (3 carriers × 2 modulation types), but important differences were found between the IEN of the two stimulus types. These data support the hypothesis that the two pathways are initially separate and that the two stimuli may be treated by common mechanisms at a later processing stage. Based on ideal observer analysis, pre-rectification internal noise could explain the difference of IEN between LM and CM stimuli detection when using binary noise as a carrier but not when using a plaid or a checkerboard. We conclude that a suboptimal rectification process causes higher IEN for CM stimuli detection compared with LM stimuli detection and that the intrinsic noise of the binary carrier had a greater impact on the IEN than the suboptimal rectification.

Introduction

Human observers are sensitive to both luminance-modulated (LM) and contrast-modulated (CM) stimuli. In the present study, we define LM stimuli as the addition of an envelope (signal) with a carrier (texture) and CM stimuli as their multiplication. Consequently, for LM stimuli, the local luminance average varies throughout the stimulus according to the envelope while the local contrast remains constant (Figure 1, left). For CM stimuli, the local luminance average remains constant and the local contrast varies throughout the stimulus according to the envelope (Figure 1, right). Therefore, because a Fourier transform can directly detect the signal frequency of LM stimuli, this type of stimulus is typically characterized as Fourier, first order, or linear. However, CM stimuli are not considered as Fourier stimuli because the signal frequency is not present in the Fourier domain. Therefore, CM stimuli are characterized to be non-Fourier, second order, or nonlinear stimuli (Cavanagh & Mather, 1989; Chubb & Sperling, 1988). 

Figure 1

 

Luminance profile of LM and CM stimuli as a function of space (one dimension).

Nonlinear processing

Wilson, Ferrera, and Yo (1992) proposed a two-stage model for the detection of LM and CM motion. This model may be summarized as follows: both stimuli are initially processed by V1. For LM stimuli, this contrast detection corresponds to the signal or envelope itself and the stimuli do not require anymore processing before MT processes the perceived motion. For CM stimuli, the contrast detection occurs at higher spatial frequencies corresponding to the carrier; therefore, the treatment uses another path and the information passes through V2 for a second-order rectification process before attaining MT. After a second-order rectification process, a CM stimulus becomes similar to a LM stimulus (Chubb & Sperling, 1988; Solomon & Sperling, 1994; Sperling, Chubb, Solomon, & Lu, 1994). Therefore, the two stimuli could be merged and then treated by the same mechanisms (Baker, 1999). Therefore, the processing of these two stimulus types is initially separated but may be common at a later stage. More recently, many similar models have been developed and are typically referred to as filter-rectify-filter models (Clifford & Vaina, 1999; Nishida & Sato, 1995; Prins & Kingdom, 2003). Figure 2 shows an example of such a model where an extra process is required for CM stimuli processing. Although this class of model seems to be more popular, other models have also been developed. Some motion models propose that both LM and CM stimuli are treated by common mechanisms (Benton & Johnston, 2001; Johnston & Clifford, 1995a, 1995b; Johnston, McOwan, & Buxton, 1992; Taub, Victor, & Conte, 1997). Consequently, in motion perception, the idea that LM and CM stimuli are processed by common mechanisms is still largely debated. This debate has carried over in spatial vision (which is the object of the present study) where the processing of static LM and CM stimuli has been compared. 

Figure 2

 

Filter-rectify-filter model. For CM stimuli, the first filtering stage processes the carrier so that, after a rectification process, the second stage can detect the envelope. For LM stimuli, no rectification is required.

Evidence for separate mechanisms

Evidence from spatial vision studies suggests that LM and CM stimuli are, at least initially, processed by separate mechanisms. Nishida, Ledgeway, and Edwards (1997) found that after adapting to one type of stimulus (LM or CM) the sensitivity to the same type of stimulus was affected, but not the sensitivity to the other. Schofield and Georgeson (1999) did not find any inter-type subthreshold summation while intra-type subthreshold summation was found. The same authors showed strong evidence suggesting that LM and CM stimuli are not merged after a second-order rectification process (Georgeson & Schofield, 2002). They first showed that the recognition of the stimulus type (LM vs. CM) was almost as good as the detection of each type (LM vs. noise or CM vs. noise). They also demonstrated that observers do not confuse the two stimulus types when they are combined because the recognition of the two stimuli combined in-phase or out-phase (LM + CM vs. LM − CM) is similar to their detection (LM + CM vs. noise or LM − CM vs. noise). 

Evidence for common mechanisms

Although important evidence suggests that LM and CM stimuli are processed by separate mechanisms, there is still evidence suggesting the opposite. First, the processing of LM and CM stimuli by human observers shares similar properties. Schofield and Georgeson compared the sensitivity to these stimuli as a function of stimulus size (Schofield & Georgeson, 1999) and presentation time (Schofield & Georgeson, 2000). They found similar spatial and temporal integration for both stimulus types. In their experiments, the sensitivity curves of LM and CM stimuli as a function of a spatial or a temporal window were parallel. These results indicate a similar spatiotemporal integration for both modulation types although the sensitivity was greater for the LM stimulus. Similar behaviors generally suggest that both stimuli could be processed, at least partially, by common mechanisms. However, the implications of these findings do not necessarily lead to such conclusions. Two separate mechanisms could have similar behaviors. 

In another study, Georgeson and Schofield (2002) showed direct evidence of an interaction between the processing of LM and CM stimuli. They found that, after adaptation to a given stimulus type (LM or CM), the perceived contrast of the other stimulus type was reduced almost by the same proportion as the one of the same type. 

Purpose of the present study

To investigate if LM and CM stimuli are processed by common or separate mechanisms, the sensitivity variations to these stimuli are generally compared as a function of different parameters. As mentioned above, similar function shapes have been observed in certain conditions, which suggests that the main difference between the two is their sensitivity. Therefore, we propose to decompose the sensitivity to evaluate at which processing level the two mechanisms differ. 

Based on the assumptions that the internal noise and calculation are contrast invariant (see below), the sensitivity may be separated into two parameters (Legge, Kersten, & Burgess, 1987; Pelli, 1981, 1990): internal equivalent noise (IEN) and calculation efficiency (CE). The goal of the present study was to elucidate if LM and CM sensitivities differ because of a difference of IEN, CE, or both. 

Evaluating the IEN and CE

The internal noise is the signal deterioration introduced by different processing levels that limit the observer's sensitivity (e.g., optical noise caused by eye imperfections, photon-noise, neuronal noise…). The calculation is the observer's ability to detect a given signal embedded in noise. Both the internal noise and calculation limit the sensitivity of a noise-free signal. Once the internal noise is added to the stimulus, the observer's task consists in detecting the signal embedded in noise. 

Contrast-invariant calculation signifies that the observer's performance only depends on the signal-to-noise ratio (SNR) (Pelli, 1981, 1990). Therefore, equally modifying both the signal contrast (c) and effective noise contrast (N_eff) will not affect the observer's performance. The effective noise represents the combination of the internal (observer) and external (stimulus) noise. 

Assuming that the observer's calculation is contrast invariant, the minimum SNR necessary to detect the signal based on a given threshold criterion would be constant:  

$k=\frac{c}{N_{\mathrm{e}\mathrm{f}\mathrm{f}}}.$

(1)

 

The CE may be defined as the minimum SNR necessary to detect the signal of an ideal observer relative to the observer's SNR:  

$\mathrm{C}\mathrm{a}\mathrm{l}\mathrm{c}\mathrm{u}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{c}\mathrm{y}=\frac{k_{\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{a}\mathrm{l}}}{k},$

(2)

where k_ideal is the k parameter for the ideal observer. An ideal observer is a theoretical observer using all the information available to optimally perform the task. Therefore, k_ideal represents the smallest SNR (k) mathematically possible to detect the signal (c) based on a given threshold criterion. 

Assuming that the internal noise is also contrast invariant, the impact of the internal noise will be constant as a function of the signal and external noise contrast (Pelli, 1981, 1990). The root mean square (RMS) contrast of the effective noise (N_eff) will be  

$N_{\mathrm{e}\mathrm{f}\mathrm{f}}=\sqrt{N^2+N_{\mathrm{e}\mathrm{q}}^2},$

(3)

where N and N_eq represent the RMS contrasts of the external noise and IEN, respectively. The IEN models the impact of the internal noise on the sensitivity. Note that if the external noise contrast is equal to the IEN (N = N_eq), the effective noise (N_eff) will be √2 times greater. Therefore, assuming that the calculation is contrast invariant (Equation 1), the IEN will be equal to the external noise contrast that raises the signal contrast threshold (c) by a factor of √2. 

Based on the two assumptions that the internal noise and calculation are contrast invariant, the function between the signal contrast (c) threshold and the external noise contrast (N) can be deduced by combining Equations 1 and 3:  

$c=k\sqrt{N^2+N_{\mathrm{e}\mathrm{q}}^2},$

(4)

and the relation between the squared signal contrast (c²) and the external noise variance (N²) would be linear:  

$c^2=k^2(N^2+N_{\mathrm{e}\mathrm{q}}^2).$

(5)

 

Such linear relation for the detection grating in Gaussian white noise has been found in the past (Pelli, 1981), which supports the hypothesis that the internal noise and calculation are contrast invariant. For a review on the relation between external noise and detection threshold, which permits to decompose the sensitivity in IEN and CE, refer to Legge et al. (1987) and Pelli (1981, 1990). 

When the external noise contrast is relatively small compared with the IEN (N << N_eq), varying the external noise does not affect significantly the effective noise (Equation 3), and therefore the signal contrast threshold is relatively constant as a function of the external noise. However, in high external noise conditions, varying the external noise has a great impact on the total amount of noise and thereby the signal contrast threshold increases as a function of the external noise. 

Given that the processing of two stimuli differs in IEN but not in CE, then in high external noise conditions, the internal noise should not be significant and no important threshold difference should be observed. The left graph of Figure 3 illustrates this hypothesis. However, if the IENs are equal and the CEs differ, the detection thresholds should be different in all external noise conditions. Indeed, the two curves would be parallel (Figure 3, right). 

Figure 3

 

Examples of detection threshold patterns as a function of the external noise. On the left, both patterns have the same calculation efficiencies but the red pattern shows less IENs. The graph on the right shows the opposite; both patterns have the same IENs but the red pattern has a greater CE.

Predictions based on separate mechanisms

If LM and CM stimuli are processed by separate mechanisms, the predictions are straightforward: each stimulus pathway should have its own IEN and CE. Consequently, the probability of having the same IEN or CE for both stimuli would be low. Indeed, the observer's ability to detect a LM signal in LM noise would probably differ from its ability to detect a CM signal in CM noise. In other words, in high external noise conditions, which result in nonsignificant internal noise, the sensitivity to LM and CM stimuli would probably differ. 

Predictions based on common mechanisms

According to the two-stage (or filter-rectify-filter) model, the difference between LM and CM stimuli processing is that CM stimuli detection requires an extra second-order rectification process. Consequently, deriving its predictions is more complex because the impact that a suboptimal rectification would have on IEN and CE must be determined. 

The CM stimulus may be defined as the multiplication of a modulation [M(x,y)] with a texture [T(x,y)], where the modulation is defined by lower spatial frequencies relative to the texture, and the texture local RMS contrast (T_RMS) is constant throughout the stimulus. Therefore, the modulation represents the global texture contrast variation. The RMS contrast near the position (x,y) is equal to T_RMSM(x,y). The rectification consists in estimating the local (carrier spatial frequency) RMS contrast. This estimation should be applied locally over the entire stimulus (i.e., for all (x,y) positions), which would reconstruct a similar modulation [T_RMSM(x,y)] as the one defining the stimulus [M(x,y)]. 

Consequently, after a rectification, a CM stimulus is converted into an effective stimulus [T_RMSM(x,y)] similar to a LM stimulus without a texture, that is, the modulation M(x,y). In the LM stimulus, each position represents the luminance intensity. For the rectified CM stimulus, each position of the effective stimulus would represent the local contrast modulation of the CM stimulus. Therefore, after a rectification, a CM stimulus would be analogue to a LM stimulus and both could be treated by common mechanisms. 

Because a rectification is likely to be suboptimal, let us represent the estimation of the local contrast [T_RMSM(x,y)] by a normal distribution centered at βT_RMSM(x,y) and with a standard deviation of N_rect. The β represents the gain parameter affecting the strength of the rectification output. N_rect represents internal noise that could be added during the rectification process. 

Suppose a CM stimulus [M(x,y)T(x,y)] with a contrast modulation [M(x,y)] composed of a signal [S(x,y)] with contrast S_in embedded in noise [N(x,y)] with contrast N_in. Using the previously defined rectification, the signal and noise contrast at the output of the rectification would be scaled by a factor of βT_RMS and noise would also be added (N_rect). Consequently, the signal (S_in) and noise (N_in) contrast of the input of the rectification process would result, after the rectification process, as  

$S_{\mathrm{o}\mathrm{u}\mathrm{t}}=\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}}{S}_{\mathrm{i}\mathrm{n}},$

(6)

 

$N_{\mathrm{o}\mathrm{u}\mathrm{t}}=\sqrt{(\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}}{N}_{\mathrm{i}\mathrm{n}}{)}^2+N_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}^2},$

(7)

and the SNR would pass from S_in/N_in to  

$\begin{array}{c}\frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}=\frac{\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}}{S}_{\mathrm{i}\mathrm{n}}}{\sqrt{(\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}}{N}_{\mathrm{i}\mathrm{n}}{)}^2+N_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}^2}}\\ \frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}=\frac{S_{\mathrm{i}\mathrm{n}}}{\sqrt{N_{\mathrm{i}\mathrm{n}}^2+(N_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}/\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}}{)}^2}}\end{array}$

(8)

 

By defining  

$N{\prime }_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}=N_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}/\beta T_{\mathrm{R}\mathrm{M}\mathrm{S}},$

(9)

we obtain  

$\frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}=\frac{S_{\mathrm{i}\mathrm{n}}}{\sqrt{N_{\mathrm{i}\mathrm{n}}^2+N_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}^{\prime 2}}}$

(10)

 

Therefore, based on this type of rectification and on the two-stage model, a suboptimal rectification would increase the IEN and would not affect the CE. In other words, if the strength of the input noise is relatively low compared with the noise added by the rectification (N_in << N′_rect), then the impact of the input noise would not be significant:  

$N_{\mathrm{o}\mathrm{u}\mathrm{t}}\approx N{\prime }_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}},$

(11)

and the rectification process would decrease the SNR:  

$\begin{array}{c}\frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}\approx \frac{S_{\mathrm{i}\mathrm{n}}}{N{\prime }_{\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}}\\ \frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}<<\frac{S_{\mathrm{i}\mathrm{n}}}{N_{\mathrm{i}\mathrm{n}}}\end{array}$

12

 

Consequently, in low external noise conditions (small N_in) the rectification process would affect the observer's performance. However, if the input CM noise is relatively high (N_in>>N′_rect), then the noise added by the rectification (N′_rect) would not be significant:  

$N_{\mathrm{o}\mathrm{u}\mathrm{t}}\approx N_{\mathrm{i}\mathrm{n}},$

(13)

and the rectification would not affect the SNR:  

$\frac{S_{\mathrm{o}\mathrm{u}\mathrm{t}}}{N_{\mathrm{o}\mathrm{u}\mathrm{t}}}\approx \frac{S_{\mathrm{i}\mathrm{n}}}{N_{\mathrm{i}\mathrm{n}}}$

(14)

 

Therefore, if the SNRs of a LM and a CM stimulus are equal, then in high noise conditions the rectification should not affect the SNR of the CM stimulus. If both stimuli are treated by common post-rectification mechanisms, then the same sensitivity should be observed in high external noise conditions (negligible internal noise) for both stimulus types, which would result in similar CEs. 

Methods

Subjects

Four volunteers aged between 26 and 35 years participated in the study. Their vision was normal or corrected to normal. 

Stimuli

To compare the subjects' performance on the detection of LM and CM stimuli, the luminance average and contrast average over the entire stimulus were the same for both stimulus types. The only difference between the two stimuli was that, for the LM stimulus, the modulation (signal + external noise) was applied to the luminance profile while the contrast remained constant throughout the stimulus and vice versa for the CM stimulus. Mathematically, the luminance of the pixel at position (x,y) for the LM stimuli was defined by the addition of a texture to a luminance modulation:  

$L_{\mathrm{L}\mathrm{M}}(x,y)=L_0\left[M\right(x,y)+T(x,y\left)\right],$

(15)

where L₀ is the stimulus luminance average, which was fixed to 59 cd/m² for the present study. M(x,y) and T(x,y) represent, respectively, the modulation and the texture of the pixel at position (x,y). The texture was added to the LM stimulus to give both stimuli the same contrast average. Therefore, the two stimulus types were similar with the exception that for the CM stimulus, the modulation was applied to the texture instead of the luminance:  

$L_{\mathrm{C}\mathrm{M}}(x,y)=L_0[1+M(x,y\left)T\right(x,y\left)\right].$

(16)

 

Because the modulation and the texture should not affect the stimulus luminance average, the average of M(x,y) and T(x,y) over the entire stimulus (all x and y) must be 1 and 0, respectively. 

The present study had the objective of comparing the IEN and CE for LM and CM stimuli sensitivity. To derive the IEN and CE, the signal threshold must be evaluated in different external noise conditions. Indeed, the modulation profile [M(x,y)] was composed of a signal [S(x,y)], which the subject had to detect, embedded in external noise [N(x,y)]:  

$M(x,y)=1+S(x,y)+N(x,y).$

(17)

 

Using this stimulus definition implies that the signal and the external noise are both of the same modulation type. Therefore, the LM and CM IENs were not of the same modulation type. This should remain in mind when comparing the results of the IEN for the LM and CM stimuli detection. Indeed, measuring the IEN actually measures the impact of the internal noise on the task being accomplished, in our case the detection of the signal. However, having the same modulation type for the signal and external noise enables a direct comparison between the LM and CM CEs. Indeed, the CE is the efficiency of detecting the signal embedded in noise. Therefore, the capacity of extracting a LM signal embedded in LM noise can be directly compared with the capacity of extracting a CM signal embedded in CM noise. 

Signal

The signal was a Gabor patch (Figure 4, left), vertically oriented, modulating either the luminance or contrast profile of the stimulus (Equation 18, Figure 5). Because the CM stimulus requires a texture defined by high spatial frequencies relative to the signal, the spatial frequency (f) of the Gabor patch was set to a low spatial frequency of 1 cycle per degree (cpd). The phase (p) of the sine wave was randomly set at each stimulus presentation. The standard deviation (σ) of the Gabor patch was set to 1 deg of visual angle,  

$S(x,y)=c\sin (fx+p)\exp (-\frac{x^2+y^2}{2{\sigma }^2}),$

(18)

 

Figure 4

 

Gabor patch signal or envelope (left). Gaussian-filtered noise (right).

Figure 5

 

LM (left) and CM (right) Gabor patch with three types of carriers: plaid (top), checkerboard (center), and binary noise (bottom).

where c represented the contrast of the signal, which corresponds to the Michelson contrast once the signal [S(x,y)] is integrated in the modulation (M(x,y), Equation 17). The contrast (c) was the dependent variable. 

Carriers

Because the present study evaluates the IEN, the carrier used should be chosen to minimize the masking effect on the LM stimulus. Therefore, we chose a carrier for which its spatial frequency did not interfere with the signal spatial frequency (1 cpd). The noise that the carrier will introduce at the signal frequency will be intrinsically present in the IEN evaluated. Therefore, the first carrier used was a plaid composed of two perpendicular oblique sine waves of 7.54 cpd. At each stimulus presentation, the phase of both sine waves varied randomly. The amplitude of the sine waves was set so that the brightest and darkness peeks of the unmodulated texture [T(x,y)] was −0.5 and 0.5. Therefore, the difference of luminance between the two peeks was equal to the stimulus luminance average (L₀). In other words, for both stimulus types, the contrast average of the whole stimulus was set equal to the luminance average (L₀) of the whole stimulus (Figure 6). 

Figure 6

 

Three different types of carriers: plaid (top), checkerboard (center), and binary noise (bottom).

A disadvantage of using a plaid as a carrier is that some values are near 0. Because, for the CM stimuli, the carrier is multiplied with the signal and the noise, low values decrease both the signal and the noise, which do not affect the performance of an ideal observer. However, for a human observer, having internal noise makes the detection of low signal strength in low external noise undetectable. To maximize the contrast of the carrier, a checkerboard was also used. Note that a plaid can be seen as a smoothed checkerboard. The element size was 6 × 6 pixels (0.094 deg of visual angle). 

The third carrier used was the most widely used: binary noise. The element size was also 6 × 6 pixels. Therefore, the only difference between this carrier and the previous one is that its element positions are randomized. 

External noise

Gaussian-distributed white noise was used (Figure 4, right, and Figure 7). For uncorrelated white noise, the spectral density curve as a function of the spatial frequency is flat (Legge et al., 1987; Pelli, 1981, 1990). For LM stimuli, noise frequencies far from the signal frequency have little impact on the detection of the signal. However, for the CM stimulus, because the noise modulation type is in contrast modulation, the noise should affect the signal (1 cpd) but not the carrier (7.54 cpd). Therefore, a band-pass filter (>0.5 and <2 cpd) was applied to the noise. This filter did not change the noise energy at the signal frequency. 

Figure 7

 

LM (left) and CM (right) noise with three types of carriers: plaid (top), checkerboard (center), and binary noise (bottom).

As shown in the Results section, subjects had a high IEN for the CM stimuli detection. To vary significantly the total amount of noise (IEN plus external noise) to derive the IEN and CE, a large amount of external noise had to be used. Filtering the external noise also had the benefit of reducing the luminance contrast range used by the noise, which permitted to increase the external noise energy at 1 cpd without truncating luminance values. 

The contrast root mean square (N) of the noise was set to 0, 0.0125, 0.050, and 2.00 for LM stimuli and 0, 0.050, 2.00, and 4.00 for CM stimuli. More noise was introduced for CM stimuli because of two reasons. First, because the modulation was multiplied with the texture ranging between −0.5 and 0.5, two times more noise could be used without exceeding the monitor luminance range. Second, as mention above, CM had more IEN so greater external noise was required to derive the IEN and CE. 

Ideal observer

As stated previously, an ideal observer is a theoretical observer mathematically computing the optimal solution. Consequently, defining an ideal observer generally consists in deriving the smallest SNR (k_ideal) sufficient to perform a task for a given threshold criterion. However, the present section will only show that an ideal observer has the same sensitivities to both LM and CM stimuli. Because the CE (k_ideal/k) is defined relative to the optimal SNR (k_ideal) and that the optimal SNR is the same for both LM and CM stimuli (k_{ideal LM} = k_{ideal CM}), deriving the exact optimal SNR is not necessary and is beyond the scope of the present study. The relative difference between CEs of LM and CM stimuli may be compared directly:  

$\frac{\mathrm{C}{\mathrm{E}}_{\mathrm{L}\mathrm{M}}}{\mathrm{C}{\mathrm{E}}_{\mathrm{C}\mathrm{M}}}=\frac{k_{\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{a}\mathrm{l}\mathrm{L}\mathrm{M}}/k_{\mathrm{L}\mathrm{M}}}{k_{\mathrm{i}\mathrm{d}\mathrm{e}\mathrm{a}\mathrm{l}\mathrm{C}\mathrm{M}}/k_{\mathrm{C}\mathrm{M}}}=\frac{k_{\mathrm{C}\mathrm{M}}}{k_{\mathrm{L}\mathrm{M}}}$

(19)

 

In other words, if the CE for detecting LM stimuli is the same as the CE for detecting CM stimuli (CE_LM = CE_CM), then the SNR required for detecting LM stimuli will be equal to the SNR required for detecting CM stimuli (k_LM = k_CM). 

The luminance profile of the LM stimuli may be given by combining Equations 15 and 17:  

$L_{\mathrm{L}\mathrm{M}}(x,y)=L_0[1+S(x,y)+N(x,y)+T(x,y\left)\right],$

(20)

and for CM stimuli by combining Equations 16 and 17:  

$L_{\mathrm{C}\mathrm{M}}(x,y)=L_0[1+(1+S(x,y)+N(x,y)\left)T\right(x,y\left)\right].$

(21)

 

Because the average luminance (L₀) is constant in all the testing conditions, it may be abstracted from the stimulus equation and the stimuli may be defined by their contrast function C(x,y) (Linfoot, 1964) instead of their luminance function [L(x,y)]:  

$C(x,y)=L(x,y)/L_0-1.$

(22)

 

The contrast functions of LM and CM stimuli are as follows:  

$C_{\mathrm{L}\mathrm{M}}(x,y)=S(x,y)+N(x,y)+T(x,y),$

(23)

 

$C_{C\mathrm{M}}(x,y)=[1+S(x,y)+N(x,y\left)\right]T(x,y).$

(24)

 

The expected contrast profile when there is no signal (S(x,y) = 0) is T(x,y) and is known to the ideal observer. Therefore, it may be subtracted from the stimulus equation without affecting the ideal observer's performance:  

$C\prime (x,y)=C(x,y)-T(x,y),$

(25)

 

$C{\prime }_{\mathrm{L}\mathrm{M}}(x,y)=S(x,y)+N(x,y),$

(26)

 

$C{\prime }_{\mathrm{C}\mathrm{M}}(x,y)=\left[S\right(x,y)+N(x,y\left)\right]T(x,y).$

(27)

 

Consequently, the performance of an ideal observer to C′_LM(x,y) and C′_CM(x,y) is identical to the one using C_LM(x,y) and C_CM(x,y), respectively. 

The tasks for an ideal observer may be summarized as detecting an LM signal [S(x,y)] in LM noise [N(x,y)] or detecting a CM signal [S(x,y)T(x,y)] in CM noise [N(x,y)T(x,y)]. 

Because the ideal observer has perfect knowledge of the texture [T(x,y)], such an observer may remove it from computation. Therefore, another equivalent CM stimulus may be defined as:  

$\begin{array}{c}C{″}_{\mathrm{C}\mathrm{M}}(x,y)=C{\prime }_{\mathrm{C}\mathrm{M}}(x,y)/T(x,y)\\ C{″}_{\mathrm{C}\mathrm{M}}(x,y)=S(x,y)+N(x,y)\end{array}.$

(28)

 

Consequently, the performance to C″_CM(x,y) will be identical to C′_CM(x,y), which is the same as using C_CM(x,y). 

Because C′_LM(x,y) = C″_CM(x,y) and the ideal observer performance to L_LM(x,y) and L_CM(x,y) is identical to the one using C′_LM(x,y) and C″_CM(x,y), respectively, the performance of an ideal observer using L_LM(x,y) will be equal to the one using L_CM(x,y) as long as it has perfect knowledge of the texture [T(x,y)]. Consequently, for an ideal observer, detecting a LM signal [S(x,y)] in LM noise [N(x,y)] is equivalent as detecting a CM signal [S(x,y)T(x,y)] in CM noise [S(x,y)N(x,y)]. In other words, an ideal observer will require the same SNR (k_ideal) for detecting both stimulus types. 

To be abstracted, a texture must be precisely known [i.e., the texture values T(x,y) at all the positions (x,y) must be known]. For the plaid carrier, the frequency, amplitude, and orientation of the carrier are known except that its phases change randomly at each presentation. However, the phase can easily be deduced precisely because the spatial frequency of the carrier is higher than the rest of the stimulus (signal and noise). Consequently, the local variation only depends on the carrier. For LM stimuli, the signal and noise, which are at lower spatial frequencies, will only change the local mean luminance. For CM stimuli, the signal and the noise will only change the local carrier contrast. Therefore, the phase of the plaid can be detected and the value of the texture [T(x,y)] can be precisely computed at each pixel position (x,y) and abstracted from the equation. 

For a checkerboard and a binary noise carriers, each position (x,y) may have two possible values: −0.5 and 0.5. Because it is impossible to have negative luminance pixel values, the luminance range of each pixel [L(x,y)] must be, for a symmetrical reason, between 0 and 2L₀. Based on these constraints and on Equations 3 and 4, the modulation [M(x,y)] can theoretically range between 0.5 and 1.5 for LM stimuli, and between 0 and 2 for CM stimuli. For both LM and CM stimuli, a texture element [T(x,y)] of −0.5 or 0.5 will cause the luminance value at that same position to be bellow or above the luminance average (L₀), respectively. Consequently, an ideal observer can precisely recompute the original texture for all the carriers used in the present study. 

Procedure

Hardware

The monitor, which was the only luminance source in the room, was a 19-in. ViewSonic E90FB .25 CRT screen and was calibrated using a Minolta CS100 photometer. A Pentium 4, 3.2 GHz with a 10-bit Matrox Parhelia512 graphic card computed the stimuli. This graphic card was not limited in the number of simultaneously displayed color and therefore could display 1024 different grey levels for a given image. This was necessary because the detection threshold of the LM stimuli in certain conditions was relatively low. The distance between the monitor and the subject was 1.14 m and each pixel on the screen was 0.016 × 0.016 deg of visual angle. 

Psychophysical methods

The constant stimuli paradigm was used to evaluate the subjects' threshold in different conditions using a two-interval force-choice procedure. A block was composed of 28 trials: 1 stimulus type (either LM or CM), 4 noise conditions, and 7 signal-contrast levels. Five pseudorandom blocks were performed before the subject was free to rest. At that time, the stimulus modulation type was switched and the subject was advised of this change. Therefore, LM and CM stimuli alternated until 20 blocks of each stimulus type were performed. 

Data analysis

For each subject, each stimulus type, each carrier, and each noise condition, the detection threshold (75% correct) was evaluated by fitting a Weibull function using the bootstrap technique. Afterwards, for each subject, each stimulus type, and each carrier, Equation 4 was fitted to deduce the two parameters: k and N_eq. The fitting was achieved by minimizing an error function using Excel Solver (Newton method). The error function was the sum over each noise condition of the squared difference in log units between the detection threshold and the predicted threshold by Equation 4. 

Results

Figure 8 shows individual results when a plaid was used as a carrier. All observers had similar patterns. In no- or low-external noise conditions, detection thresholds for CM stimuli were higher than those for LM stimuli. However, the threshold differences were generally not significant in high-noise conditions. Figures 9 and 10 show the results for the checkerboard and binary noise carriers. Although the detection threshold differences in no- or low-external noise conditions are smaller, the same pattern was observed. These results strongly suggest that all observers had similar CEs but different IENs for processing LM and CM stimuli. These results are confirmed in Table 1 where the differences of IEN and CE between LM and CM stimuli processing are listed. The differences of CEs are relatively small (0.02, −0.09, and 0.07 log units using a plaid, checkerboard, or binary noise, respectively, as a carrier) compared to the differences of IENs (1.03, 0.61, and 0.33 log units, respectively). We can also note that the differences in IENs were not the same in all three conditions. 

Table 1

 

IEN and CE differences between LM and CM stimuli (CM IEN or CE minus LM IEN or CE in log units). The last row corresponds to the standard error.

	Plaid		Checkerboard		Binary noise
	IEN	CE	IEN	CE	IEN	CE
ela	1.10		0.02	0.44	−0.30	0.53	0.19
Il	0.99	−0.03	0.57	−0.17	0.20	0.04
mer	0.98	0.12	0.71	0.12	0.25	−0.03
ra	1.04	−0.04	0.72	−0.01	0.33	0.09
Mean	1.03	0.02	0.61	−0.09	0.33	0.07
±	0.03	0.04	0.08	0.11	0.08	0.05

Figure 8

 

Individual results using a plaid as a carrier. The error bars were calculated using bootprob.

Figure 9

 

Individual results using a checkerboard as a carrier. The error bars were calculated using bootprob.

Figure 10

 

Individual results using binary noise as a carrier. The error bars were calculated using bootprob.

Figure 11 shows the averaged CE over all observers in all the conditions. As shown, neither the stimulus type nor the carrier type affected the CE. These results are not surprising for LM stimuli because the carrier only plays a masking role. In high noise conditions, the impact of the carrier (or mask) becomes not significant. Therefore, there is no reason to expect a threshold difference in high-noise conditions for LM stimuli. What is more surprising is that, in high-noise conditions, the same thresholds were observed for CM stimuli and this was true for all carriers. 

Figure 11

 

Relative CE for LM and CM stimuli using three different carriers: plaid, checkerboard, and binary noise. Error bars show the standard error. Note that for comparative reasons, the same range was used as in Figure 12.

Differences in IEN where noted when using different carriers (Figure 12). For LM stimulus sensitivities, using a checkerboard as a carrier resulted in slightly more IEN than using the plaid but much less than when using binary noise. These results are not surprising given the carrier's masking role for LM stimuli. For CM stimulus sensitivities, the checkerboard generated the least amount of IEN. 

Figure 12

 

IEN for LM and CM stimuli using three different carriers: plaid, checkerboard, and binary noise. Error bars show the standard error.

Discussion

Same calculation efficiencies

The results of the current study clearly show that there is no, or very little, difference in CE between LM and CM stimuli detection. This implies that observers are just as efficient for extracting LM signals from LM noise as they are for extracting CM signals from CM noise. Consequently, these results suggest that, after a second-order rectification, both stimulus types are processed by common mechanisms. 

However, Georgeson and Schofield (2002) found evidence suggesting that LM and CM stimuli are not merged or confused after a rectification process because recognition (LM vs. CM) and detection (LM vs. noise and CM vs. noise) threshold for LM and CM stimuli are similar. We argue that this does not imply that both stimuli are treated by separate post-rectification mechanisms. We suggest that although the same mechanisms are processing two stimuli that were initially treated by separate mechanisms, the different properties of the two stimuli are not necessarily lost. 

For example, if we compare the detection and recognition of two LM gratings with opposite phases, there is no reason to expect a difference between the two. Initially, it is not the same neuron population treating the two stimuli. Therefore, the two pathways are initially different. However, it would be unlikely that different mechanisms would be treating the two stimuli at a later stage. Consequently, the properties of the two stimuli are still available after initial processing although both stimuli are, at a later stage, processed by common mechanisms. 

Different IENs

The results clearly attribute the difference between LM and CM sensitivities to a difference of IEN. If the same mechanisms treat both stimulus types after a second-order rectification, then the difference in IEN must either come from internal noise prior to the rectification or be caused by a suboptimal rectification. To address this question, an ideal observer with pre-rectification IEN will be considered. 

Based on the definition described above for LM and CM stimuli, an ideal observer has the same performance for both stimuli. This is true given that an ideal observer does not have any internal noise. However, when detecting CM stimuli, before the rectification process, some internal noise (e.g., optical noise) is likely to be added to the stimulus. In the presence of high external noise (LM or CM), the internal noise will be negligible and the performance will not be significantly affected. Therefore, the CE would be unaffected by pre-rectification noise. In the absence of external noise, however, the CM detection task consists in detecting a CM signal [S(x,y)T(x,y)] in LM noise [N(x,y)] compared with the LM detection task, which consists in detecting a LM signal [S(x,y)] in LM noise [N(x,y)]. Therefore, the two tasks would only differ by their signal [S(x,y) vs. S(x,y)T(x,y)]. 

When the task is to detect a LM signal [S(x,y)] in LM noise [N(x,y)], the energy of the signal may be defined as (Legge et al., 1987; Pelli, 1981, 1990):  

$E_{\mathrm{L}\mathrm{M}}=\underset{Y}{\int }\underset{X}{\int }{S}^2(x,y)\mathrm{d}x\mathrm{d}y,$

(29)

where X and Y are the width and height of the image, respectively. 

When the task is to detect a CM signal [S(x,y)T(x,y)] in LM noise [N(x,y)], the energy of the signal may be defined as  

$E_{\mathrm{C}\mathrm{M}}=\underset{Y}{\int }\underset{X}{\int }\left[S\right(x,y\left)T\right(x,y){]}^2\mathrm{d}x\mathrm{d}y.$

(30)

 

If there is no statistical relation between the texture [T(x,y)] and the signal [S(x,y)], the signal energy of CM stimuli can be  

$E_{\mathrm{C}\mathrm{M}}\approx \frac{\underset{Y}{\int }\underset{X}{\int }{S}^2(x,y)\mathrm{d}x\mathrm{d}y\underset{Y}{\int }\underset{X}{\int }{T}^2(x,y)\mathrm{d}x\mathrm{d}y}{XY}.$

(31)

 

Consequently,  

$E_{\mathrm{C}\mathrm{M}}\approx E_{\mathrm{L}\mathrm{M}}\frac{\underset{Y}{\int }\underset{X}{\int }{T}^2(x,y)\mathrm{d}x\mathrm{d}y}{XY}.$

(32)

 

Because the RMS contrast of the texture is  

$T_{\mathrm{R}\mathrm{M}\mathrm{S}}=\sqrt{\frac{\underset{Y}{\int }\underset{X}{\int }{T}^2(x,y)\mathrm{d}x\mathrm{d}y}{XY}},$

(33)

and the energy of the CM stimulus is  

$E_{\mathrm{C}\mathrm{M}}\approx E_{\mathrm{L}\mathrm{M}}{T}_{\mathrm{R}\mathrm{M}\mathrm{S}}^2.$

(34)

 

In other words, for the same signal [S(x,y)], the energy of the LM stimulus [S(x,y)] will be 1/T_RMS² times greater (T_RMS² < 1 because −1 < T(x,y) < 1) than the energy of the CM stimulus [S(x,y)T(x,y)]. Because the energy is proportional to the squared contrast, to have the same energy level as the LM stimulus the CM contrast must be 1/T_RMS times greater. Therefore, in the same noise condition [N(x,y)], the LM sensitivity of the ideal observer's with LM internal noise would be 1/T_RMS times greater than the CM sensitivity. 

Consequently, simulating early (pre-rectification) noise causes an ideal observer to have 1/T_RMS times more IEN for CM stimuli detection than for LM stimuli detection. Comparing LM and CM IENs (or sensitivities), if we suppose that the significant internal noise occurs prior to the rectification, a factor of T_RMS should be considered. Therefore, pre-rectification IEN (N_{eq pre-rect}) may be defined as:  

$N_{\mathrm{e}\mathrm{q}\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{t}}=\{\begin{array}{cc}{N}_{\mathrm{e}\mathrm{q}}& \mathrm{}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{L}\mathrm{M}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{l}\mathrm{i}\\ N_{\mathrm{e}\mathrm{q}}{T}_{\mathrm{R}\mathrm{M}\mathrm{S}}& \mathrm{f}\mathrm{o}\mathrm{r}\mathrm{C}\mathrm{M}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{u}\mathrm{l}\mathrm{i}\end{array}$

(35)

 

The RMS contrast (T_RMS) of the plaid, checkerboard, and binary noise carriers were 0.25, 0.5, and 0.5, respectively. Using binary noise as a carrier, the results (0.33 log) show a difference of IEN (N_eq) near the factor predicted by the ideal observer with LM internal noise (2 or 0.30 log). Therefore, by compensating for the texture contrast (N_{eq pre-rect}), there was no significant difference between LM and CM IEN (Figure 13). These results suggest that the significant noise occurred prior to the rectification and was common to both tasks. 

Figure 13

 

Pre-rectification IEN for LM and CM stimuli using three different carriers: plaid, checkerboard, and binary noise. Assuming that the main portion of internal noise occurs prior to the rectification, the IEN for CM stimuli was multiplied by the texture RMS. Error bars show the standard error.

One particularity of the binary noise carrier is that it has intrinsic noise. As shown in the Methods section, this noise does not affect the ideal observer performance. However, it does affect the performance of a human observer because the LM stimuli detection threshold is greater using the binary noise carrier than the checkerboard. Note that the only difference between the checkerboard and binary noise carriers is the randomness of the elements' position. From these results, we conclude that intrinsic noise of the binary noise carrier increases the IEN for LM and CM stimuli detection. Therefore, the IEN measured when using this type of carrier could be caused by the carrier intrinsic noise. 

Using a plaid or a checkerboard as a carrier, the ideal observer with pre-rectification internal noise does not explain the difference between the IEN measured for LM stimuli and the one measured for CM stimuli (Figure 13). Consequently, the important difference between internal noises does not occur before the rectification. If both stimuli are processed by common mechanisms after the rectification process (same CE) and the pre-rectification noise cannot explain the difference of IEN observed, the difference of IEN must come from a suboptimal rectification process. This suggests that, for binary noise, the intrinsic noise of the carrier was greater than the IEN caused by the suboptimal rectification, and therefore the IEN introduced by the suboptimal rectification was not significant. 

Conclusion

One of the main differences between LM and CM stimuli processing is that the human observer is less sensitive to CM than LM stimuli. We address this question by decomposing the sensitivities in IENs and CEs. The results show no difference of CE and indicate that the IEN is responsible for the sensitivity difference. Based on a rectification model, these results support the hypothesis that the two stimulus types could be treated by common mechanisms after a second-order rectification process. 

To investigate the main source of internal noise for CM stimuli detection, an ideal observer with pre-rectification internal noise was built. Based on ideal observer analysis, pre-rectification internal noise could explain the difference of IEN between LM and CM stimuli detection when using binary noise as a carrier but not when using a plaid or a checkerboard. We conclude that a suboptimal rectification process causes higher IEN for CM stimuli detection compared with LM stimuli detection and that the intrinsic noise of the binary carrier had a greater impact on the IEN than the suboptimal rectification. 

Acknowledgments

This research was supported by NSERC graduate fellowship to RA and NSERC operating grant to JF. 

Commercial relationships: none. 

Corresponding author: Rémy Allard. 

Email: [e-mail]. 

Address: 3744 Jean-Brillant, Montréal, Québec, Canada H3T 1P1.