Isolating objective and subjective filling-in using the drift diffusion model

Abstract

Spatial context is known to influence the behavioral sensitivity (d′) and the decision criterion (c) when detecting low-contrast targets. Of interest here is the effect on the decision criterion. Polat and Sagi (2007) demonstrated that, for a Gabor target positioned between two similar co-aligned high-contrast flankers, the observers’ reports of seeing the target (Hit and False Alarm) decreased with increasing target–flanker distance. This effect was more pronounced when the distance was randomized within testing blocks compared to when it was fixed. According to signal detection theory (SDT), the latter result suggests that the decision criterion is adjusted to a specific distance-dependent combination of signal (S) and noise (N) when the S and N statistics are fixed, but not when they vary across trials. However, SDT cannot differentiate between changes in the decision bias (the criterion shift) and changes introduced by variations in S and N (the signal and noise shift). To circumvent this limitation of SDT, we analyzed the reaction time (RT) data within the framework of the drift diffusion model (DDM). We performed an RT analysis of the target–flanker interactions using data from Polat and Sagi (2007) and Zomet et al. (2008; 2016). The analysis revealed a stronger dependence on flankers for faster RTs and a weaker dependence for slower RTs. The results can be explained by DDM, where an evidence accumulation process depends on the flankers via a change in the rate of the evidence (signal and noise shift) and on observers’ prior knowledge via a change in the starting point (criterion shift), leading to RT-independent and RT-dependent effects, respectively. The RT-independent distance-dependent response bias is attributed to the observers’ inability to learn multiple internal distributions required to accommodate the distance-dependent effects of the flankers on both the signal and noise.

Introduction

Detection of an oriented target improves in the presence of similar, co-aligned, high-contrast flankers (Morgan & Dresp, 1995; Polat & Sagi, 1993; Polat & Sagi, 1994; Solomon & Morgan, 2000; Woods, Nugent, & Peli, 2002). For oriented Gabor targets, contrast sensitivity is doubled when the distance between the target and flankers is about three times the Gabor wavelength (Polat & Sagi, 1993). These spatial interactions are suggested to be a manifestation of the brain processes involved in contour filling-in, in texture segmentation, and in perceptual grouping (i.e., contour integration) (Sagi, 1995; Zhaoping & Jingling, 2008). The earlier experiments, cited above, used the bias-free, two-alternative forced choice (2AFC) method, considered efficient in estimating visual sensitivity (d′); however, 2AFC provides no insights into the perceived quality of targets, which is expected to be affected by filling-in processes (Anstis, 2010). Polat and Sagi (2007), employing the Yes/No method, found, in addition to detection facilitation, a distance-dependent detection bias; observers’ tendency to report “target present” increased at short target–flanker distances regardless of the presence of the target (Hit) or absence (False Alarm [FA]). This suggests that the gap between flankers is filled in with task-relevant information, supporting the “filling-in” hypothesis. Both the increased target sensitivity and the observed detection bias are thought to be caused by lateral interactions in the visual cortex, activated by the flankers. Report biases are also affected by decision strategies, possibly related here to statistical priors derived from the known characteristics of natural images (Geisler, Perry, Super, & Gallogly, 2001). To better understand the contributions of lateral interactions and decision strategies to the detection bias, we present a reaction time (RT) analysis of the experimental results collected in the previous Yes/No experiments (Polat & Sagi, 2007; Zomet, Amiaz, Grunhaus, & Polat, 2008; Zomet, Polat, & Levi, 2016). The data were modeled using signal detection theory (SDT) (Green & Swets, 1966) and the drift-diffusion model (DDM) (Ratcliff & McKoon, 2008; Ratcliff, Smith, Brown, & McKoon, 2016; Shadlen & Kiani, 2013). In the subsequent sections, we elucidate the unified SDT–DDM framework employed to model the data. 

The SDT approach

Following SDT, it is assumed that observers base their decisions on noisy sensory activity within the brain (referred to as the “internal response”), monotonically increasing with stimulus strength. For low-contrast targets, the internal target response distribution, referred to as Signal, or p_S(x), may overlap with the internal noise distribution representing no target, termed Noise, or p_N(x), thus leading to detection errors (Figure 1). Consequently, Yes responses can be correct (Hit, where P_Hit = the area under the green shaded curve in Figure 1A) or incorrect (FA, where P_FA = the area under the red-shaded curve in Figure 1A). SDT provides tools to compute a decision criterion from the P_Hit and P_FA values, which is the normalized internal response level above which the observer produces a Yes decision (denoted by the blue vertical line in Figure 1A). This criterion is assumed to be observer dependent, and it can shift according to task demands and the stimulus properties available to the observer. However, when observing a specific change in the Hit and FA rates, such as the increased rates seen in our experiments, SDT cannot distinguish between two potential causes: (1) a shift in the decision criterion toward lower response levels (Figure 1B), or (2) an elevation in activity levels at the target location, shifting both the signal and noise distributions to higher response levels (Figure 1C). In the realm of SDT, the first cause (a criterion shift) is believed to depend on the observers’ decision strategies, which are flexible and aim to optimize the task outcome, including the error rates, costs, and values, making the task outcome inherently subjective. On the other hand, the second cause is deemed sensory driven, or objective, and is tied to the stimulus (such as flankers), influencing the response of the system. Regarding the experimental paradigm studied here, one might anticipate an increase in Yes responses in the presence of flankers due to (1) the observers’ expectation for the gap between flankers to be filled in, or (2) the increased sensory activity at the target location (as in Figure 1C), prompted by the input from the flankers. Both causes might reflect adaptation of the visual system to the statistics of edge co-occurrence in natural images (Geisler et al., 2001). 

Figure 1.

 

Illustration of the problem addressed in this work. (A) The standard SDT scheme: p_N(x), the noise distribution (the broken curve); p_S(x) the signal distribution (the continuous curve); and the decision criterion (the blue vertical line). The regions of Hit and FA are highlighted in green and red, respectively; the areas under the highlighted segments are P_HIT and P_FA. (B) The decision criterion is shifted toward lower internal responses, thus increasing P_HIT and P_FA. (C) p_N(x) and p_S(x) are shifted toward higher activity levels without a criterion shift relative to (A), producing the same increase in P_HIT and P_FA as in (B). Our goal here is to decide between (B) and (C).

Linking SDT to DDM

To disentangle the inherent ambiguity within SDT and distinguish between the contributions driven by the sensory activity and those influenced by the decision criteria, we examined the experimental results using the DDM framework (Ratcliff & McKoon, 2008; Ratcliff et al., 2016; Shadlen & Kiani, 2013). Unlike SDT, DDM introduces a temporal dimension to the decision process, allowing RT-based predictions. According to DDM, observers accumulate evidence both in favor of and against the presence of a target. Early models (Gold & Shadlen, 2001; Link & Heath, 1975; Stone, 1960) followed Wald's (1947) sequential probability ratio test, suggesting that each time interval produces a log-likelihood ratio (LLR) value. This value assesses the odds ratio for one stimulus being present versus the other, and it accumulates over time intervals until a decision is triggered. This occurs when the accumulated value reaches one of two thresholds (bounds)—for example, +a or –a for positive or negative decisions, respectively. The starting point of the accumulator (sp) can be selected to incorporate expectations, prior information (e.g., in the present context, the statistics of natural images), and the subjective value of the decision, such as payoff and reward. The rate of evidence accumulation, termed the “drift rate,” increases with the target sensitivity, resulting in faster attainment of the decision bounds. When the internal response offers no evidence of the target presence or absence, such as when p_S(x) = p_N(x), the drift rate (v) is zero. Positive and negative drift rates correspond to target-present and target-absent trials, respectively. Thus, within the SDT framework, we assume that LLR(x) = log[p_S(x)/p_N(x)] is integrated over time, where p_S(x) and p_N(x) (as illustrated in Figure 1) represent the momentary distributions of the sensory evidence (x) in the signal (S) and noise (N) trials. More formally, for a time-varying response x(t) and an accumulated value L, we have L(t) = L(t – 1) + LLR[x(t)], for all t > 0, with L(0) = sp. The mean drift rate (v) in the S and N trials (v_S and v_N, respectively) is assumed to be proportional to the expected value of LLR[x(t)] over the corresponding S and N trials. A decision is reached when L(t) ≥ a (a positive decision) or when L(t) ≤ –a (a negative decision). Importantly, note that the effect of L(0) on L(t) is expected to diminish with time as L(t) accumulates evidence and noise (Dekel & Sagi, 2020b). 

Although offering an efficient method for sequential hypothesis testing, a critical limitation of the likelihood model lies in its requirement for knowledge of the signal and noise distributions, necessary for each x(t) value so that LLR[x(t)] can be accumulated. This necessity is often deemed challenging, if not unattainable, particularly in typical psychophysical experiments characterized by a limited number of trials. An alternative approach, employed by DDM, directly integrates the sensory evidence (Ratcliff & McKoon, 2008; Shadlen & Kiani, 2013). Gold and Shadlen (2001) proposed the difference between the momentary response and the criterion level as an alternative to the likelihood ratio computation, although the method of criterion setting is left open. The approach presented here explicitly assumes, as described below, that, in uncertain environments where observers encounter diverse stimuli with varying internal distributions, they fail to accurately estimate these distributions. Consequently, they base their decision on a mixed distribution, applying a single decision criterion to all stimuli (Gorea, Caetta, & Sagi, 2005; Gorea & Sagi, 2000). 

Interacting decision criteria

Consider the case where various stimuli are presented in an experiment, yielding stimulus-dependent S and N distributions, when there are varying target–flanker distances between trials (Mix condition) (Figure 2). The findings of Gorea and Sagi (2000) suggest that observers are unable to learn the individual distributions, as required for optimal performance. Instead, they merge all S and N distributions (related to the different distances) into single S and N distributions, estimated to represent the average of the individual distributions. For the decision-making process, observers employ only one criterion that is optimized for these single S and N distributions. Consequently, it is predicted that only one accumulator is used in the mixed condition, with the estimated evidence for or against target presence being blind to the originating, distance-dependent distribution. Therefore, we expect zero evidence (i.e., the criterion) to correspond to the presence/absence of targets regardless of the specific flanker configurations. In essence, this is determined globally by amalgamating the diverse distributions related to the various target–flanker distances (as depicted in Figure 3). 

Figure 2.

 

Lateral masking and the SDT criterion. (A) Stimuli used in Polat and Sagi (2007). Observers detected the presence versus the absence of the low-contrast target stimulus presented between two high-contrast flankers aligned with the target (except for the 15λ condition used for reference). The distance between the target and flankers was varied (3λ, 4λ, 6λ, 9λ, 12λ, and 15λ) between blocks of trials (the Fix condition) or within one block of mixed trials (the Mix condition). (B, C) Group results from Polat and Sagi (2007) (Figure 3) showing the effect of flankers on target detection, as measured by SDT (B) criterion (Cr) and (C) sensitivity (d′). Shown are the means ± SEM across seven observers (see Methods). Note the close to uniform criterion level in the Fix condition compared with the larger range in the Mix condition. d′ does not differ much between conditions, although it is somewhat lower in the Mix condition, possibly due to the increased uncertainty involved in this condition. The d′ curves show lateral facilitation at shorter distances, although they are smaller than the typical facilitation observed with the standard 2AFC method (Polat & Sagi, 1993; Polat & Sagi, 2007).

Figure 3.

 

The proposed SDT modeling of the measured criterion change. (A) Internal response distributions thought to be involved in the studied task (normal distributions, σ = 1). There are six noise distributions (broken lines) and six signal distributions (continuous lines), with the shortest and longest target–flanker distances denoted in red and blue, respectively. We assumed that observers cannot estimate all of these distributions when encountered randomly within a block of trials, and instead we used the mean distributions of the noise and signal (the broken and continuous black curves, respectively). (B) LLR functions computed for the noise and signal pairs in (A). We expected a Yes response when LLR > 0 and No otherwise. It was assumed that observers based their decisions on the black curve (the LLR of the means) instead of the specific distance curves; thus, they are biased. (C) Predicted criterion shifts derived from (B). For the Fix condition, we assumed ideal observers (unbiased). (D) The d′ values associated with the distribution pairs are presented in (A). These values are computed as the difference between the paired signal and noise distributions.

In this study, we undertook an analysis of the task involving the detection of a low-contrast Gabor patch in the presence of flankers (Figure 2A). The effect of the flankers depends on the target–flanker distance, showing improved sensitivity at intermediate target–flanker distances (2λ < distance < 8λ, where λ is the wavelength of the Gabor stimuli). This phenomenon, known as range-dependent lateral facilitation, induces an impression of filling-in, leading to an elevation in the FA rate in Yes/No detection experiments (Polat & Sagi, 2007). This effect can be quantified by the decision criterion (Figure 2B) as defined in SDT. Although the criterion can be adjusted by observers when different target–flanker distances are blocked (Figure 2B, Fix), it was found to depend on the flankers when the different target–flanker distances are mixed (Figure 2B, Mix). This result can be explained by observers failing to independently adjust multiple criteria when the stimulus strength, which is distance dependent, is randomized within a block of trials, as illustrated in Figure 3. To explain these effects, the flankers can be assumed to increase the response level of the target stimulus population; this effect depends on the target–flanker distance. Figure 3 illustrates the challenge faced by observers in such a task. In each trial, an internal response is drawn from one of 12 distributions, corresponding to six distances, both with and without the target present (6 × 2, as shown in Figure 3A). For an observer to make unbiased decisions, these 12 distributions need to be correctly estimated, with an unbiased criterion set for each distance based on the corresponding pair of distributions (noise and signal). Instead, we propose that observers utilize only two distributions: the average noise distribution and the average signal distribution (denoted by the black curves in Figure 3A). The ratio between these distributions provides the input to their decision: Yes, if LLR(x) > 0; No, otherwise (x is the internal response; see the black curve in Figure 3B). In a blocked condition (where the distance is fixed), observers can possibly estimate the specific distance-dependent distribution and derive an unbiased likelihood ratio value for each distance (illustrated by the colored curves in Figure 3B). The anticipated decision criteria for the example outlined in Figure 3A are depicted in Figure 3C. 

It is also possible that observers bias their decisions for each distance independently, based on prior knowledge about the statistics of filling-in between contour elements, as shown by Gorea and Sagi (2000) for targets having different prior probabilities. Such biases may affect the predictions shown in Figure 3C, depending on the specific priors assigned to different distances. Thus, as outlined earlier, we can anticipate the presence of two contributing sources to the criterion effect: (1) activity based, affecting the statistics of internal responses (leading to shifted p_S(x) and p_N(x) distributions, akin to Figure 1C); and (2) biases corresponding to observers’ prior knowledge of the stimuli and the task at hand (resulting in a shifted criterion, similar to Figure 1B). 

Note that the above discussion concerns the mechanisms of criterion setting and is mute regarding observers’ sensitivity to changes in target contrast. Sensitivity depends on the internal response gain, which corresponds to the difference between the means of the S and N distributions. In the context of SDT, sensitivity is described by d′, which is computed in a way that is assumed to be criterion independent (Figures 2C and 3D). 

A criterion that disappears with RT and that does not disappear with RT

In prior studies involving tasks such as detection, tilt after effect, and tilt illusion, we have demonstrated that perceptual biases arising from shifts in decision criteria (as in Figure 1B) diminish as the RTs lengthen. In contrast, biases attributed to sensory interactions (as in Figure 1C) remain unaffected by RTs (Dekel & Sagi, 2020b). DDM provides a coherent explanation for these phenomena. In the DDM framework, shifts in decision criteria are implemented by modifying the starting point in the evidence accumulation process. Consequently, when the process takes longer to terminate (manifested as a slower RT), decisions are less biased due to noise accumulation (Dekel & Sagi, 2020b). Conversely, alterations in decision criteria arising from sensory interactions are characterized by a change in the rate at which evidence accumulates. As a result, such effects exhibit a minimal dependence on RT. In the present work, we adopted this approach to analyze lateral masking data collected from previous studies which had not been previously subjected to RT analysis (Polat & Sagi, 2007; Zomet et al., 2008; Zomet et al., 2016). 

The present project

The experimental findings previously presented show a clear dependency of decision bias on the target–flanker distance when the different distances are mixed but not so much when the observers are presented with a fixed distance. The mixture distribution model predicts distance-dependent decision biases (i.e., criterion shifts) when distances are mixed, caused by internal-response shifts interfering with the formation of efficient representations of the internal distributions associated with different stimuli. In addition, there may be expectation-dependent decision biases resulting from observers adopting different decision rules for different stimuli. These expectation-based biases are presumed to disappear at slow RTs, whereas biases predicted by the mixture distribution model are expected to be present at slower RTs under experimental conditions where different stimuli (here, different distances) are mixed but not when the distance is fixed. Based on these arguments, we present four predictions: 

• P1. The criterion dependence on distance (Figure 2B) is expected to be larger at faster RTs compared with slower RTs (the effects of starting point). At slower RTs, we expect the criterion to depend on the distance when trials of different distances are mixed (the effect of mixing distributions), but not when the distances are blocked.
• P2. The criterion dependence on distance is expected to be larger at faster RTs due to biases introduced by the starting point of the accumulator, sp = L(0). These biases are reduced with increasing RT due to the accumulation of internal noise (P1). However, high levels of external noise (Zomet et al., 2016), introduced at t = 0, are expected to dominate the internal noise and reduce the effect of the accumulation starting point. Thus, we predict RT to have a reduced effect on the dependence of the criterion on distance.
• P3. The dependence of the criterion on distance is expected to decrease with the decreasing slope of the log-likelihood function presented in Figure 3B. It becomes evident that the slope decreases when the S and N distribution width (σ) is increased. For the specific model presented, assuming normal distributions, the slope is proportional to (‹S› – ‹N›)/σ². Thus, in the presence of increasing noise (increasing σ values), when the S and N means are kept constant, the dependence of the criterion on distance is expected to decrease and to vanish at slower RTs.  When the S and N difference is increased with σ, the dependence of the criterion on distance is expected to be preserved at higher noise levels. These predictions are tested by analyzing the results from experiments where external noise is added to the target (Zomet et al., 2016).
• P4. The dependence of the criterion on distance is expected to be larger at faster RTs; thus, it will be reduced in slower observers (as discussed above, according to DDM, the starting-point–dependent bias decreases with RT) (Dekel & Sagi, 2020a). Here we analyzed data from a group of observers diagnosed for depression (Zomet et al., 2008), showing slower RTs. We expect the faster RTs of this group to have a reduced dependence of criterion on distance.

Next, we will test these predictions. 

Methods

Experimental data

Here, we analyzed unpublished RT data from previously reported experiments (Polat & Sagi, 2007; Zomet et al., 2008; Zomet et al., 2016), as detailed below and summarized in Table 1. All experiments measured the detection of low-contrast vertical Gabor patch “targets” in the presence of two lateral high-contrast Gabor patch “flankers” (Figure 2A). Flankers were located at varying distances from the target (3–15 λ, where λ is the wavelength of the Gabor patch). In all experiments, the 15λ distance used horizontally oriented flankers (Figure 2A), presumably nulling any lateral interaction with the vertical target, whereas all other distances used vertically oriented targets. Auditory feedback was used to denote detection errors. From Polat and Sagi (2007), we recovered the original data for most observers (six and five observers out of seven for Mix and Fix, respectively). In the remaining publications, all of the original data were recovered, as well as an unpublished pilot study employing external noise, as in Zomet et al. (2016), with numerous repetitions per participant. The observers in all the experiments differed, with the exception of those in Polat and Sagi (2007), where five observers were shared between Fix and Mix. The stimulus parameters and experimental setup, detailed below, were nearly identical in the different experiments (see the differences in Table 1). 

Table 1.

 

Summary of the experimental data. ^*The 15λ distance used flankers with a horizontal orientation. ^**Missing data from the original study.

 

Stimuli and procedure Polat and Sagi (2007)

The stimuli consisted of Gabor patches with wavelength λ = 0.11°, modulated from a background luminance of 40 cd·m⁻² (Figure 2) (Polat & Sagi, 2007). Stimuli were presented on a Philips multiscan 107P color monitor (Philips, Amsterdam, the Netherlands) using a PC system. The effective size of the monitor screen was 24 × 32 cm, which at the used viewing distance of 150 cm subtends a visual angle of 9.2° × 12.2°. Observers viewed the stimuli binocularly in a dark cubicle, where the only ambient light came from the display screen. 

Stimuli consisted of a low-contrast Gabor target and two high-contrast (60%) Gabor flankers (Figure 2). The target–flanker distances used were 1λ, 2λ, 3λ, 4λ, 6λ, 9λ, 12λ, and 15λ. Baseline thresholds, against which spatial interactions were compared, were obtained using orthogonal target and flankers with an inter-element distance of 15λ. The contrast of the target was constant for each observer at all target–flanker distances; the exact value depended on the contrast detection threshold for that observer (range 3%–4%). In the Yes/No task, the observers were asked to detect the target that was shown in a single presentation and to report whether the target was present (Yes) or absent (No) by pressing the left and right mouse keys, respectively. A visible fixation circle indicated the location of the target; it disappeared when the trial started. Observers activated the presentation of the trials at their own pace. When the 2AFC task was used, there were two stimulus intervals (80 ms each), presented 800 ms apart, both containing the flankers; however, the target was presented in only one of the intervals. 

There were two main experimental procedures. In the Mix procedure, the trials with different target–flanker distances were presented in random order, whereas in the Fix procedure the different target–flanker distances were blocked. In the Fix procedure, the target–flanker distance was changed randomly between blocks. In both procedures, each distance was presented 50 or 20 (depending on the experiment) times in a session in which the target was present in about half of the trials (a probability of 0.5). 

Stimuli and procedure Zomet et al., 2008

The stimuli and the task (Zomet et al., 2008) were the same as in Polat and Sagi (2007), using the Mix procedure, but with a wavelength where λ = 0.16° and with a stimulus duration of 100 ms. Stimuli were presented on a ViewSonic E70 color monitor with display dimensions as above (ViewSonic, Brea, CA). The two groups (patients and controls) did not differ significantly (p = 0.097) regarding the mean contrast threshold. The mean contrast threshold of the control group was 5.12, and that of the patient group was 6. In each session, 20 trials for each target–flanker separation were presented, with a total of 120 trials per session. 

The patients were hospitalized in the Psychiatry Department at Sheba Medical Center. These patients were diagnosed by psychiatrists as suffering from major depression disorder (MDD), according to the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV). All patients were found to be currently depressed during our testing period and were being treated with antidepressants and benzodiazepine medications (for more details, see Zomet et al., 2008). 

Stimuli and procedures Zomet et al., 2016

The method was similar to the method described above (Zomet et al., 2008), but with only four target–flanker distances (3λ, 4λ, 6λ, and 15λ) in the main experiment (Zomet et al., 2016). Stimuli were displayed on a Sony CPD-G400 Multiscan color monitor (1024 × 768 pixels; Sony, Tokyo, Japan). The effective size of the monitor was 26 × 35 cm, which at a viewing distance of 150 cm subtended a visual angle of 9.7° × 11.4°. 

White noise (containing a broad range of random orientations and spatial frequencies) at differing levels of contrast was presented at the target location and was superimposed on the target when present. For all observers, the noise contrast was normalized to their noise threshold detection threshold, measured, and separately estimated using an adaptive staircase method (79% correct); it was presented at 0, 0.5, 1, and 2 times their noise detection threshold (noise threshold units [NTU]). For a given noise level, the stimuli were presented in random order and all target–flanker separations were mixed (Mix procedure). Each block consisted of 20 trials at each of the four target–flanker separations (80 trials per block). There were four blocks, each with a different external noise level (0, 0.5, 1, and 2 NTU). The starting noise level was randomized between participants. Participants repeated each noise level three times for a total of 960 trials (20 × 4 flank distances × 4 noise levels × 3 repetitions). There was also a pilot experiment employing a set of six distances as in Polat and Sagi (2007). In this experiment, the target contrast was increased with increasing noise levels to yield (approximately) constant sensitivity. 

Signal detection theory analysis

We used the standard definitions of the sensitivity (d′) and the internal criterion (c) (Green & Swets, 1966):  

\begin{eqnarray}d^{\prime} = z\left( {{P_{Hit}}} \right) - z\left( {{P_{FA}}} \right) \quad \end{eqnarray}

(1)

 

\begin{eqnarray}c = - 0.5 \cdot \left[ {z\left( {{P_{Hit}}} \right) + z\left( {{P_{FA}}} \right)} \right] \quad \end{eqnarray}

(2)

where P_Hit is the probability that an observer correctly reported that the target is present in target-present trials, P_FA is the probability that the observer incorrectly reported that the target is present in target-absent trials, and z is the inverse cumulative normal distribution function. To avoid saturation, the P_Hit and P_FA probabilities were clipped to the range \([ {\frac{1}{{2n}},\frac{{2n - 1}}{{2n}}} ]\), where n is the number of trials in the measurement. 

RT behavioral analysis

We binned trials by categorizing RT into four equal-quantity bins from the fastest to the slowest. Binning was done separately for each experimental block and each trial type to avoid confounds (observer × experimental block × target–flanker distance × target stimulus [present/absent]). Task performance was quantified in each RT bin using the standard measures of sensitivity (d′) (Equation 1) and the decision criterion (c) (Equation 2) from SDT (Green & Swets, 1966). 

Statistics

All statistics were assessed using linear mixed-effects models (MATLAB 2013a fitlme(); MathWorks, Natick, MA), as detailed in the text. We considered two main factors to contribute to criterion modulation: the target–flanker distance (D_tf) and RT. We tested for significant contributions of these two factors and their interactions with the measured criterion (c) (Equation 2), assuming:  

\begin{eqnarray} && c = {\alpha _0} + {\alpha _1}*RT_{{\rm{bin}}} + {\alpha _2}*{D_{{\rm{tf}}}} + {\alpha _3}*R{T_{{\rm{bin}}}}*{D_{{\rm{tf}}}} \quad \end{eqnarray}

(3)

with D_tf (six or four levels; see Table 1, Target–flanker distance) and RT_bin (four bins, 0:3, fast to slow) defined as a continuous effect, and the observers as random effects (slopes and intercepts). Our main interest is in α₂, which measures the change in c resulting from increasing D_tf, and α₃, which measures the RT-dependent addition to α₁, so that the D_tf slope equals α₂ + α₃ × RT_bin. This simple model accounted for much of the variance in the data; the adjusted R² value was between 0.5 and 0.8; the lower values were obtained in the external-noise experiments and in the experiments with patients. 

Results

First, we confirm that, for the dataset analyzed here, the criterion dependency on distance differs between the two experimental conditions Fix and Mix. The original results are presented in Figure 2B. In the Fix condition, observers better adjusted their criterion, as manifested by the reduced dependency on the target–flanker distance (a smaller c[D_tf] slope), the consequence of the different decision requirements presumably imposed by the mixing of different target–flanker distances in the Mix condition. Indeed, testing the dataset analyzed here with the linear mixed-effects model for the effect of the experimental condition (Mix: E = 1; Fix: E = 0), assuming  

\begin{eqnarray*}c = {\alpha _0} + {\alpha _1}*{E} + {\alpha _2}*{D_{{\rm{tf}}}} + {\alpha _3}*{E}*{D_{{\rm{tf}}}}\end{eqnarray*}

showed a significant effect of the condition on the c(D_tf) slope, where α₃ = 0.035, t(260) = 2.97, p = 0.003; the slope more than doubled in the Mix condition (α₂ + α₃ = 0.061 vs. α₂ = 0.026). The c(D_tf) slope in the Fix condition (α₂ = 0.026) did not reach statistical significance (p = 0.09). The intercept showed no statistical difference between conditions (α₁ = −0.19, p = .09). 

P1. Effects of RT on the criterion slope

To test our first prediction (P1), we examined the dependence of the c(D_tf) slope on RT. First, we considered the experimental data of the Mix condition from Polat and Sagi (2007). In this experiment, trials having different target–flanker distances were mixed within a block (Figure 2A). Our RT analysis, presented in Figure 4A, clearly shows that the criterion (c) had lower values for slower RTs, especially at larger target–flanker distances (D_tf), implying that the criterion slope as a function of distance is reduced with increasing RT. This claim is strongly supported by the linear mixed-effects model described in Equation 1, showing that the interaction between the RT bin index and distance (α₃ = −0.02) was significant, t(140) = −3.67, p < 0.001. The c(D_tf) slope of the fastest RT bin (α₂ = 0.09) was significant, t(140) = 4.68, p < 0.001; whereas, the slope at the slowest RT bin (α₂ + 3 × α₃ = 0.03) approached statistical significance, t(140) = 1.97, p = 0.05, suggesting criterion modulation at slow RTs. 

Figure 4.

 

Target–flanker distance and RT, analysis of Polat and Sagi (2007). (A, C) Shown are (A) criterion (c, Equation 2) and (C) sensitivity (d′, Equation 1), as a function of the target–flanker distance (in units of wavelength, λ), when the different target–flanker distances are mixed within blocks (Mix experiment). Data were split into four bins (colors) by sorting the measured RTs. The results indicated: (1) a more biased criterion setting at the faster RTs, (2) a weaker modulation of the criterion by the flankers at the slower RTs, and (3) a flanker-dependent criterion in all RTs, even the slowest. The sensitivity (d′) was lower with slower RTs, but only in the slowest time bin (blue). (B, D) Behavioral data of the Fix experiment, whereby trials were blocked by the target–flanker distance, showing reduced effects. Error bars are ±1 SEM.

We note here that the criterion modulation differs from the sensitivity modulation. Polat and Sagi (2007) found a small but significant modulation of sensitivity (d′) by the flankers, as seen in Figure 2C for the available data. (Note that a much stronger modulation of sensitivity by the flankers is found when the detection task is replaced by a 2AFC discrimination task; see Polat & Sagi, 1993; Polat & Sagi, 2007.) The expected sensitivity modulation with distance is non-monotonic, showing a maximal effect at 3λ. Applying the RT analysis here revealed a gradual reduction of d′ at slower RTs, with values decreasing from ∼1.5 at fast RTs to ∼1 at slow RTs, t(142) = −4.01, p < 0.001, for modulation of d′ by RT when ignoring the target–flanker distance (Figure 4C). There was no significant interaction between RT and the target–flanker distance, t(140) = −0.72, p = 0.5. Importantly, the criterion and sensitivity exhibited different modulations by RT, as the criterion was modulated to a much greater extent than sensitivity, and its modulation dynamics differed (Figure 4A vs. Figure 4C). 

The RT effects in the Fix condition are qualitatively similar to the effects observed in the Mix condition (Figure 4A vs. Figure 4B). Here, the linear mixed effects model, described in Equation 1, showed that the interaction between the RT bin index and distance (α₃ = −0.01) was significant, t(116) = −2.86, p = 0.005. The c(D_tf) slope of the fastest RT bin (α₂ = 0.04) was significant, t(116) = 3.07, p = 0.003), whereas the slope with the slowest RT bin (α₁ + 3 × α₃ = 0.01) was not statistically different from zero, t(116) = 0.64, p = 0.52, suggesting, as predicted, no criterion modulation at slow RTs when the target–flanker distances are blocked. 

P2. effects of added external noise on the interaction between RT and c(D_tf)

The second prediction was tested using results from experiments in which different levels of external noise are added to the target (see Methods and Zomet et al., 2016) using a procedure that is otherwise identical to the one considered in the previous section. Different noise levels were added in different blocks, randomly permuted between observers (see Methods and Zomet et al., 2016). We considered two variants: one where the contrast was maintained and the noise was increased, leading to a reduced d′ value when the noise level increased (Main, n = 12 observers) (Figure 5B). The other, an unpublished pilot of that study, in which the target contrast was increased with noise to maintain a roughly fixed d′ (Pilot, n = 7) (Figure 5C); this pilot is particularly convenient for performing the RT analyses because of the large number of trials per observer) (Table 1). 

First, we considered the case when the level of external noise was zero (No noise) (Figures 5B and 5C; Figures A3 and A4). In this condition, the experiment is equivalent to the one considered in the previous section (Mix condition), so we expected similar results. The results indeed show a strong interaction between RT and the c(D_tf) slope. Specifically, the c(D_tf) slope was reduced with RT; for Pilot, t(164) = −4.66, p < 0.001, and for Main, t(188) = −3.01, p = 0.003. In addition, the criterion slope at the slowest RT was positive; for Pilot, t(164) = 4.62, p < 0.001, and for Main, t(188) = −3.88, p < 0.001, as predicted for the Mix condition (the estimated slopes are presented in Figure 8). 

To test prediction P2, we examined the interaction between RT and the c(D_tf) slope in the presence of noise. As shown in Figure 5 and Figure 8, introducing noise reduces the interaction of the criterion and the RT. This is seen in Figure 8, manifested by more similar c(D_tf) slopes for the fast and slow RT bins in the presence of noise. There is one exception, when the noise level was 75% of the noise threshold in the pilot experiment, t(164) = −3.39, p < 0.001; otherwise, as predicted, none of the interactions reached statistical significance (p = 0.1–0.9). This effect was not due to a reduction in sensitivity (d′), as shown in Figure 5C, where the target contrast was increased with the added noise, so that the sensitivity was fixed. 

Figure 5.

 

External noise and the RT effect, analysis of Zomet et al. (2016). (A) An example of stimuli with different added noise levels. (B) The main experiment where added noise reduced sensitivity (d′). (C) The pilot experiment where the target contrast increased with noise, so that the sensitivity remained stable across noise levels. The error bars are ±1 SEM.

P3, effects of added external noise on the c(D_tf) slope

Of interest here are the results with the highest noise level. Here we expected the Main experiment and the Pilot to diverge. Indeed, examining the c(D_tf) slopes at the slowest RT show that it is significant in the Pilot experiment, in which the target contrast was increased with noise, t(164) = −3.01, p = 0.003, but not in the Main experiments in which the target contrast was not scaled with noise, t(172) = 0.51, p = 0.5. 

P4. individuals with slow RTs showed no RT effect for criterion

To test the relationship between individuals’ bias and RT, we applied the RT analysis to the experimental data of Zomet et al. (2008). In this study, patients diagnosed with depression (Patients) and their matched controls (Controls) performed an experiment similar to the ones analyzed above. Specifically, different target–flanker distances were mixed within a block, and there was no noise, comparable to the Mix experiment of Polat and Sagi (2007) and the no-noise condition of Zomet et al. (2016). Unlike these studies, the number of trials was lower, and observers were, on average, older and slower (see Table 1). 

The results of the RT analysis are presented in Figure 6. The criterion slope as a function of the target–flanker distance was reduced with RT, though not significantly for the patients. For RT:slope interaction, Controls: t(764) = −2.41, p = 0.02; Patients: t(644) = −0.52, p = 0.6. The sensitivity (d′) was reduced with RT—Controls: t(764) = −2.22, p = 0.03; Patients: t(644) = −3.06, p = 0.002—independent of the target–flanker distance—Controls: t(764) = 0.19, p = 0.85; Patients: t(644) = −1.46, p = 0.14. 

Figure 6.

 

The RT effect in slow individuals. (A) RT analysis of Zomet et al. (2008). Annotations follow Figure 4. The results showed no RT effect for criterion in depressive patients and controls. (B) Shown for each observer is the change in criterion (Δc) from the fastest to the slowest RT bin of the 15λ measurement, as a function of the mean RT. The results indicated a correlation between RT and Δc between experiments and within experiments for the Zomet et al. (2008) control data. The procedural differences between experiments are summarized in Table 1.

To explain the small effect of RT on the criterion, we considered the simple idea described in Dekel and Sagi (2020a)—namely, that individuals with reduced bias can be explained by having slower decision times. Indeed, we found that observers in Zomet et al. (2008) had extremely slow RTs, showing, on average, ∼1600 ms for Controls and ∼2700 ms for Patients (see the x-axis of Figure 6B). This is much slower than the ∼650 ms measured for observers in similar experimental conditions (Figures 4A and 5, no noise). Overall, the differences between experiments can possibly be explained by RT. In addition, we considered differences between observers in the control group of Zomet et al. (2008), where wide inter-individual differences in RT were measured (see the x-axis of Figure 6B). We correlated individual RTs with the individual size of the criterion modulation by RT (for 15λ distance); we found a significant correlation (adjusted R = 0.54, p < 0.001) (Figure 6B). To summarize, the difference between the experiment by Zomet et al. (2008) and the other experiments appears to be well explained by slower RTs. In addition, the difference in RT between experiments and observers can possibly be attributed to age and/or practice effects (Table 1). 

Summary of the results

The experimental results show a clear dependency of decision bias on the target–flanker distance, which is expected to disappear at slower response times if it was caused only by the criterion setting (sp in DDM), but not if it was caused by changes in the internal distributions associated with the different stimuli (affecting the drift rate in DDM). Our results, summarized in Figures 7 and 8, support the latter. 

Figure 7.

 

The differences between experiments disappear with slow RTs. Shown for all experiments without noise are the criterion measurements as a function of the target–flanker distance in four RT bins. It can be observed that the experimental differences are abolished in the slowest RT bin (bin 4).

Figure 8.

 

Fast (Q1, first RT quantile) and slow (Q4, fourth RT quantile) distance effects on criteria: a summary of all studies. Shown are the c(D_tf) slopes fitted to the respective RT bins using the linear model described by Equation 3. All fast distance effects except one (the highest noise level in Zomet et al., 2016; main) are statistically significant (see the Results). The slow effects are more uniform across conditions, all significant except for the Fix condition (see P1) and the noise-amp = 0.75 condition in the pilot experiment (see the Discussion), and the noise-amp = 2 condition in the main experiment (see P3).

Figure 7 depicts c(D_tf) functions for the four RT bins (1 being the fastest). In the slowest RT bin the differences between experiments and conditions are largely abolished (but note the flat Fix curve), as clearly seen when comparing all curves in bin 4 (slow RTs). In the fastest RTs (bin 1; see Figure 7), the curves corresponding to the different conditions present a much larger variability. 

DDM predictions

To try to better understand the observed interactions with RT, we considered the idea that the detection task is performed by the gradual accumulation of noisy evidence, as in the standard DDM (Gold & Shadlen, 2001; Ratcliff et al., 2016). Here we considered some general effects of model parameters on its behavior (a more detailed model parametrization and fitting results are provided in Figure A2). As described in the Introduction, two important parameters in the model affect the criterion: (1) the starting point (sp) of the accumulation process, which leads to increased bias in faster decisions if put closer to one of the bounds (Figure 9, second row) (Dekel & Sagi, 2020b); and (2) the rates at which evidence is accumulated for target-present and target-absent trials (Figure 9, first row). As shown in Figure 9, this simple idea can explain the criterion measurements quite well. We provide here some information as to how this model works by describing the predictions of four simple model variations and analyzing the effect of the drift rate and the starting point on the results. The analysis is sufficiently general for its predictions to be independent of the specific assumptions made regarding the drift rate (see the Introduction). For simplicity, the drift rate is considered here as an abstract variable monotonically related to the log-likelihood ratio (Gold & Shadlen, 2001). 

Figure 9.

 

Modeling the influence of flankers on target detection using DDM. Shown are the parameters (the first two rows) and the theoretical predictions (the last four rows) of the four models described in the text. The sensitivity estimate (d′, Equation 1) is approximately the difference in the drift rate between the target present and absent trials. In Models 3 and 4, the starting point is biased (sp < 0.5), leading to an overall reduction in criterion (c, Equation 2) with RT. In Model 4, the reduction in slope with RT is explained by a flanker-dependent starting point (variable sp). Overall, Model 4 can explain both the reduction in overall criterion with RT and the reduction in the c (distance) slope with RT. See the text for model details and Figure A2 for fitting Model 4 to the experimental data.

Model 1: An unbiased flanker-independent starting point, target-dependent drift rate

Here, the drift rate (v) depends on the target stimulus (v₊ = +1 for target present, v_– = −1 for target absent), and the starting point (sp) is fixed at 0.5 (unbiased). This model predicts no bias (c = 0) at all RTs, with fixed d′, independent of distance. This model fails to explain the data. 

Model 2: As in model 1 with a flanker-dependent drift rate

Here, at short distances, the difference between v₊ and v_– is increased, thus showing larger d′ values at these distances. Both v₊ and v_– are positive at short distances (supporting the positive responses), thus showing a negative criterion (c < 0) at these distances, RT independent. This model can explain the distance dependence, but not the RT dependence of the measured criterion. 

Model 3: A biased flanker-independent starting point, flanker-dependent drift rate

The same as in Model 2, but with a biased starting point (sp). The effect of this bias (sp < 0.5) is short term, resulting in higher criteria (c) in trials with fast RTs; slow RTs show criteria equal to those produced by Model 2. The RT effect on c is distance independent. This model predicts an RT effect but cannot explain the dependence of the criterion slope on RT (Figure 8). 

Model 4: A flanker-dependent starting point and drift rate

Like Model 3 but with sp allowed to change with distance, resulting in distance-dependent RT effects, the RT-dependent slope of the c(distance) curve. This model captures the main features of the experimental results. Overall, the DDM framework considered here can qualitatively account for the most salient RT effects, but a precise modeling, requiring several more parameters and assumptions, remains somewhat speculative. In the Appendix (Figure A2) we present a detailed fit of Model 4 to the behavioral data, using the fast-dm software with the Kolmogorov–Smirnov (KS) setting (Voss & Voss, 2007). 

DDM drift rate and SDT d′

Note that the DDM analysis allows for the derivation of the target–flanker distance-dependent sensitivity functions for both the target-present and target-absent conditions (the idea in Figure 9, fitting in Figure A2), unlike the previously used SDT methods that provided only a differential sensitivity estimate (i.e., d′). As seen in Figure A2, the fitted drift rates for both target-present and target-absent gradually decrease with longer target–flanker distances, as predicted by the theory illustrated in Figure 1. The differential sensitivity, which is the difference in the fitted drift rates between the target-present and target-absent trials, was mostly fixed, showing a small decrease at longer target–flanker distances. This is consistent with the slightly improved d′ value in the presence of proximal flankers. We noted that the fitted drift rate in the target-absent trials was usually negative. A negative drift indicates that the evidence supports the negative (target-absence) response. That is, the reduced sensory response in the target-absent trials is mapped into negative values (see the Discussion). This finding is reasonable in the sense that what is accumulated is target-presence evidence, with “zero” values corresponding to no evidence for or against target presence (see the Introduction). 

Discussion

Lateral masking affects the detection of low-contrast targets (Polat & Sagi, 1993). This effect is usually quantified using measures of behavior derived from SDT (Green & Swets, 1966): sensitivity (d′, Equation 1) and the decision criterion (c, Equation 2). Although the bias-independent sensitivity measure (d′) is the standard measure used to quantify the perceptual effects, our interest here lies in the effects on the decision criterion (bias), shown in Figure 2. SDT, with limited access to internal distributions (Gorea & Sagi, 2000), provides an observer-independent account of the biases found, as illustrated in Figure 3 (see the Introduction). Here, for the first time, to the best of our knowledge, an RT analysis of the lateral masking effect was performed, attempting to isolate the subjective (observer dependent) and objective (observer independent) factors underlying the observed biases in perception. 

To explain the effects of RT, we extended the time-independent SDT explanation (Figure 1) by using the drift diffusion model. More specifically, we modeled the influence of the flankers on the detected target (Figure 1C, signal shift) as a change in the rate of evidence accumulation (the drift rate), which leads to the time-independent effects on the measured criterion, which are maintained even at the slowest RTs. The time-dependent effects on criterion were modeled as a change in the starting point (Figure 1B, criterion shift). 

We found that mixing the trials of different target–flanker distances affects mostly the slow RT trials. The fixed-distance procedure, unlike the mixed-distance procedure, produces negligible distance dependence at the slow RTs but not at the fast RTs (Results, P1). This result can be explained by a mismatch between the actual internal distributions associated with the stimuli and the one (averaged across distances) used by the observer to make a decision concerning the target's presence. 

We found that adding external noise (Zomet et al., 2016) affects mainly the fast RTs (Results, P2), equating fast and slow effects, implying that the influence of the starting point becomes negligible due to the presence of external noise when accumulation starts. For the condition where the target contrast was not increased to compensate for the increased noise, we found reduced effects on the criterion at both fast and slow RTs, as predicted by the change in the slope of the LLR function (P3, Figure 3). 

In Zomet et al. (2008), the observers are slow (because of age or lack of practice); thus, we found the criterion effects to be much smaller, showing little or no dependence on RT (Results, P4). This agrees with our previous results showing that individual differences in TAE can be explained by RT differences (Dekel & Sagi, 2020a). 

The presented RT analysis provides further insight into processes underlying statistical decisions. Of particular interest are the limitations imposed on such decisions. We identified here a limit on the number of statistical distributions that can be efficiently learned by observers facing decisions on stimuli sampled from different distributions. Considering the example presented in Figure 3, limitations regarding the internal representations of response distributions are expected to be universally expressed (i.e., to introduce biases in slow responses). Indeed, slow distance-dependent biases were shown in 11 experimental conditions out of 14 (see Figure 8). As discussed above, we predicted two of the three exceptions. Of critical importance for the present theoretical framework is the reduced effect in the Fix condition of Polat and Sagi (2007). Here, unlike in the Mix condition, the different target–flanker distances are blocked, leaving us with only two internal distributions to learn (signal and noise); thus, the imposed constraints do not apply. A second predicted exception, described above (P3), concerns the absence of slow RT effects in the presence of high external noise. The third exception, not explained within the theoretical framework presented here, is the experimental condition with the external noise amplitude approaching the noise threshold (noise amplitude = 0.75 threshold). We attribute this reduced distance effect to the increase in the False Alarm rate at slow RTs (otherwise present only at target locations near the flanker) and to the reduced sensitivity at all distances (d′ approaching zero), caused by the external noise level approaching the threshold, as indicated by the results presented in Figure A4. 

Within the context of the DDM framework, the slow RT effect can be quantified as the drift rate asymmetry (Models 2 to 4, Figure 9 and Figure A2), which is the sum of the up (target present) and down (target absent) drift rates (v₊ + v_–). For an unbiased observer, this sum is expected to be zero. Figure 10A presents the drift rate asymmetry in the fitting results (Figure A2) for both the Mix and the Fix conditions of Polat and Sagi (2007), showing a marked difference between conditions. In contradistinction, biases due to shifts in the starting point (sp) are very similar in the Mix and the Fix conditions (Figure 10B). Accordingly, we attribute the differences in results between the Mix and the Fix conditions to objective factors affecting decisions—that is, to differences in the internal responses rather than to subjective factors such as priors (see the Introduction). We can conclude that the biases in the Mix condition can be explained by the distance-dependent excitatory effects of the flankers, operating in both target-present and target-absent trials, underlying the perceived filling-in effect. 

Figure 10.

 

(A) Drift rate asymmetry, quantified as the sum of the up and down drifts, evaluated by fitting Model 4 (Figure A2) to the data of Polat and Sagi (2007). A positive bias leads to increased Hit and FA rates and, thus, to lower decision criteria. Note the difference between the two conditions. (B) Drift starting point (sp), evaluated by fitting Model 4 to the data of Polat and Sagi (2007). Biases due to prior information or payoff are indicated by deviations from sp = 0.5. Both experimental conditions show unbiased sp at short distances but a lower sp at longer distances, resulting in lower Hit and FA rates at these distances (higher criteria).

Although additional modeling details remain somewhat speculative, the basic idea used here, of separately considering RT-dependent and RT-independent effects and interpreting them as changes in how evidence is interpreted (the drift rate) and what priors exist before the evidence (the starting point), seems quite robust. 

Acknowledgments

Supported by a grant from the Basic Research Foundation, administered by the Israel Academy of Science (2494/21 to UP and DS). 

Commercial relationships: none. 

Corresponding author: Dov Sagi. 

Email: [e-mail]. 

Address: Department of Brain Sciences, The Weizmann Institute of Science, Rehovot, Israel. 

Appendix

Figure A1.

 

Target–flanker distance and RT. Shown, in six experiments (columns), are five behavioral measures (rows): criterion (c), sensitivity (d′), hit rate, False Alarm rate, and RT, as a function of the target–flanker distance. Data were split into four bins (colors) by sorting the measured RTs. Error bars are ±1 SEM.

Figure A2.

 

Fitting behavior to Model 4. The top two rows present the recovered parameters: the drift rates (v₊, v_–, and their mean) in the first row and the starting points in the second row. Rows 3 to 6 present the recovered behavior (c, d′, P_Hit, and P_FA). To obtain analytical model distributions and to fit Model 4 to behavioral data, we used the fast-dm software with the Kolmogorov–Smirnov (KS) setting (Voss & Voss, 2007). Fitting was performed separately for each observer. The drift rate (v) was fitted separately for each combination of the target stimulus (target present/absent) and the target–flanker distance (four or six possible distances, resulting in 8 or 12 parameters); the starting point (sp) was fitted separately for each target–flanker distance (four or six possible distances, resulting in 4 or 6 parameters),. The non-decision time (t₀) and bounds-separation (a) were fitted once for all configurations (2 parameters). Overall, the model recovers the important features of the experimental results shown in Figure A1 (R² = 0.78, comparing all 88 RT binned criteria, third row, broken into experiments; from left to right, R² = 0.87, R² = 0.6, R² = 0.93, R² = 0.79).

Figure A3.

 

RT and external noise. RT analysis of the main experiment of Zomet et al. (2016), with the layout following Figure A1. The first row is reproduced from Figure 5B.

Figure A4.

 

RT and external noise. RT analysis of the pilot experiment of Zomet et al. (2016). The first row is reproduced from Figure 5C.