Although confidence is commonly believed to be an essential element in decision-making, it remains unclear what gives rise to one's sense of confidence. Recent Bayesian theories propose that confidence is computed, in part, from the degree of uncertainty in sensory evidence. Alternatively, observers can use physical properties of the stimulus as a heuristic to confidence. In the current study, we developed ideal observer models for either hypothesis and compared their predictions against human data obtained from psychophysical experiments. Participants reported the orientation of a stimulus, and their confidence in this estimate, under varying levels of internal and external noise. As predicted by the Bayesian model, we found a consistent link between confidence and behavioral variability for a given stimulus orientation. Confidence was higher when orientation estimates were more precise, for both internal and external sources of noise. However, we observed the inverse pattern when comparing between stimulus orientations: although observers gave more precise orientation estimates for cardinal orientations (a phenomenon known as the oblique effect), they were more confident about oblique orientations. We show that these results are well explained by a strategy to confidence that is based on the perceived amount of noise in the stimulus. Altogether, our results suggest that confidence is not always computed from the degree of uncertainty in one's perceptual evidence but can instead be based on visual cues that function as simple Heuristics to confidence.

*higher*for oblique orientations. Rather than being consistent with Bayesian decision theory, we show that these results are better explained by the participant's ability to perceive the noise in the stimulus—a heuristic to confidence.

_{tj}of the

*j*th Gabor element on the

*t*th trial was determined as follows:

*t*from one of four possibilities (22.5°, 67.5°, 112.5°, or 157.5°), θ

_{t}is the mean stimulus orientation on trial

*t*, \({\sigma _{{e_t}}}\) is the external noise level on this trial (standard deviation of Gaussian noise, chosen from 0.5°, 2°, 4°, 8°, or 16°) and σ

_{0}= 7° models a small jitter of the base orientation across trials. In words, we first created 20 conditions, one for each combination of base orientation (22.5°, 67.5°, 112.5°, 157.5°) and noise level (standard deviation of Gaussian noise: 0.5°, 2°, 4°, 8°, 16°). For each stimulus condition, we then created 105 orientation trials by adding small orientation offsets to the condition's base orientation (the offsets are randomly sampled from a Normal distribution, \({\cal N}( {0,\;\sigma _0^2} )\)) to reduce predictability. We then generated, for each trial, 36 patches using a Gaussian distribution centered on that trial's stimulus orientation, and with noise level determined by the stimulus condition. Each of these Gabor patches was subsequently positioned on a grid defined by three concentric rings (of radius 2°, 4°, and 6° v.a.) around the fixation target (the grid was not visible to participants). Individual elements were first positioned evenly within each ring in the grid, and then random jitter (uniformly distributed; min = −0.75°, max = 0.75°) was added to both polar coordinates of the grating's location, such that spatial predictability was reduced.

*SD*). The two stimuli always had the same mean orientation, and their presentation order was randomized across trials. Patch location was determined independently for the two stimuli, and using identical procedures as in Experiments 1 and 2. After two experimental blocks of practice, participants completed 36 blocks of 50 trials each, over three days. On each day, all 12 orientations were presented in random order; however, within one block only one mean orientation was shown.

*SD*

_{beh}) and mean level of confidence across all trials in each of the bins was computed in two steps: first, for each (base) orientation separately, and then averaged over orientations. To test for significance, a linear regression was used:

_{0}(sbj) is a subject-specific intercept term, σ

_{e}(bin) is the amount of external stimulus noise in a given bin, β

_{1}is the associated regression coefficient, and ∈ (sbj, bin) is the residual error for a given subject × bin (resulting in a total of 11 independent variables for each model: 10 subject-specific intercepts and 1 slope coefficient on the external noise). β, β

_{0}(sbj), and ∈ were estimated separately for confidence and behavioral variability.

_{0}(sbj) is a subject-specific intercept,

*d*is the mean stimulus orientation expressed as distance to the nearest cardinal (i.e., horizontal or vertical) for each subject on each trial,

*v*is the variance across the 36 patches, β

_{1}and β

_{2}are the corresponding regression coefficients, and ∈ (sbj, trial) is the residual error. The residuals of this analysis, that is, confidence after removing the effects of mean stimulus orientation and local changes in variance across patches (confidence

_{corrected}= ∈(sbj, trial)), were then used to sort trials into three bins of increasing confidence for each observer and each of the five external noise levels independently. Behavioral variability was computed for each of these bins, resulting in three values of behavioral variability (one for each level of confidence) for each level of external noise. To test for significance, a two-way analysis of variance was conducted on behavioral variability values, with external noise and confidence as factors. This was followed by a sequential comparison between the low and medium and medium and high confidence levels, using an analysis of variance with the same two factors. To test whether confidence predicted behavioral variability when external noise was low, behavioral variability was linearly regressed on confidence, using data for the lowest external noise level only:

_{0}(sbj) is a subject-specific intercept term, confidence

_{corrected}(sbj,bin) is the average confidence within each confidence bin (corrected as explained above), β

_{1}is the associated regression coefficient, and ∈ (sbj, bin) is the residual error for a given subject × bin (resulting in a total of 11 independent variables for each model: 10 subject-specific intercepts and one slope coefficient for confidence).

*d*(bin) is the orientation of the stimulus within a bin, defined in terms of its distance to the nearest cardinal orientation (i.e., horizontal or vertical). The regression models also included an intercept for each individual observer, β

_{0}(sbj), resulting in a total of 9 independent variables in each model.

*n*= 36 stimulus elements (two-alternative forced choice). The elements of the first stimulus (standard) were drawn from a Gaussian distribution with variance \(\sigma _s^2\), whereas those of the second (comparison) stimulus were drawn from a Gaussian distribution with variance (σ

_{s}+ Δσ)

^{2}. Because the elements were drawn from a Gaussian distribution, the variance across the

*n*stimulus elements followed a scaled χ

^{2}distribution across trials, with

*n*-1 degrees of freedom. Adding internal noise \(\sigma _i^2\), the probability of reporting that perceived variance of the standard \(s_1^2\) is greater than that of the comparison \(s_2^2\) becomes:

*F*is the cumulative distribution function of the

*F*-distribution, with degrees of freedom

*n*-1 and

*n*-1. See (Morgan et al., 2008) for more detailed derivations. We estimated internal noise value \(\widehat {\sigma _i^2}\) for each participant and base orientation independently, using maximum-likelihood estimation and the data of Experiment 3.

*p*(

*m*|θ). We assumed that the measurements are drawn from a Normal distribution centered on θ, with variance determined by both internal and external sources of noise,

*p*(

*m*|θ) = \( {\cal N}( {\theta ,\sigma _{{\rm{int}}}^2 + \sigma _{{\rm{ext}}}^2} )\). For the compound stimulus considered here, both sources of noise are directly related to the across-trial variance in individual stimulus elements (see

*Simulations*). External noise refers to noise in the environment, and we considered two sources of noise internal to the observer. The first source of internal noise was independent of stimulus orientation, and could be high, medium or low. The second source varied across orientation stimuli, with smaller levels of noise for cardinal (horizontal and vertical) than oblique orientations. This pattern of internal noise models an oblique effect in orientation perception (Appelle, 1972; Girshick et al., 2011; Tomassini et al., 2010).

*p*(θ|

*m*) that could have led to the current measurement. Assuming a flat stimulus prior, this becomes

*m*} from this compound stimulus. When the observer takes the sensory measurements, internal noise adds further variability. Thus the sensory measurement

_{j}*m*, of the orientation of the

_{j}*j*th stimulus patch, follows a Normal distribution:

*m*}, the observer computes an estimate of the amount of external noise in the stimulus. The observer does this by maximum-likelihood estimation, finding the value of σ

_{j}_{e}that maximizes the probability of the observed data (the measurements {

*m*}):

_{j}_{e}must be strictly non-negative (or in other words, that the sample variance cannot be exceeded), the following equation to compute \(\widehat {{\sigma _e}}\) can be derived analytically:

*k*is the number of measurements taken by the observer, which may be fewer than the total number of stimulus elements

*n*. In words, the observer computes the sample variance of the orientation measurements {

*m*}, and subtracts the variance that can be explained by internal noise, to obtain a perceptual estimate of external noise variance. If the entire sample variance can be accounted for by internal noise, then the perceived external noise variance is 0.

_{j}*b*is a parameter that determines the observer's maximum confidence (when \(\widehat {{\sigma _e}} = 0\)). Thus the Heuristics observer's confidence is inversely proportional to the estimated external noise variance: the more noise the observer perceives in the stimulus, the less confident the observer will be. Note that this model is not normative, so we simply defined confidence as a function of the amount of perceived noise in the stimulus. Although the estimated amount of stimulus noise tends to be more accurate when the observer takes more measurements from the stimulus, this perceived noisiness (and, hence, confidence) will not decrease (increase) with larger

*k*. However, had we nonetheless incorporated the

*k*parameter into Equation 16 (thereby creating higher levels of confidence when the observer samples more orientation patches), this would have merely scaled the predictions, and their overall pattern would have remained the same.

*m*arose because of sources both internal and external to the observer (see also above):

*k*individual stimulus elements to calculate the most likely mean stimulus value, so that behavioral variability due to external noise alone is given by \(\sigma _{{\rm{ext}}}^2 = \;\sigma _e^2/k\). In line with the actual experimental parameters (see

*Experimental Design*), we set σ

_{e}to 120 linearly spaced values from 0.5° to 16° for our simulations of Experiment 1 (Figures 2a–b and 2e–f), and to 0.5° for Experiment 2 (Figures 4a–b). Similarly, we assumed that \(\sigma _{{\rm{int}}}^2 = \;\sigma _i^2/k\), with internal noise σ

_{i}set to 6°, or either low, medium or high (i.e., 4°, 6°, and 8°) for our simulations of Experiment 1 (Figures 2a–b, and 2e–f, respectively). We used \(k = \sqrt n = 6\) in our simulations, as human observers typically pool across approximately the square root of the

*n*stimulus elements for their decisions (Dakin, 2001; Moerel, Ling, & Jehee, 2016; Whitney & Yamanashi Leib, 2018). However, using a different value of

*k*does not qualitatively change any of our predictions (Supplementary Figure S3). For the simulations of Experiment 2 (Figures 4a–b), internal noise was dependent on stimulus orientation, with smaller levels of noise for cardinal than oblique orientations:

_{i}was set to 120 linearly spaced values between 0.5° and 16°, and σ

_{e}was 6°. For all simulations, we added a constant baseline of \(\sigma _{{\rm{baseline}}}^2 = {4^2}\) (Equation 17 ); note that this parameter does not affect the overall pattern of effects in the simulated data. For both model observers, the reported stimulus orientation (i.e. their behavioral estimate) was obtained using Equation 11 and taking the (circular) mean of the distribution, as outlined above. Confidence estimates were obtained using Equations 12 and 16, for the Bayesian and Heuristics model observer, respectively. For each model observer, confidence values were normalized across experiments to lie in between 0 (minimum confidence) and 1 (maximum confidence).

*Data Analysis*). For each of the 12 measured orientations and for each individual observer, we then predicted their level of confidence using these estimates and Equations 12 and 16. We subsequently averaged across observers to arrive at Figure 5b.

*m*changes from one trial to the next, and the relationship between the stimulus orientation and its measurements is described by a probability distribution,

*p*(

*m*| θ). We assume that the measurements are drawn from a Gaussian distribution centered on θ, with variance determined by a combination of internal and external sources of noise, \(p( m|\theta ) = {\cal N}( {\theta ,\sigma _{{\rm{int}}}^2 + \sigma _{{\rm{ext}}}^2} )\). External noise refers to noise that occurs in the environment; for instance, because physical stimulus elements are variable. We model two sources of internal noise. The first source of internal noise is independent of stimulus orientation, and could be high, medium or low. It captures trial-by-trial fluctuations in, for example, the observer's attentional state or fatigue. The second source varies with stimulus orientation, with smaller levels of noise for cardinal (horizontal and vertical) than oblique orientations. This pattern of internal noise models the well-known oblique effect in orientation perception (Appelle, 1972; Girshick et al., 2011; Tomassini et al., 2010).

*p*(θ |

*m*), which describes the range of orientations that is consistent with the current measurement

*m*. The mean of the posterior distribution serves as the observer's orientation estimate, while its width (variance,\(\;\sigma _{\rm{p}}^2\) ) can be taken as a metric on the degree of sensory uncertainty in the estimate. The Bayesian (normative) hypothesis holds that confidence is based on the degree of sensory uncertainty. Thus, assuming that the brain computes sensory uncertainty, the question is whether observers are aware of this uncertainty and use it in their confidence reports. The hypothesis predicts that when the observer's perceptual evidence is imprecise, the level of confidence should be low. In our simulations, we quantify this inverse relationship between confidence and uncertainty as follows (Equation 12):

*m*of stimulus element

_{j}*j*follow a Normal distribution (see Equation 13). The variance of the distribution is described by \(\sigma _i^2( \theta ),\) the internal noise variance in the measurements of individual orientation patches as a function of the overall stimulus orientation θ, and \(\sigma _e^2\), the variance in individual elements due to external sources of noise. Both sources of noise are directly related to the across-trial variance in the observer's orientation judgements (see

*Methods*). We assume that the observer has learned, from prior experience, how internal noise varies as a function of stimulus orientation; that is, the observer knows \(\sigma _i^2( \theta )\). The observer uses this information and the set of measurements {

*m*} to compute, for each trial, the mostly likely estimate of the amount of external noise in the stimulus \(\widehat {{\sigma _e}}\) (see Equation 14). More specifically, the observer computes the sample variance of the orientation measurements, and subtracts the variance that can be explained by internal noise, to obtain the most likely estimate of external noise variance for that trial's stimulus. If all sample variance can be accounted for by internal noise, then the perceived external noise variance is 0—in other words, the stimulus “looks” like it contained no external noise.

_{j}*b*is a parameter that determines the observer's maximum confidence (when \(\widehat {{\sigma _e}} = 0\)). Thus the Heuristics observer's confidence is inversely proportional to the estimated external noise variance: the more noise the observer perceives in the stimulus, the less confident the observer will be. See

*Methods*for more detailed definitions and derivations. See Supporting Materials for a third observer model, which computes confidence from the ability to

*detect*noise, rather than the perceived amount of noise, in the stimulus. Its predictions are qualitatively similar to the ones obtained from the perceived-noise model observer discussed here (Supplementary Figures S1 and S2).

*Methods*). As expected, the participants’ orientation estimates were strongly affected by the level of external noise in the stimulus. Specifically, higher levels of orientation noise in the stimulus led to significantly greater variability in reports of perceived orientation (

*r*= 0.91,

*t*(39 = 14.4,

*p*< 0.001) (Figure 3a). Corroborating the predictions of both models, confidence was also highly affected by changes in external noise (Figure 3b). Subjects consistently reported lower levels of confidence with increasing amounts of external noise in the stimulus (

*r*= −0.93,

*t*(39) = −17.1,

*p*< 0.001). Indeed, a direct comparison between confidence and behavioral performance revealed a significant inverse relationship between reported confidence and behavioral variability (

*r*= −0.85,

*t*(39) = −10.3,

*p*< 0.001). Altogether, this indicates that participants were able to meaningfully estimate their own level of confidence in the task.

*Methods,*Equation 6), so that the bins more closely reflected internal fluctuations in confidence. Behavioral variability and mean level of confidence across all trials in each bin were computed and compared. We considered three possible outcomes. First, our simulations indicated that if observers compute confidence based on the uncertainty in their sensory evidence, as suggested by the Bayesian model, then greater levels of confidence should be linked to more precise (less variable) behavior (Figures 2a, c, e). Second, if observers use a heuristic strategy to confidence based on the perceived amount of noise, then the relationship between confidence and behavioral variability should be inconsistent within external noise levels, as internal and external noise push this relationship in opposite directions (Figures 2b, d, f). Third, if observers use neither of these strategies, then we should observe no reliable link between confidence and behavioral variability at all. Corroborating the Bayesian observer model, we found a positive relationship between confidence and precision in behavior (Figure 3b). Specifically, even within levels of external noise, behavioral orientation estimates were reliably less variable when reported confidence was higher (main effect of confidence, (

*F*(2, 18) = 45.1,

*p*< 0.001). Additional analyses revealed that higher levels of confidence consistently predicted better behavioral performance when comparing sequentially between confidence bins (high versus medium:

*F*(1, 9) = 29.9,

*p*< 0.001; medium versus low:

*F*(1, 9) = 35.3,

*p*= < 0.001). We also found a positive relationship between confidence and behavioral precision for even the lowest level of external noise (i.e. σ

_{e}= 0.5), when trial-by-trial fluctuations in behavior are likely dominated by internal sources of variance (

*r*= 0.51,

*t*(19) = -2.6,

*p*= 0.016). Control analyses verified that these results do not strongly depend on the number of bins analyzed (main effect of confidence, 2 bins,

*F*(1, 9) = 42.2,

*p*< 0.001; 4 bins:

*F*(3, 27) = 24.4,

*p*< 0.001). Thus these results appear consistent with the Bayesian notion that confidence is computed from the degree of uncertainty in the observer's perceptual estimates.

*r*= 0.59,

*t*(87) = 6.91,

*p*< 0.001). Our simulations indicated that if confidence is computed from the precision in sensory evidence, then reported confidence should follow this oblique effect in orientation judgments, with lower levels of confidence for oblique compared to cardinal orientations. If, on the other hand, confidence is based on the perceived amount of stimulus noise, the opposite pattern should emerge and confidence should be higher for oblique than cardinal orientations. Interestingly, comparing confidence between stimulus orientations revealed that participants were more confident about their orientation estimates for oblique than cardinal stimuli (

*r*= 0.34,

*t*(87) = 3.38,

*p*< 0.001), despite lower levels of performance for these stimuli (Figure 4). These findings run contrary to the predictions of the Bayesian observer model and instead provides support for the Heuristics observer model, wherein confidence is based, in part, on external properties of the stimulus.

*Methods*), and found that internal noise was reliably higher for oblique compared to cardinal orientations (Figure 5a;

*r*= 0.48,

*t(*87) = 5.12,

*p*< 0.001). Altogether, this indicates that observers better perceive external stimulus noise around cardinal than oblique orientations—much like we assumed in our modeling work.

**Citation gender diversity statement**

*Neural Computation,*1–28, doi:10.1162/neco_a_01141.

*PLoS Computational Biology*, 11(10), e1004519, doi:10.1371/journal.pcbi.1004519. [CrossRef] [PubMed]

*Psychological Bulletin*, 78(4), 266–278, doi:10.1037/h0033117. [CrossRef] [PubMed]

*Proceedings of the National Academy of Sciences,*115(23), 6082–6087, doi:10.1073/pnas.1800795115. [CrossRef]

*Perception & Psychophysics*, 61(7), 1369–1383, doi:10.3758/bf03206187. [CrossRef] [PubMed]

*PLoS Computational Biology*, 5(9), e1000504, doi:10.1371/journal.pcbi.1000504. [CrossRef] [PubMed]

*Proceedings of the National Academy of Sciences*, 107(48), 20834–20839, doi:10.1073/pnas.1007704107. [CrossRef]

*Journal of Experimental Psychology: Human Perception and Performance*, 43(8), 1520–1531, doi:10.1037/xhp0000404. [CrossRef] [PubMed]

*Spatial Vision*, 10(4), 433–436. [CrossRef] [PubMed]

*Nature Communications*, 10(1), 1719, doi:10.1038/s41467-019-09330-7. [CrossRef] [PubMed]

*Journal of Vision,*16(1), 1–10, doi:10.1167/16.1.17. [CrossRef]

*Journal of the Optical Society of America A*, 18(5), 1016–1026. [CrossRef]

*PLoS One*, 10(3), e0120870, doi:10.1371/journal.pone.0120870. [CrossRef] [PubMed]

*Nature Neuroscience*, 16(1), 105–110, doi:10.1038/nn.3279. [CrossRef] [PubMed]

*Proceedings of the National Academy of Sciences*, 115(43), 11090–11095, doi:10.1073/pnas.1717720115. [CrossRef]

*Proceedings of the National Academy of Sciences,*95(23), 13988–13993, doi:10.1073/pnas.95.23.13988. [CrossRef]

*Journal of Experimental Psychology: Human Perception and Performance*, 26(5), 1534–1548, doi:10.1037/0096-1523.26.5.1534. [CrossRef] [PubMed]

*Eneuro,*7(3), doi:10.1523/ENEURO.0326-19.2020.

*Nature Neuroscience*, 14(7), 926–932, doi:10.1038/nn.2831. [CrossRef] [PubMed]

*Neural Computation*, 28(9), 1840–1858, doi:10.1162/NECO_a_00864. [CrossRef] [PubMed]

*Proceedings of the National Academy of Sciences*, 117(15), 8391–8397, doi:10.1073/pnas.1918143117. [CrossRef]

*Philosophical Transactions of the Royal Society B: Biological Sciences*, 367(1594), 1322–1337, doi:10.1098/rstb.2012.0037. [CrossRef]

*TRENDS in Neuroscience*, 27(12), 712–719, doi:10.1016/j.tins.2004.10.007. [CrossRef]

*Attention, Perception, & Psychophysics*, 77(4), 1295–1306, doi:10.3758/s13414-015-0843-3. [CrossRef]

*Journal of Experimental Psychology: General*, 140(1), 117–139, doi:10.1037/a0022171. [CrossRef] [PubMed]

*Consciousness and Cognition*, 21(1), 422–430, doi:10.1016/j.concog.2011.09.021. [CrossRef] [PubMed]

*Neuroscience of Consciousness*, 2015(1), niv002, doi:10.1093/nc/niv002.

*Attention, Perception, & Psychophysics*, 78(3), 923–937, doi:10.3758/s13414-016-1059-x. [CrossRef]

*Neuron*, 88(1), 78–92, doi:10.1016/j.neuron.2015.09.039. [CrossRef] [PubMed]

*Journal of Vision*, 16(3), 36, doi:10.1167/16.3.36. [CrossRef] [PubMed]

*Journal of Vision*, 8(11), 9 1–8, doi:10.1167/8.11.9. [CrossRef] [PubMed]

*Nature Human Behaviour*, 1(11), 810–818, doi:10.1038/s41562-017-0215-1. [CrossRef] [PubMed]

*Spatial Vision*, 10(4), 437–442. [CrossRef] [PubMed]

*Cortex,*93, 119–132, doi:10.1016/j.cortex.2017.05.017. [CrossRef]

*Nature Human Behaviour,*1(7), 1–8, doi:10.1038/s41562-017-0139. [CrossRef]

*Psychological Review*, 117(3), 864–901, doi:10.1037/a0019737. [CrossRef] [PubMed]

*Nature Neuroscience*, 19(3), 366–374, doi:10.1038/nn.4240. [CrossRef] [PubMed]

*Behavoral and Brain Sciences,*41,1–107, doi:10.1017/S0140525X18000936. [CrossRef]

*Journal of Neuroscience*, 31(29), 10741–10748, doi:10.1523/JNEUROSCI.1478-11.2011. [CrossRef] [PubMed]

*Journal of Neurophysiology*, 107(6), 1556–1563, doi:10.1152/jn.00985.2011. [CrossRef] [PubMed]

*Frontiers in Psychology*, 7, 851, doi:10.3389/fpsyg.2016.00851. [CrossRef] [PubMed]

*Neuron*, 90(3), 499–506, doi:10.1016/j.neuron.2016.03.025. [CrossRef] [PubMed]

*Journal of Experimental Psychology: Human Perception and Performance*, 42(5), 671–682, doi:10.1037/xhp0000179. [CrossRef] [PubMed]

*Vision Research*, 50(5), 541–547, doi:10.1016/j.visres.2009.12.005. [CrossRef] [PubMed]

*Neuroimage*, 180(Pt A), 78–87, doi:10.1016/j.neuroimage.2017.08.015. [CrossRef] [PubMed]

*Journal of Neuroscience*, 39(41), 8164–8176, doi:10.1523/JNEUROSCI.3212-18.2019. [CrossRef] [PubMed]

*Nature Neuroscience*, 18(12), 1728–1730, doi:10.1038/nn.4150. [CrossRef] [PubMed]

*Nature Neuroscience*, 23(1), 122–129, doi:10.1038/s41593-019-0554-5. [CrossRef] [PubMed]

*Annual Review of Psychology*, 69, 105–129, doi:10.1146/annurev-psych-010416-044232. [CrossRef] [PubMed]

*BioRxiv*, 2021.01.08.425966. Retrieved from http://biorxiv.org/content/early/2021/01/09/2021.01.08.425966.abstract.

*Elife,*5, doi:10.7554/eLife.17688.

*Consciousness and Cognition*, 27, 246–253, doi:10.1016/j.concog.2014.05.012. [CrossRef] [PubMed]