Cross-modal metacognition: Visual and tactile confidence share a common scale

Abstract

Humans can judge the quality of their perceptual decisions—an ability known as perceptual confidence. Previous work suggested that confidence can be evaluated on an abstract scale that can be sensory modality-independent or even domain-general. However, evidence is still scarce on whether confidence judgments can be directly made across visual and tactile decisions. Here, we investigated in a sample of 56 adults whether visual and tactile confidence share a common scale by measuring visual contrast and vibrotactile discrimination thresholds in a confidence-forced choice paradigm. Confidence judgments were made about the correctness of the perceptual decision between two trials involving either the same or different modalities. To estimate confidence efficiency, we compared discrimination thresholds obtained from all trials to those from trials judged to be relatively more confident. We found evidence for metaperception because higher confidence was associated with better perceptual performance in both modalities. Importantly, participants were able to judge their confidence across modalities without any costs in metaperceptual sensitivity and only minor changes in response times compared to unimodal confidence judgments. In addition, we were able to predict cross-modal confidence well from unimodal judgments. In conclusion, our findings show that perceptual confidence is computed on an abstract scale and that it can assess the quality of our decisions across sensory modalities.

Introduction

We explore the world with multiple senses. What we perceive is the result of committing to perceptual decisions that are derived from uncertain sensory information. Along with these perceptual decisions usually comes a subjective, probabilistic estimate of how confident we are that this decision is correct (Fleming, Dolan, & Frith, 2012) or self-consistent (Caziot & Mamassian, 2021). Perceptual confidence refers to an observer's ability to evaluate, monitor, and control their own perception and has been established as one type of “metacognition” (Fleming, Dolan, & Frith, 2012; Fleming & Lau, 2014; Mamassian, 2016). Typically, subjective confidence judgments and objective perceptual performance are correlated—irrespective of whether confidence judgments were made within the visual (Barthelmé & Mamassian, 2010), tactile (Pierce & Jastrow, 1884), or auditory modality (Emmerich, Gray, Watson, & Tanis, 1972). Perceptual confidence has been shown to be essential for behavioral decision-making (Desender, Boldt, & Yeung, 2018), setting decision criteria (van den Berg, Zylberberg, Kiani, Shadlen, & Wolpert, 2016), as well as allocating appropriate resources (Aguilar-Lleyda, Lemarchand, & de Gardelle, 2020). These functions become even more relevant in noisy environments, in which it would be helpful to rely on cues from different modalities. For instance, in everyday life, we could imagine that we want to cross a road on a rainy and foggy day. But before actually doing it, we might rather look twice, check whether we hear an engine or feel the ground vibrating. Considering our confidence in each of these perceptual decisions may help us decide whether it is safe to cross the road or not. However, this would require that confidence can be efficiently compared across modalities. 

Over the last years, the question how confidence can be compared across perceptual tasks, including different modalities, or even across functional domains, such as perception and cognition, has gained increasing attention. In particular, the question whether one common metacognitive mechanism monitors performance across different tasks or whether specific mechanisms monitor individual tasks has been vividly debated (Mazancieux, Fleming, Souchay, & Moulin, 2020; Morales, Lau, & Fleming, 2018; Rouault, McWilliams, Allen, & Fleming, 2018). Typically, both possibilities have been explored using a correlational approach (Faivre, Filevich, Solovey, Kühn, & Blanke, 2018; Mazancieux et al., 2020; Song et al., 2011). Specific tasks are completed separately, followed by confidence judgments on a rating scale. Generality and specificity of metacognitive mechanisms, respectively, is then evaluated based on the shared variance in metacognitive performance (i.e., the capacity to estimate the accuracy of task performance) that is observed across tasks. Only few studies have addressed the question more directly by comparing confidence judgments not only within tasks but also across tasks (e.g., Baer & Odic, 2020; de Gardelle, Le Corre, & Mamassian, 2016; de Gardelle & Mamassian, 2014). Despite these different approaches, findings mainly point toward a common metacognitive mechanism—at least within the perceptual domain. Studies investigating the link between confidence judgments in cognitive and perceptual tasks have provided mixed results (for a review, see Rouault et al., 2018). 

Within the perceptual domain, it has been shown that observers can directly compare their confidence across two different visual tasks (de Gardelle & Mamassian, 2014) as well as across a visual and an auditive task (de Gardelle et al., 2016). As cross-modal confidence judgments were possible without any costs in metacognitive sensitivity and only minor costs in response times compared to unimodal confidence judgments, confidence seems to be represented in an abstract, modality-independent format. In other words, confidence can be estimated using a “common currency” across different perceptual decisions. This interpretation is also supported by evidence from correlational approaches showing similar metacognitive performance across visual, tactile, and auditory tasks (Faivre et al., 2018; Song et al., 2011; Ais, Zylberberg, Barttfeld, & Sigman, 2016). Furthermore, a general representation of confidence has been described already early in development, at least by the age of six years (Baer & Odic, 2020), highlighting the need for a mechanism that enables the integration and comparison of perceptual decisions. 

At the neural level, metacognitive processes across tasks involve shared and distinct brain regions, depending on the task at hand (Morales et al., 2018; Rouault et al., 2018), with domain-general components relying on a network encompassing the prefrontal and cingulate cortex (Fleming & Dolan, 2012; Fleming, Huijgen, & Dolan, 2012; Morales et al., 2018). Given the involvement of the prefrontal cortex, metacognitive processes are thought to be closely related to cognitive control operations (Fernandez-Duque, Baird, & Posner, 2000; Fuster, 2000; Klever, Mamassian, & Billino, 2022; Roebers, 2017), supporting general mechanisms in metacognition (see Rouault, Lebreton, & Pessiglione, 2022). 

A common currency between the visual and tactile senses appears particularly useful because both senses are closely tied to each other when performing actions. Although tactile confidence was first investigated more than 100 years ago (Pierce & Jastrow, 1884), our understanding of it—especially in multisensory situations—still lags behind (Faivre, Arzi, Lunghi, & Salomon, 2017). So far, a common currency between the visual and tactile senses is only supported by one study finding a moderate correlation between metacognitive ability in a vibrotactile and a contrast discrimination task (Faivre et al., 2018). Because these two perceptual tasks were performed separately, potential effects of cross-modal interactions might be obscured. When asking participants to directly compare their confidence across tasks, their ability to adequately judge their confidence could be affected by favoring one modality over the other. Recently, it has been proposed that the tactile sense might provide higher subjective certainty than vision when faced with ambiguous evidence in an illusory setting—even when tactile decisions were less accurate (Fairhurst, Travers, Hayward, & Deroy, 2018). In contrast, confidence was higher in vision than touch in unambiguous cases and seemed to optimally track objective accuracy. This selective overconfidence in the tactile sense could arise from an observer's belief that touch provides more directness (Deroy & Fairhurst, 2019). Both the visual and tactile senses are actively used to sample information on our surroundings (Findlay & Gilchrist, 2003; Gibson, 1962). Given the direct proximity to the target, information obtained from the tactile sense could provide greater reassurance as it might make us feel we sampled this information more actively rather than passively (Deroy & Fairhurst, 2019). Critically, overconfidence in the tactile sense could challenge the idea of a common currency between visual and tactile senses. However, given its specificity to perceptual ambiguity, the findings by Fairhurst et al. (2018) might simply highlight the notion that confidence is best understood as a measure of self-consistency rather than correctness (Caziot & Mamassian, 2021). In other words, both visual and tactile perceptions might be inaccurate, but the tactile bias might be more consistent across repeated estimates. Interestingly, when Fairhurst et al. (2018) used a measure of subjective accuracy (i.e. an observer's internal response consistency) instead of objective accuracy, confidence seemed to optimally track subjective accuracy in both modalities. 

Here, we investigated whether confidence serves as a common currency between the visual and tactile sense—two senses that closely interact and are especially relevant for the planning and execution of actions. Given this close interaction, we wanted to directly examine how well observers can compare their confidence across a visual and a tactile task. To this end, we applied the confidence forced-choice paradigm (Mamassian, 2020; Mamassian & de Gardelle, 2021), where participants performed two perceptual tasks in succession and then selected the perceptual decision that they think is more likely to be correct. If confidence was modality-specific, we would expect that confidence judgments across perceptual modalities are harder than within the same modality. Conversely, if it was modality-independent, confidence judgments should not be affected by whether the perceptual tasks involved the same or different modalities. As cross-modal confidence judgments might be costly in terms of processing time, we additionally considered potential differences in response times. Furthermore, we explored whether individual differences in cognitive control capacities are linked to confidence, which could point toward the involvement of further similar (i.e., domain-general) processes (Klever, Mamassian, & Billino, 2022; Rouault et al., 2018). 

Methods

Participants

A total of 56 participants (13 males) with a mean age of 24.1 years (SD = 5.8 years) took part in this study. Sample size was determined on the basis of previous studies using similar experimental procedures for measuring perceptual confidence across different tasks (cf., de Gardelle et al., 2016; de Gardelle & Mamassian, 2014). The experimental design required a minimum of 24 participants for counterbalancing the four different trial configurations (see Figure 1). A power calculation done with G*Power toolbox (Faul, Erdfelder, Lang, & Buchner, 2007) indicated that a sample size of 48 participants allowed detection of moderate effects between different experimental conditions with a power of 92% and an α-level of .05. Because we expected, based on previous studies using similar perceptual measurements, that quality of some data sets might not allow for the planned analyses, our final sample comprised few additional participants. 

Figure 1.

 

Procedure and subtasks of the confidence forced-choice paradigm. (A) Schematic illustration of the overall trial procedure and the four different trial configurations. Participants completed two perceptual tasks in succession (either visual-visual, tactile-tactile, visual-tactile or tactile-visual) and then provided a forced-choice confidence judgment (i.e. they indicated which of the two perceptual decisions [first or second] they felt more confident being correct). (B) Visual task. Participants first saw a fixation dot, which was followed by the simultaneous presentation of two Gabor patches. Then, they decided which of the two Gabor patches appeared higher in contrast. (C) Tactile task. First, participants were presented with a fixation dot. Then, they received two simultaneous vibrations on both index fingers and decided afterward on which finger the vibration felt stronger.

All participants had normal or corrected-to-normal vision and no history of ophthalmologic, neurologic, or psychiatric disorders. We characterized individual cognitive control abilities for each participant using a battery of established tasks that covers key facets of executive functions (Diamond, 2013). These functions include updating ability, as measured with the Digit Symbol Substitution Test (Wechsler, 2008), shifting ability, as measured with Part B of the Trail Making Test (Kortte, Horner, & Windham, 2002; Reitan & Wolfson, 1985), inhibition ability, as measured with the Victoria Stroop Test color naming (Mueller & Piper, 2014; Stroop, 1935), and nonverbal reasoning ability, as measured with subtest 3 of the LPS-2 (Kreuzpointner, Lukesch, & Horn, 2013). To obtain a robust, composite measure of cognitive control capacities, we z-standardized the scores for each task and then averaged them for each participant. Given the nature of our metacognitive task, we additionally assessed the maximal backward digit span (Härting et al., 2000) to evaluate short-term memory capacity that could present a confounding issue. 

Methods and procedures were approved by the local ethics committee of the Faculty of Psychology and Sports Science, Justus Liebig University Giessen, and were carried out in accordance with the guidelines of the Declaration of Helsinki (World Medical Association, 2013). Participants provided written informed consent before the experiment and were compensated with course credits or money. 

Setup

Visual stimuli were presented on a calibrated 32″ Display++ LCD monitor (Cambridge Research Systems, Rochester, UK) with a spatial resolution of 1920 × 1080 pixels and a refresh rate of 120 Hz (non-interlaced) using the Psychophysics Toolbox (Brainard, 1997; Kleiner, 2010) in Matlab (The Mathworks, Inc., Natick, MA, USA). The background was average gray. Participants sat at a table in a darkened room with their head stabilized on a chin rest. The eye monitor distance was 100 cm, leading to a display size of 41° × 23°. Luminance of white and black pixels was 112.7 and 0.1 cd/m², respectively, as measured with a CS-2000 Spectroradiometer (Konica Minolta, Tokyo, Japan). Tactile stimuli were applied by custom-made vibrotactile devices (Engineering Acoustics Inc., Casselberry, FL, USA). They were attached on the tip of both index fingers using silicone finger sleeves. Participants comfortably rested their hands shoulder-width apart on foam pads in front of them. Because of the setup for tactile stimulation, manual response input was excluded. Thus we used gaze positions as response input. Eye positions were recorded using an SR Research Eyelink 1000 Desktop Mount system (SR Research Ltd., Mississauga, Ontario, Canada). 

Stimuli and procedure

Metacognitive performance was assessed in an established confidence forced-choice paradigm (Mamassian, 2020; Mamassian & de Gardelle, 2021). The paradigm has been proposed to derive a bias-free measure of confidence and avoids some confounds emerging from confidence rating scales. The approach focuses on metacognitive sensitivity, i.e. an observer's ability to adequately judge the quality of their perceptual decisions, rather than confidence bias (Mamassian & de Gardelle, 2021). Figure 1A summarizes the configuration of an individual trial. Each trial comprised two consecutive perceptual decision tasks and a confidence task. The consecutive perceptual decision tasks could either tap the same modality (i.e., visual-visual or tactile-tactile) or different modalities (i.e., visual-tactile or tactile-visual). After accomplishing the perceptual tasks, participants indicated which of the two perceptual decisions they felt more confident about. Two types of confidence judgments can be distinguished: unimodal judgments (i.e., within the visual or tactile modality) and cross-modal judgments (i.e. across the visual and tactile modalities). Based on the four possible trial configurations, the experiment was divided into four separate blocks with 112 trials each, resulting in a total of 448 trials. To minimize task switching costs, the trial configuration was kept constant within each block. Block order was counterbalanced across participants. Before each block, participants completed 14 training trials to familiarize themselves with the respective trial configuration. After each block, they had the opportunity to take a break. Before data collection, we provided an introduction to our procedure. In particular, participants practiced the single perceptual tasks and became acquainted with providing responses via gaze. The visual and tactile tasks are described in the following. 

The visual task (Figure 1B) started with a 500 ms presentation of a central black fixation dot that subtended 0.2°. Then, two vertical Gabor patches were simultaneously shown for 180 ms on the left and right of the fixation dot at 4.2° eccentricity. All Gabor patches had a spatial frequency of 0.8 cyc/°. The standard deviation of the Gaussian envelope was 1°, and the phase was randomized. Of these two Gabor patches, one always had a fixed contrast at 22% (standard Gabor patch), while the contrast of the other Gabor patch was adapted throughout the experiment (test Gabor patch). Laterality of standard and test Gabor patch was randomized. Next, the fixation dot turned blue and two dark-gray response squares were shown at 6.8° eccentricity left and right from the fixation dot. Participants’ task was to decide whether the left or right Gabor patch appeared higher in contrast by looking at the respective response square. When a square was selected, it turned darker. Based on participants’ decision, the contrast for the test Gabor patch of the next trial was adapted by one of two randomly interleaved three-down/one-up staircases in steps of 3%: One staircase had a starting value of 31% and aimed at responses favoring the test stimulus ∼80% of the time; the other had a starting value of 13% and aimed at responses favoring the standard stimulus ∼80% of the time. This procedure was based on the methods used in the visual-auditory confidence study of de Gardelle and colleagues (2016). The interleaved double-staircase method allowed for an overall stable performance level across the different task conditions and for minimizing response biases (see also Cornsweet, 1962). In particular, with the interleaved procedure we aimed to avoid systematic biases in confidence judgments due to unidirectional stimulus intensity changes. 

The tactile task (Figure 1C) began with the same fixation dot configuration as the visual task. Then, participants received two simultaneous vibrations for 500 ms on both index fingers at a frequency of 200 Hz. Of these two vibrations, one had a fixed intensity, defined as peak-to-peak displacement, of 0.13 mm (standard vibration). The intensity of the other vibration (test vibration) was adapted throughout the experiment. Again, laterality of standard and test stimuli was randomized. When the horizontal response squares were shown, participants had to decide whether the vibration on the left or right index finger felt stronger by looking at the according square. Using similar staircases to the visual task with starting values of 0.08 mm and 0.18 mm, respectively, the intensity for the test vibration of the next trial was adapted in steps of 0.02 mm. 

After the completion of two perceptual tasks, the confidence judgments were also given by gaze. A blue central fixation dot and two dark-grey response squares were shown at 6.8° eccentricity below and above the fixation dot. The response squares were numbered and associated with the first or second perceptual decision. The mapping was visualized on the screen and balanced across participants. By looking at one of the two squares, participants indicated which perceptual decision (first or second) they felt more confident being correct. 

Data analyses

Perceptual decisions were separated according to modality (visual or tactile) and comparison type of the confidence judgment (unimodal or cross-modal), resulting in four conditions: visual unimodal, visual cross-modal, tactile unimodal and tactile cross-modal. Based on participants' confidence judgments, we divided perceptual decisions in each condition into two confidence sets: The first set included perceptual decisions that were chosen in the confidence task (i.e., they were associated with a relatively higher confidence). Accordingly, we labeled this set as chosen. The second set comprised all perceptual decisions and was labelled as unsorted. Please note that due to the design of the confidence forced-choice paradigm the number of perceptual decisions chosen as confident is equal in both unimodal conditions (i.e., visual and the tactile conditions). For the cross-modal conditions, the number of chosen decisions for each modality can vary due to possible biases toward either the visual or tactile modality, respectively. On average, we observed a marginal bias toward choosing tactile decisions as more confident, favoring them in 53.7% of the judgments. Though statistically significant, t(55) = 2.30, p = 0.025, d = 0.31, we considered the absolute imbalance as minor, most importantly not complicating our further psychometric analyses. 

We evaluated perceptual performance separately for each confidence set and condition by fitting cumulative Gaussian functions to the percentage of responses in which observers favored the test stimulus over the standard stimulus. The inverse standard deviation of the fitted psychometric functions provides a measure of sensitivity. We used the Psignifit 4 toolbox in Matlab for the fitting process because it yields an accurate estimation of psychometric functions in a Bayesian framework even if the measured data are overdispersed (Schütt, Harmeling, Macke, & Wichmann, 2016). Goodness of fit of the psychometric functions was assessed with the measure of deviance D, which supported good fits between the model and the data. By inspecting boxplots for the derived sensitivity measures, we identified two participants who showed visual or tactile sensitivities that deviated more than 1.5 times the interquartile range from the range borders. We considered these measures as outlier data and discarded the participants from further analyses to reduce unsystematic noise in our data. 

To analyze metacognitive efficiency (i.e., the relative sensitivity gain driven by confidence), we calculated a confidence modulation index (CMI) according to Equation 1. The CMI quantifies metacognitive ability as the gain in sensitivity from the set of unsorted trials to the set of chosen trials standardized by the sensitivity derived from the unsorted trials. Thus CMIs will increase with better metacognitive sensitivity. If an individual observer shows low metacognitive sensitivity, CMIs will be close to zero. Importantly, as a unit-free proportional measure, the CMI allows us to compare metacognitive sensitivity across both modalities. CMIs were arcsine-square-root transformed for variance stabilization.  

\begin{eqnarray}\!\!\!\!\!\!\!\!\!\!\!\!\! CMI = 100 \times \frac{{Sensitivit{y_{chosen}} - Sensitivit{y_{unsorted}}}}{{Sensitivit{y_{unsorted}}}}\;\;\, \end{eqnarray}

(1)

 

Processing measures for perceptual decisions as well as confidence judgments were explored using median response times (RT). We excluded RTs <150 ms and >3000 ms because they were considered as anticipatory or delayed, respectively. Please note that perceptual and confidence response times were not measured equivalently, as perceptual decisions were made via horizontal saccades and confidence judgments via vertical saccades. Although reaction times for horizontal and vertical saccades, respectively, are generally found similar, different underlying mechanisms might trigger systematic differences (Becker & Jürgens, 1990, but see Dafoe, Armstrong, & Munoz, 2007). Because perceptual decision times are influenced by stimuli intensity and confidence (Baranski & Petrusic, 1994; Kiani, Corthell, & Shadlen, 2014), we separated the effects of both parameters using a model that was successfully applied in previous studies (de Gardelle et al., 2016; Klever et al., 2022). First, we normalized stimulus values for each participant, confidence set, modality and comparison type. This was realized by calculating the signed distances S between the stimulus intensities and the respective point of subjective equality in standard deviation units of the psychometric function. Next, we divided the normalized stimulus values into five bins and calculated the median response time, as well as the average confidence judgment C (encoded as 0 for unchosen and 1 for chosen perceptual decisions) for each bin. Then we fitted an exponential model with three free parameters (as defined by Equation 2) to the median RTs, separately for each condition. The estimated parameters are the following: α provides the baseline RT, β reflects the exponential change in RT due to stimulus difficulty, and γ captures the linear decrease in RT due to confidence.  

\begin{equation}RT\left( S \right) = \alpha - \beta \times {e^{ - \frac{1}{2}{S^2}}} - \gamma \times C\end{equation}

(2)

 

Perceptual sensitivities and RT were analyzed separately for each modality using repeated-measures analyses of variance (ANOVAs) with the within-subject factor confidence set (chosen vs. unsorted) and the within-subject factor comparison (unimodal vs. cross-modal). To compare metacognitive sensitivity across modalities, we submitted CMIs to a repeated-measures ANOVA with the within-subject factor modality (visual vs. tactile) and comparison (unimodal vs. cross-modal). Two-sided t-tests were used to further analyze CMIs and RT parameters. In case of unequal variances as indicated by Levene's test, degrees of freedom were adjusted. Associations between CMIs and cognitive measures were scrutinized by correlational analyses. For all statistical analyses, a significance level of α = .05 was applied. Descriptive values are reported as means ± 1 SEM, unless stated otherwise. 

Results

We initially explored response patterns across all combinations of modality (visual vs. tactile) and comparison (unimodal vs. cross-modal). Then we analyzed for each modality and comparison how perceptual sensitivity functions were shaped by whether they were derived from chosen or unsorted confidence sets, respectively. In a next step, we combined these data in a confidence modulation index (CMI) and compared metacognitive efficiency across modalities. Finally, we considered contributions of processing speed and cognitive resources in the formation of confidence. 

Overview of response patterns

A rough indicator of metacognition is given by differences in confidence with regard to correct and incorrect trials. Typically, participants should report higher confidence when their perceptual decision was objectively correct and lower confidence when their decision was objectively incorrect. Figure 2 illustrates average confidence judgments for correct and incorrect perceptual decisions at different normalized stimulus intensity levels. 

Figure 2.

 

Average confidence judgments for correct (green) and incorrect (red) perceptual decisions at different intensity levels in the visual task (A) and tactile task (B), separately for the type of the comparison for the confidence judgments (unimodal vs. cross-modal). Intensity levels are given as the absolute difference between stimulus intensities and each participant's point of subjective equality in standard deviation units of the psychometric function. They were then divided into five equidistant bins of varying stimulus difficulty with higher values indicating lower stimulus difficulty. Confidence judgments were coded as 1 for chosen and 0 for unchosen perceptual decisions. Please note that confidence judgments were made between two perceptual decisions in a trial. The probability of choosing a decision as confident depends on the difficulty of the decision in the other interval. We collapsed confidence judgments across the different difficulties. Error bars provide 95% CIs.

The overall response patterns suggest that participants evaluated their perceptual performance appropriately in all conditions. There were no prominent differences between unimodal and cross-modal comparison conditions: Average confidence judgments were consistently higher for correct than incorrect trials. Additionally, the difference in average confidence judgments between correct and incorrect trials became more evident with decreasing stimulus difficulty. 

Psychometric analyses

We were interested in determining whether sensitivities vary systematically between the chosen and unsorted trial sets, as well as unimodal and cross-modal judgments. Figure 3 shows example psychometric functions for contrast discrimination (A) and vibrotactile intensity discrimination (B) for one representative participant. 

Figure 3.

 

Representative psychometric functions of contrast discrimination (A) and vibrotactile intensity discrimination (B) for both unimodal and cross-modal comparison types. The proportion of choosing the test stimulus over the standard stimulus is plotted as a function of stimulus intensity. Stimulus intensity is given as the difference between the test and standard stimulus, which is reported in percent for the visual task and peak-to-peak displacement in millimeters for the tactile task. Dashed lines and open dots depict data from the confident chosen trial set, solid lines, and filled dots represent data from the unsorted trial set.

As sensitivities cannot be compared across modalities, we submitted sensitivity data separately for each modality to repeated-measures ANOVAs with the within-subject factors confidence set (unsorted vs. chosen) and comparison (unimodal vs. cross-modal). The analysis yielded a strong main effect of confidence set for the visual task, F(1, 53) = 109.42, p < 0.001, η_p² = 0.67, as well as the tactile task, F(1, 53) = 172.25, p < 0.001, η_p² = 0.77. Sensitivities were consistently higher for the chosen confidence set in comparison to the unsorted confidence set, indicating that participants were able to select the perceptual decision that is more likely to be correct. Furthermore, there was a main effect of comparison for the visual task, F(1, 53) = 4.65, p = 0.036, η_p² = 0.08, but not for the tactile task, F(1, 53) = 0.03, p = 0.857, η_p² < 0.01. Visual sensitivity was overall higher when derived from perceptual decisions in the unimodal as compared to the cross-modal condition, suggesting that contrast discrimination was better when the task remained the same during one condition. The interaction between confidence set and comparison did not reach significance in either modality, visual: F(1, 53) = 0.01, p = 0.947, η_p² < 0.01, tactile: F(1, 53) = 1.20, p = 0.394, η_p² = 0.02. The absence of an interaction is particularly interesting as it suggests that participant's ability to select the perceptual decision that is more likely to be correct was unaffected by whether they had to choose between perceptual decisions from the same or different modalities. Figure 4 illustrates the effects of confidence set and comparison type on sensitivities separately for the visual task (A) and tactile task (B). 

Figure 4.

 

Average perceptual sensitivity as a function of confidence set and comparison type, separately for the visual task (A) and tactile task (B). Open bars represent mean sensitivities from the as confident chosen trial set, and filled bars show mean sensitivities from the unsorted trial set. Error bars indicate 95% CIs.

Confidence efficiency

As the measures of sensitivity do not allow for a direct comparison between the visual and tactile tasks, we further analyzed effects of modality and comparison on confidence efficiency with the help of a CMI (see Methods). For the visual task, the average CMI was 26.03 ± 2.11 in the unimodal condition and 28.90 ± 2.50 in the cross-modal condition. For the tactile task, the average CMI was 28.96 ± 1.57 in the unimodal condition and 31.90 ± 2.57 in the cross-modal condition. T-tests confirmed that CMIs were consistently greater than zero in all conditions (all ps < 0.001). Figure 5 displays average CMIs for both modalities and types of comparison. 

Figure 5.

 

Confidence modulation index (CMI) as a function of modality (visual in blue vs. tactile in orange) and comparison (unimodal vs. cross-modal). The CMI is a proportional measure reflecting the change in sensitivity from the set of unsorted trials to the chosen trials relative to the unsorted trials. Higher CMIs reflect higher metacognitive sensitivity. Colored dots represent individual data points; black dots display the mean across observers with error bars indicating 95% CIs.

We submitted CMIs to a repeated-measures ANOVA with the within-subject factors modality (visual vs. tactile) and comparison (unimodal vs. cross-modal). There was no significant main effect of modality, F(1, 53) = 2.28, p = 0.137, η_p² = 0.04, or comparison, F(1, 53) = 2.50, p = 0.120, η_p² = 0.05, and no interaction between modality and comparison, F(1, 53) < 0.01, p = 0.986, η_p² < 0.01. Since the absence of any effects would be expected from the hypothesis that confidence is stored in a modality-independent format, we calculated the corresponding Bayes Factors (BF) to back up these results. Analyses of BF provided evidence that neither modality, BF₁₀ = 0.42, nor comparison, BF₁₀ = 0.41, in isolation, nor their interaction, BF₁₀ = 0.19, have an effect on metacognitive efficiency. 

The previous analysis used the CMI that is based on the global psychometric function (see again Figure 3) where each stimulus strength presented in one interval is compared to all the other stimulus strengths in the other intervals. We can also perform a finer analysis by trying to fit the confidence choice probabilities between each stimulus strength across the two intervals. The problem with this analysis is that it requires a large number of trials (Mamassian & de Gardelle, 2021), so we decided to pool the trials across all participants after transforming their perceptual data into standard scores (substracting the perceptual bias and dividing by the sensory noise). The data were then grouped into six equal-sized bins and submitted to a model of confidence forced-choice to fit the 576 confidence choice probabilities (i.e., \((6_{{\rm{visual}}}+ 6_{{\rm{tactile}}})^{\wedge}{2}_{{{\rm{intervals}}}}\; \times 4_{{{\rm{typel-responses}}}}\)) using the Matlab code package provided in Mamassian & de Gardelle (2021). 

We considered two models. In model 1, we only fitted the confidence choice probabilities for the unimodal comparisons (visual-visual and tactile-tactile), but applied this model to all the confidence choice probabilities (Figure 6A). In model 2, we fitted the confidence choice probabilities for both unimodal and cross-modal comparisons (Figure 6B). Replicating the previous analysis, we did not find any significant difference between metacognitive abilities across the two tasks, namely confidence efficiency was 0.376 for the visual task (95% CI = [0.309, 0.463], obtained from 100 bootstraps) and 0.365 for the tactile task (95% CI = [0.294, 0.427]). Importantly, there was no difference in the goodness of fits between models 1 and 2 as estimated by the BIC (Bayesian information criterion) measure (Figure 6C). A Kolmogorov-Smirnov test indicates that the two models did not differ significantly in the quality of the fits, D(100) = .140, p = 0.261. In other words, the cross-modal confidence comparisons could be predicted very well from the unimodal comparisons, consistent with the hypothesis that confidence is computed in a modality-independent format. 

Figure 6.

 

Goodness of fits of two models of confidence choice probabilities. (A) Model 1 was only fitted to the unimodal confidence comparisons but nonetheless predicted very well both unimodal and cross-modal comparisons. There are 576 dots in this plot corresponding to all the combinations of stimulus strengths in both intervals and all four possible perceptual response categories. Dot size is proportional to the number of trials in the combination. (B) Model 2 was fitted to both unimodal and cross-modal comparisons, and it also predicted human data very well. (C) A comparison of the distributions of Bayesian information criteria (BIC) for models 1 and 2 (in dark and light purple, respectively) obtained from 100 bootstraps indicates that the two models did not differ in the quality of the fits.

Response times

We were further interested in potential differences between the unimodal and cross-modal conditions in terms of processing time and how they might contribute to the calibration of confidence. Perceptual response times were faster for visual decisions, M = 586.36 ± 21.20, in comparison to tactile decisions, M = 702.18 ± 19.67, t(53) = 6.29, p < 0.001, d = 0.86. To analyze whether perceptual response times vary systematically between the two confidence sets and the types of confidence comparison, we submitted them separately for each modality to a repeated-measures ANOVA with the within-subject factors confidence set (unsorted vs. chosen) and comparison (unimodal vs. cross-modal). We found a main effect of confidence set in the visual task, F(1, 53) = 29.87, p < 0.001, η_p² = 0.36, as well as the tactile task, F(1, 53) = 43.01, p < 0.001, η_p² = 0.45, indicating faster responses with higher confidence. Additionally, there was a main effect of comparison in both modalities; visual: F(1, 53) = 74.79, p < 0.001, η_p² = 0.59, tactile: F(1, 53) = 6.71, p = 0.012, η_p² = 0.11. However, the direction of the effect differed between both modalities: In the visual task, responses were faster in the unimodal condition relative to the cross-modal condition. Whereas in the tactile task, responses were slightly faster in the cross-modal condition compared to the unimodal condition. There was no interaction between confidence set and comparison in the visual task, F(1, 53) = 0.09, p = .768, η_p² < .01, or tactile task, F(1, 53) = 0.23, p = 0.633, η_p² < 0.01. Figure 7 illustrates effects of confidence set and comparison on median response times separately for each modality. 

Figure 7.

 

Median response times as a function of confidence set (chosen vs. unsorted) and comparison (unimodal vs. cross-modal) for the visual task (A) and tactile task (B). Bars show the mean across observers in each condition with open bars representing the chosen trial set and filled bars the unsorted trial set. Error bars indicate 95% CIs.

However, as response times are not only affected by confidence but vary critically with stimulus difficulty, we modeled the relationship between stimulus difficulty, confidence and response times (see Methods for details). Figure 8 illustrates the fitting results for each modality and type of comparison. The control analysis yielded three parameters, but only α (the generic RT) and γ (the confidence effect) are of primary interest. Both parameters were analyzed using repeated-measures ANOVAs and t-tests. In line with the previous analysis, the parameter α exhibited a significant main effect of modality, F(1, 53) = 62.04, p < 0.001, η_p² = 0.54, corroborating that responses were faster for visual decisions compared to tactile decisions. Interestingly, there was also a significant main effect of comparison, F(1, 53) = 9.34, p = 0.004, η_p² = 0.15, indicating slower responses in cross-modal blocks compared to unimodal blocks, and no interaction, F(1, 53) = 0.01, p = 0.929, η_p² < 0.01. In contrast to the previous analysis, γ (confidence effect) did not differ significantly from zero in all conditions (all ps > 0.289, all ds < 0.15), except for the visual cross-modal condition, t(53) = 2.30, p = 0.025, d = 0.31. However, an ANOVA suggests that γ was unaffected by modality, F(1, 53) = 0.22, p = 0.644, η_p² < 0.01, comparison, F(1, 53) = 0.61, p = 0.438, η_p² = 0.01, or their interaction, F(1, 53) = 0.97, p = 0.330, η_p² = 0.02. Thus the speed-up of RTs with confidence found in the previous analysis can be attributed to variations in stimulus difficulty. 

Figure 8.

 

Additional analysis controlling for the effect of stimulus difficulty on response times. Median response times for five equidistant bins of stimulus intensities (in standard deviation units of the psychometric function) separately for the unimodal (A) and cross-modal (B) condition. Data from the tactile task is shown in orange, the visual task in blue. Open squares (chosen trial set) and filled squares (unsorted trial set) depict mean response times across observers, with error bars showing 95% CIs. Dashed lines (chosen trial set) and solid lines (unsorted trial set) show the average response time in each bin as predicted by our RT model.

Response times for the confidence judgments were slightly faster in the cross-modal conditions, M = 527.91 ± 66.02 ms, as compared to the unimodal conditions, M = 594.63 ± 68.39 ms, t(53) = 2.24, p = 0.03, d = 0.30. 

Cognitive resources and confidence efficiency

Since we observed substantial variability in confidence efficiency—especially in the cross-modal conditions (see Figure 5)—we were interested in exploring the role of individual differences in cognitive control resources that could drive this variability. Correlational analyses yielded no evidence for a link between CMIs and EF scores in our data, irrespective of modality and comparison condition (all ps > .19). 

Given that the procedure of the confidence forced-choice paradigm might draw on memory resources, we additionally aimed to rule out that CMIs were compromised by these task demands. We explored the association between individual differences in short-term memory capacity and CMIs, but we found consistently across both modalities and comparison conditions no significant correlations (all ps > .36). 

Discussion

In this study, we investigated whether confidence serves as a common currency across the visual and tactile senses. A common currency would suggest that confidence for different perceptual decisions is stored in an abstract, modality-independent format, allowing for quick and efficient confidence judgments across different tasks (de Gardelle et al., 2016; de Gardelle & Mamassian, 2014). Findings from a correlational study suggested that visual and tactile confidence underlie a common mechanism (Faivre et al., 2018). However, this idea has not been directly tested, yet. When visual and tactile information compete, an observer's belief that the tactile sense provides more certainty and directness could compromise cross-modal confidence judgments (Deroy & Fairhurst, 2019; Fairhurst et al., 2018). Using the established confidence forced-choice paradigm (Mamassian, 2020; Mamassian & de Gardelle, 2021), we investigated visual and tactile confidence within and across modalities. Additionally, we characterized participants’ individual cognitive control capacities by a comprehensive score that captures key facets of executive function (Miyake & Friedman, 2012). We considered differences in processing dynamics (Baranski & Petrusic, 1994; Kiani et al., 2014), as well as cognitive control capacities (Klever et al., 2022) that might contribute to metacognitive performance. 

Our findings provide evidence for a common currency between the visual and tactile senses. We found that participants were able to evaluate the quality of their perceptual decisions within and across modalities appropriately, i.e. subjective confidence judgments and objective perceptual performance were related. Confidence was lower when perceptual decisions were incorrect and higher when they were correct. The efficiency of this link was comparable across modalities and, importantly, not compromised when confidence comparisons were made across modalities. We thereby extend previous research showing that observers can judge their confidence across different visual tasks (de Gardelle & Mamassian, 2014), as well as visual and auditory tasks (de Gardelle et al., 2016), without any loss in metacognitive ability relative to judgments within the same task. While previous research found that auditory confidence was lower than visual confidence (de Gardelle et al., 2016), our results indicate that visual and tactile confidence are comparable. Visual and tactile confidence might lie closer together since both senses are actively used to sample information on our surroundings (Findlay & Gilchrist, 2003; Gibson, 1962), making them relatively more important in everyday life. Because confidence and behavior are tightly connected (Desender et al., 2018), good confidence calibration for visual and tactile decisions might be particularly relevant for action control. In line with this action-based account would also be the involvement of higher-level action-specific components in confidence representation (Fleming et al., 2015). 

We observed a small, but significant bias toward the tactile modality when participants were asked to provide their confidence judgments in the cross-modal blocks (53.7%). This finding seems congruent with previous reports of an overconfidence in the tactile sense (cf., Fairhurst et al., 2018). Tactile overconfidence could be attributed to the belief that touch provides more directness and certainty (Deroy & Fairhurst, 2019). In an informal survey after completion of the experiment, most of our participants (48%) indicated that they had overall felt more confident about their tactile decisions, even though performance had been controlled by a staircase procedure. In contrast, higher confidence about visual decisions or no confidence differences at all were each reported by only 14 participants (26%). Interestingly, an overall tactile confidence bias (i.e., whether a participant reported a higher confidence about tactile decisions or not) was significantly linked to the observed tactile bias in confidence judgments in the cross-modal blocks (r(54) = 0.377, p < 0.01). It remains ambiguous whether participants just provided a valid summary of their confidence judgments during the experiment or whether a general confidence bias towards the tactile sense fueled confidence judgments in the cross-modal blocks. However, most critically, the bias did not affect participants’ ability to adequately compare their confidence across modalities (all ps > 0.33). 

Our findings show that observers can adequately evaluate their uncertainty underlying visual and tactile decisions on an abstract, modality-independent scale. Overall, cross-modal confidence judgments seem to be made with ease and are as efficient as unimodal confidence judgments. Even though given our study was only behavioral and so we can only speculate about underlying neural mechanisms, our results suggest that confidence processes rely on shared brain regions. These regions may be the ventromedial prefrontal cortex, dorsal anterior cingulate cortex/pre-supplementary motor area, and parietal cortex (Levy & Glimcher, 2012; Morales et al., 2018; Rouault et al., 2018). 

To evaluate whether cross-modal confidence judgments are costly in terms of processing time, we considered differences in perceptual, as well as confidence, response times between the unimodal and cross-modal blocks. We found that response times for the perceptual decisions were slightly increased in the cross-modal blocks. This effect has also been observed in previous studies applying the same paradigm to two visual tasks (de Gardelle & Mamassian, 2014), as well as a visual and an auditory task (de Gardelle et al., 2016). In general, unimodal blocks have the advantage that the perceptual task remains constant, requiring in particular no change of perceptual filters. Unimodal visual decisions in our setup had the additional advantage that attention was always directed at the screen. This might have led to even faster responses and could also explain why contrast sensitivity was higher for visual decisions within compared to across tasks (Spence, 2002). It is likely that lengthened perceptual response times in cross-modal blocks reflect task-switching costs (Kiesel et al., 2010) that are related to perceptual processes and can even occur when the switch is predictable (Rogers & Monsell, 1995). However, they could also indicate that confidence formation processes were altered in the cross-modal blocks. In support of this interpretation is the observation that confidence judgments were made faster across modalities than within. Given that perceptual response times are less informative in cross-modal blocks (i.e., less comparable between tasks), it might be possible that participants formed their confidence judgments earlier during cross-modal blocks. However, the exact timing of processes that contribute to the formation of confidence eludes examination. We suggest that confidence formation processes are altered in cross-modal blocks but are overall as efficient as within the same modality. 

Previous research suggested a conceptual and functional overlap between cognitive control and metacognition (Fernandez-Duque et al., 2000; Klever et al., 2022; Roebers, 2017; Rouault et al., 2022). Both concepts comprise aspects of monitoring and controlling one's decisions, as well as flexibly adapting behavior, making them particularly relevant in complex and challenging situations (Klever, Voudouris, Fiehler, & Billino, 2019; Miyake et al., 2000; Roebers, 2017). They are thought to rely on shared brain regions in the prefrontal cortex, which have been proposed to enable domain-general metacognitive processes (Rouault et al., 2022). In contrast to this rationale, we did not find a significant link between individual cognitive control resources and confidence efficiency in our present study. The absence of a correlation might indicate the ease of confidence judgments that do not draw substantial resources. However, given the previous evidence, we speculate that in our sample the variance in cognitive control resources might not have been sufficient to reveal an association. Indeed, our sample size was determined focusing on statistical power for detecting differences between unimodal and cross-modal confidence efficiency. In order to detect a correlation between cognitive control resources and confidence efficiency larger sample sizes might be needed to achieve appropriate statistical power. A power calculation shows that a sample size of 84 participants was needed to detect a moderate correlation with a power of 80% and an α-level of 0.05. Thus we might have failed to find a link because of a lack of statistical power. These considerations are anecdotally supported by an exploratory analysis we ran by pooling the current data set with data from our previous study on perceptual confidence using similar methods (Klever et al., 2022). Given a resulting sample size of 113 participants, we determined a significant moderate correlation between cognitive control resources and confidence efficiency (r(113) = 0.273, p < 0.01). In sum, we propose that a putative correlation between cognitive control resources and confidence efficiency awaits further clarification by appropriately powered studies for testing this hypothesis. Our current data do not allow for an appropriate conclusion. 

The existence of a common currency between the visual and tactile senses supports behavioral control in complex environments by using multisensory information efficiently. Especially in situations where no external feedback is available, it might be useful to distinguish relevant from irrelevant information and determine the tasks that should be prioritized (Aguilar-Lleyda et al., 2020; Desender et al., 2018). Furthermore, it might be helpful when judging the same attribute of an object (e.g., its size) using different senses. However, so far, the role of confidence has been mainly considered about unimodal perceptual decisions. How confidence shapes multisensory decisions remains to be explored (Deroy, Spence, & Noppeney, 2016). 

In everyday life, we usually do not make single perceptual decisions but rather multiple perceptual decisions with multiple confidence judgments over multiple stimuli. It has been shown that confidence in previous decisions “leaks” into our confidence estimates of a following task (Rahnev, Koizumi, McCurdy, D'Esposito, & Lau, 2015). This, in turn, allows us to make global confidence judgments that are helpful to predict our future performance (Lee, de Gardelle, & Mamassian, 2021). An open question for future research is whether confidence leak also occurs across modalities and whether global confidence judgments are possible for perceptual decisions involving different senses. A common currency across modalities could facilitate these processes. 

Conclusions

We conclude that visual and tactile confidence share a common scale. Observers can adequately distinguish good from bad perceptual decisions, no matter whether confidence judgments were made within the same modality or across modalities. Overall, cross-modal confidence judgments are as efficient as unimodal confidence judgments and can be made with ease, although the timing of confidence formation processes might be slightly altered between unimodal and cross-modal confidence judgments. Open questions for future research are how confidence contributes to multisensory decisions and whether global confidence judgments can be made across senses. 

Acknowledgments

The authors thank Sabine Margolf and Maren Buck for assistance with the data collection. Data are publicly available at the doi:10.5281/zenodo.7188133. 

Supported by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) – project number 222641018 – SFB/TRR 135 TP B5/A4. 

Commercial relationships: none 

Corresponding author: Lena Klever. 

Email: [e-mail]. 

Address: Experimental Psychology, Justus Liebig University Giessen, Otto-Behaghel-Str. 10F, 35394 Giessen, Germany.