Abstract
The visual sensitivity (d’) of a picture can be enhanced crossmodally by the presentation of an auditory cue that is semantically congruent rather than incongruent. Nevertheless, whether the metacognitive sensitivity (meta-d’) of a picture processing (i.e., the ability to discriminate whether one’s own perceptual judgment is correct) can be modulated by crossmodal semantic congruency remains unclear. We examined this issue by measuring the d’ and meta-d’ in a picture detection task following an auditory cue, and their quotient (meta-d’/d’, called M-ratio) is an index of metacognitive efficiency which controls the influence of task difficulty. On each trial, either an object or a scrambled picture was presented briefly and sandwiched by two random-dot masks, and an auditory cue was presented before or simultaneously with the picture. Participants had to detect the presence of an object picture, and then to rate their confidence regarding the detection judgment. The auditory cue and the object picture were either congruent (e.g., a dog barking and a dog picture) or incongruent (e.g., a piano note and a dog picture). When a naturalistic sound was presented 350 ms before or simultaneously with the picture, and when a spoken word was presented 1000 ms before or simultaneously with the picture, the d’ and meta-d’ were higher in the congruent than in the incongruent condition. Interestingly, the M-ratio was higher in the congruent than in the incongruent condition only when the spoken word and the picture were presented simultaneously, while no such difference was observed in the other three conditions. Hence, hearing a semantically-congruent (as compared to incongruent) auditory cue can facilitate not only the d’, but also the meta-d’ of visual processing. Seeing an object while hearing its name is unique in that the visual processing is highly efficient from the perceptual up to the metacognitive level.
Acknowledgement: Ministry of Science and Technology in Taiwan (MOST 107-2410-H-715-001-MY2 and MOST 107-2410-H-002-129-MY3)