The Impact of Deafness on the Use of Information During Facial Emotion Discrimination

Abstract

The integration of visual and auditory cues facilitates the detection of emotions in our interactions with others. Sensory deprivation such as deafness can be compensated through sign language, lip reading and oral communication. The diagnostic features for the recognition of emotions expressed facially in relation to the preferred means of communication and other factors of deafness (e.g. CI, cochlear implant) still remain unknown. In the present study, we used the Bubbles technique (Gosselin & Schyns, 2001) to examine whether severe-to-profound bilateral deafness leads to visual differences in decoding specific facial information. Twenty-two deaf individuals (including oral deaf, signers, and CI users) matched in age and sex with 22 hearing controls completed 3,072 trials of a fear versus happy discrimination task. On each trial, a face image was randomly selected from a set of four (two happy), mirror-reversed with a probability of 0.5 and sampled using randomly located Gaussian apertures at five scales (see Adolphs et al., 2005). The number of Gaussian apertures was adjusted on each trial to maintain a 75% correct rate. Multiple linear regressions were performed on the location of the Gaussian apertures and accuracy score for each participant. The individual regression coefficient planes were combined within subject groups (oral, signers, CI users, controls). Finally, to reveal group differences in information use for the task at hand, we computed all pairwise contrasts between these group regression coefficient planes and we applied a statistical threshold (Chauvin et al., 2005). The deaf participants using sign language differed from the other groups by using mostly the eye region to discriminate emotions. Similar patterns with the mouth as the salient area were found for hearing controls, CI users and the oral deaf. Facial information processing is therefore influenced by auditory experience and the visual strategies used in communication.