Abstract
To date cross-modal correspondences of stimuli from different sensory modalities have been widely demonstrated, although mechanisms that stand behind the phenomenon have not been fully established yet. In previous studies we investigated the role of evaluative meaning (semantic features that people impose to the external world, e.g. pleasant or interesting) in our experience of stimuli from different sensory modalities. The results showed that the elementary evaluative attributes we impose to stimuli from different modalities converge into three basic evaluative factors: affective evaluation (pleasant, positive, relaxing), arousal (impressive, powerful, interesting), and cognitive evaluation (clear, regular, meaningful) (Jankovic, 2000; Markovic, Jankovic, Subotic, 2002). We also demonstrated a significant similarity among obtained factorial structures across different sensory modalities, implying cross-modal nature of evaluative meaning. Following the establishment of a 3-D cross-modal semantic (evaluative) space, stimuli from different sensory modalities that have nothing in common (considering explicit, objective features) can now be subjected to the same metrics and compared according to their semantic similarity. In Experiment 1 respondents judged abstract visual patterns and pseudowords on the same set of evaluative attributes. By calculating the distance of stimuli in the 3-D evaluative space we have obtained mutual evaluative similarity of visual and verbal stimuli. In Experiment 2 respondents were asked to make explicit matches between abstract visual patterns and auditory presented pseudowords. The results showed that cross-modal correspondences are mostly predicted by evaluative similarity of visual and verbal stimuli. Affective evaluation appears to be the most important predictor, followed by arousal and cognitive evaluation. In conclusion we propose the model for prediction of cross-modal correspondences based on semantic (evaluative) similarity of stimuli.
This research was supported by Grant # 149039D from the Serbian Ministry of Science.