Abstract
Our purpose was to understand how the combination of the visual stimulus (motion picture) and auditory stimulus (music and/or noise sound) determine the pleasantness. Also, we investigated whether the integration of the pleasantness from vision and audition could be described as a linear processing. In the experiments, we combined the motion picture [natural scene, road traffic scene, and dynamic noise pattern, presented by the use of a head mount display (23 arc deg × 30 arc deg)] and one or two from three types of sound [relaxing music, road traffic noise, and white noise, presented by the use of two loudspeakers in a sound proof room (40–45dB)]. Also, there were conditions in which each of the motion picture and sound was presented by itself. In each trial, ten naïve participants rated pleasantness for the stimulus that was presented for 30 sec. For the combination of a single picture and single sound, the pleasantness significantly increased with the increase of the pleasantness that each of the picture and sound in the combination gave the participants when each of them was presented by itself. For the combination of a single picture and two sounds, however, combination of less pleasant picture and sounds could be more pleasant than the combination of more pleasant ones. These results indicate that, although the integration of the pleasantness for the combinations of a picture and one sound would be described as a linear processing, that for the combination of a picture and two sounds would be a non-linear processing. We are proposing that the way to integrate the pleasantness from vision and audition varies with the number of sounds in the combination, and that the interaction of the pleasantness for picture and sound could be non-linear if multiple sounds are included in the combination. The determinants of the pleasantness for audio-visual materials will be discussed.