Abstract
Purposes: Does the addition of audio enhance visual perception and memory performance within a virtual environment (VE)? Can ambient sounds boost subjective 3D quality of the visual display? Can they facilitate a sense of “being there”? Can they enhance recall and recognition of visual objects and their locations?
Method: To answer these questions 60 students, with normal vision and hearing, were tested within a VE consisting of four rooms. Each room had a different wall color with furnishings that included a bookcase filled with outline pictures of objects from different categories (Snodgrass & Vanderwart, 1980). Subjects either perused each room for 3 minutes in silence or while listening to a unique low- or high-fidelity sound associated with the room (e.g., thunderstorm in yellow room). Afterward, they filled out a questionnaire form, and then recalled the objects seen in each room. Finally, in a forced-choice recognition test they matched each object to a specific room.
Results and Discussion: Ambient sounds significantly increased the subjective 3D quality of a visual display as well as enhancing the sense of “being there.” Adding audio can effectively compensate for low visual fidelity of typical VEs used in psychotherapy and entertainment, with a smaller increment in bandwidth than upgrading the visual display's temporal and spatial resolutions. Moreover, high-fidelity audio resulted in better recall and recognition performance than low-fidelity or no audio conditions. The multi-modal display creates a richer perceptual environment to anchor objects to specific locations within the environment, thus enhancing recognition and recall.
Supported by the National Science Foundation