Abstract
Viewing a dynamic audiovisual scene has inherent challenges for where and when gaze is allocated because of the competing and transient sensory information. The applied craft of film production has developed intuitive solutions for guiding viewers' gaze through visual and sound editing techniques, for example sound designers believe that increasing the loudness of dialogue relative to background ambient noise orients a viewers' attention to the speaking character. A fundamental assumption of these techniques is that a viewer's gaze is attracted to audiovisual elements in a scene and inversely less attracted to visual events without sound. Empirical evidence of viewing behaviour to dynamic scenes has predominantly focused on visual features, the role of sound as an influence on viewers' gaze is less clear. This study utilised a found experiment, Mike Figgis's experimental feature film, Timecode (2000) which contains four continuous perspectives of interrelated events displayed using a 2x2 split-screen, where each quadrant has an isolatable sound mix. We investigated the influence of sound on gaze behaviour to a 4 minute 50 second excerpt by manipulating the presence of sound across the four quadrants one at a time with abrupt sound cuts shifting sound 16 times (each quadrant represented four times). Forty-eight participants free-viewed the clip whilst being eye-tracked (sound order was counterbalanced across participants). Sound representation to a quadrant significantly increased the proportion of gaze to that region. Gaze was also influenced by time, as later sound representations of a quadrant had a significantly higher proportion of gaze than earlier ones. Fixation durations to sound regions were significantly longer than those to visual only quadrants. The auditory and visual salience values are also considered as predictors of gaze between the quadrants. These preliminary results suggest that dynamic scene viewing behaviour is significantly influenced by the inclusion of corresponding sound.
Meeting abstract presented at VSS 2017