Purchase this article with an account.
Lauren Y. L. Cao, Stephanie Yung, Winnie Wang, Dirk B. Walther; How sound influences gaze when we watch movies. Journal of Vision 2020;20(11):850. doi: https://doi.org/10.1167/jov.20.11.850.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Movies tightly control their audience’s attention, eliciting higher levels of eye movement synchrony than unedited footage of natural scenes or amateur videos (Dorr et al. 2010; Hasson et al., 2008). Existing research on the influence of sound on visual attention during movie viewing has shown that viewers follow speech turn-taking more closely when hearing related sound (Coutrot & Guyader, 2014). Here, we seek to examine the degree to which sound contributes to the strong attentional synchrony elicited by Hollywood films.
We recorded eye movements for subjects watching commercial films paired with the movie’s original soundtrack (original sound condition), the soundtrack from another movie (alternate sound condition), or white noise (ambient sound condition). Ninety-two participants each viewed six movie clips, two in each sound condition. Using inter-subject correlation (ISC) analysis, we examined how sound influences participants’ eye movements. We calculated ISC between participants who watched the same movie paired with different soundtracks: original vs. original (ceiling), original vs. alternate, and original vs. ambient. We also computed the ISC between participants who watched different movies paired with the same soundtrack and the ISC of participants watching different movies paired with different soundtracks (floor). We calculated ISC using all eye movements made within an experimental block as well as time-resolved ISC using a sliding time-window. Additionally, we used multiple regression to identify the visual and auditory characteristics that elicit strong attentional synchrony.
We found that pairing movies with a non-original soundtrack lowers similarity in eye movements between viewers for all movies analyzed. Auditory characteristics also influence time resolved ISC. Overall, these findings show both sound and image factor in the control that movies exert over our gaze with visual information playing the chief role in guiding attention. Our results offer insight into how sound contributes to the attentional synchrony elicited by Hollywood films.
This PDF is available to Subscribers Only