September 2021
Volume 21, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2021
Selective processing of social interactions during naturalistic movie viewing
Author Affiliations
  • Haemy Lee Masson
    Johns Hopkins University
  • Leyla Isik
    Johns Hopkins University
Journal of Vision September 2021, Vol.21, 1961. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Haemy Lee Masson, Leyla Isik; Selective processing of social interactions during naturalistic movie viewing. Journal of Vision 2021;21(9):1961.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The ability to recognize interactions between two or more people in a complex visual scene is a crucial skill that helps us navigate the social world. Previous studies have revealed that the brain selectively represents others’ social interactions in the posterior superior temporal sulcus (pSTS). These studies, however, have all relied on simple artificial stimuli. It is unclear to what extent social interaction selectivity exists in the real world where social interactions co-vary with many other sensory and social features (such as faces, voices, theory of mind). To address this, a few studies have adopted a naturalist movie viewing paradigm, but none have looked at the unique variance explained by social interactions beyond that explained by other covarying features in natural movies. The current study utilizes machine learning-based fMRI analyses and computer vision techniques to uncover the brain mechanisms uniquely underlying naturalistic social interaction perception. We analyzed two publicly available fMRI datasets, collected while participants watched two different commercial movies in the MRI scanner. By performing voxel-wise encoding and variance partitioning analyses, we demonstrate that a socio-affective model (a linear combination of an agent speaking, social interactions, theory of mind, perceived valence, and arousal) independently contribute to predicting brain responses in the STS and the dorsal medial prefrontal cortex. Importantly, the STS and the precuneus show unique selectivity for scenes that contain social interactions, even after the effects of all other visual and social features, including the presence of faces and theory of mind, have been controlled for. This selectivity generalized across both sets of movie data despite the fact that data came from different genres, subjects, and labs. Together, these findings suggest that social interaction perception recruits dedicated neural circuits during natural viewing and is an essential dimension of social understanding.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.