September 2018
Volume 18, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2018
The Emergent Encoding of Human Interactions in the Brain
Author Affiliations
  • Jon Walbrin
    Bangor University, Wales
  • Kami Koldewyn
    Bangor University, Wales
Journal of Vision September 2018, Vol.18, 438. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jon Walbrin, Kami Koldewyn; The Emergent Encoding of Human Interactions in the Brain. Journal of Vision 2018;18(10):438.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Recent evidence suggests that static human interactions are encoded holistically, rather than in a part-by-part manner (Baldassano et al., 2017; Ding et al., 2017; Papeo et al., 2017). Another recent study demonstrates that a region of the posterior superior temporal sulcus (pSTS) is selective for social interaction perception (Isik et al., 2017). Interestingly, this region is proximal to regions of cortex shown to play a key role in visual feature integration (Pollmann, et al., 2014). We devised a novel fMRI experiment to determine if the pSTS demonstrates an 'emergent' response to dynamic human-human interactions. Participants viewed brief videos of interactions between two actors engaging in 3 types of interaction (arguing, celebrating, laughing), as well as 'solo' versions of these stimuli (i.e. the same videos, but with one of the actors removed). Using a similar approach to Baldassano et al. (2017), we trained a classification algorithm to differentiate fMRI voxel patterns underlying the perception of each interaction type, then tested performance on untrained interaction patterns and, crucially, on pattern averages evoked by solo videos (i.e. training on interaction patterns, testing on patterns averaged from pairs of corresponding solo stimuli). We hypothesized that multiple regions might significantly classify interactions, but that any region demonstrating emergent encoding of interactive behaviour (e.g. pSTS) would perform significantly worse when cross-classifying to solo pattern averages. We tested this approach in multiple (independently localized) social brain regions. Along with the pSTS, the extrastriate body area and STS face area classified interactions significantly above chance. Crucially, however, the pSTS was the only region to show an emergent encoding effect (i.e. significantly poorer cross-classification of solo pattern averages, compared to interaction classification). These findings provide strong evidence for the role that the pSTS plays in the integration of dynamic human information that is central to interaction perception.

Meeting abstract presented at VSS 2018


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.