August 2023
Volume 23, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2023
Intuitive physics guides visual tracking and working memory: The dynamics of neural processing in expectation violation
Author Affiliations
  • Halely Balaban
    Massachusetts Institute of Technology (MIT)
  • Kevin Smith
    Massachusetts Institute of Technology (MIT)
  • Joshua Tenenbaum
    Massachusetts Institute of Technology (MIT)
  • Tomer Ullman
Journal of Vision August 2023, Vol.23, 4903. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Halely Balaban, Kevin Smith, Joshua Tenenbaum, Tomer Ullman; Intuitive physics guides visual tracking and working memory: The dynamics of neural processing in expectation violation. Journal of Vision 2023;23(9):4903.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

People have strong expectations about the basic behavior of everyday objects (e.g., that they shouldn’t suddenly disappear), which guide our perception, actions, and predictions. Core physical knowledge is so basic it is present in very early infancy and in non-human animals, as demonstrated in numerous behavioral experiments relying on surprise in the face of physically-impossible events. Yet, the online dynamics of processing physically-impossible events, leading to surprise and following it, remain hidden. To uncover them, we leveraged neural markers of online processing, by embedding short videos of physically-possible or physically-impossible scenes in a visual working memory task. In two EEG experiments, we measured the contralateral delay activity (CDA) as the scene unfolded, to examine whether violations of physical-expectations (a) disrupt the ability to smoothly track objects in the scene, and (b) later modify the scene’s interpretation. Our stimuli were modeled after those used in behavioral studies with adults and infants, and more recently AI benchmarks. In Experiment 1, objects crossed a stage, moving behind an occluder which briefly hid them. The animations either proceeded as expected, or included violations of object-permanence, in which an object disappeared or appeared. We found that the violations disrupted object tracking, causing working memory to reset before it could represent the new and correct number of objects in the scene. In Experiment 2, the stimuli were minimally changed to include a hole behind the occluder, meaning that appearance and disappearance could be explained away. Despite perceptual input during the critical period being identical to Experiment 1, the changes in high-level expectations translated to uninterrupted object tracking in Experiment 2, and even allowed participants to continue representing the object that disappeared ‘down the hole’. Our results suggest that intuitive physics expectations shape online representations, and are critical for the ability to track objects in dynamic scenes.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.