Purchase this article with an account.
RT Pramod, Michael Cohen, Kirsten Lydic, Josh Tenenbaum, Nancy Kanwisher; Evidence that the Brain’s Physics Engine Runs Forward Simulations of What will Happen Next. Journal of Vision 2020;20(11):1521. doi: https://doi.org/10.1167/jov.20.11.1521.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Human vision enables us not only to recognize what is where, but to understand the physical properties, relations and forces in a scene and use this information to predict what will happen next. Recent work suggests that these intuitive physical inferences are based on probabilistic simulations of a mental physics engine akin to the physics engines used in video games. Indeed, parietal and frontal regions have been implicated as the “brain’s physics engine”, as they are strongly engaged during intuitive physical inference, and they contain information about object mass. Here, we used fMRI to test the hypothesis that these brain regions conduct simulations of what will happen next. Specifically, we predicted a higher response in these regions for static images of real-world scenes that depict a) unstable configurations of objects or of people in precarious positions (expected to induce forward simulation) than b) stable configurations (where less simulation is expected). Six subjects fixated a cross through the experiment (verified via eye-tracking), and performed an orthogonal 1-back task on stimuli arranged in a blocked design. As predicted, we found significantly higher responses in independently-defined parietal “physics regions” when participants viewed unstable vs stable scenes (p=0.004 for a paired t-test across subjects). Moreover, similar effects were found in visual motion area MT, also consistent with greater simulation for unstable than stable stimuli. This increased response is unlikely to reflect differential eye movements, low-level stimulus differences (as stable versus unstable stimuli elicit equal responses in V1 and were not decodable in early layers of a CNN), or differential attention (as no increased response was found for animate rather than physical instability, e.g. a person being chased by a shark). These results suggest that “the brain’s physics engine” computes information about physical stability based on forward simulations of what will happen next.
This PDF is available to Subscribers Only