August 2012
Volume 12, Issue 9
Vision Sciences Society Annual Meeting Abstract  |   August 2012
Intuitive mechanics in visual reasoning about complex scenes with unknown forces
Author Affiliations
  • Peter Battaglia
    BCS, MIT
  • Jessica Hamrick
    BCS, MIT\nComputer Science, MIT
  • Joshua Tenenbaum
    BCS, MIT\nComputer Science, MIT
Journal of Vision August 2012, Vol.12, 590. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Peter Battaglia, Jessica Hamrick, Joshua Tenenbaum; Intuitive mechanics in visual reasoning about complex scenes with unknown forces. Journal of Vision 2012;12(9):590. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

People have a powerful "physical intelligence" -- the capacity to infer physical properties of objects and predict future states in complex dynamical scenes -- which is central to how they interpret their environment and plan safe and effective actions. With little or no training and only a short visual presentation, people can easily judge subtle properties of scenes and anticipate future events, even when uncertain factors, like external forces, and more complex properties, like friction and physical connectedness, are involved. For instance, when you place a cup on your desk, you visually inspect the scene and identify locations likely to be robust to someone bumping it, factoring in surface friction and connections between objects. We hypothesize that the brain relies on a sophisticated system of knowledge about physics, i.e. an "intuitive mechanics", which applies to a wide variety of common arrangements of objects, and which allows it to make these rich inferences and plans.

We developed a probabilistic model to capture this hypothesis, which input visual stimuli containing arrangements of objects, and which predicted potential physical consequences under external forces, friction, and connectedness. To evaluate the model in scenarios like the desk example above, we presented participants with scenes depicting virtual tables with random stacks of red and yellow blocks, and asked them to judge which color would more likely fall off if an unknown bump were applied. In some cases we informed them, either verbally or visually, that some colors were connected (i.e. reds are attached to reds), or that some colors were subject to greater or lesser friction, and measured whether and how people's judgments responded to these manipulations. We compared people with our model and found good correspondence; in particular, people demonstrated accurate reasoning about potential external bumps and appropriate sensitivity to friction and connectedness properties.

Meeting abstract presented at VSS 2012


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.