September 2021
Volume 21, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2021
Perception of soft materials relies on physics-based object representations: Behavioral and computational evidence
Author Affiliations & Notes
  • Wenyan Bi
    Yale University
  • Aalap D. Shah
    Yale University
  • Kimberly W. Wong
    Yale University
  • Brian Scholl
    Yale University
  • Ilker Yildirim
    Yale University
  • Footnotes
    Acknowledgements  This project was funded by ONR MURI #N00014-16-1-2007 awarded to BJS.
Journal of Vision September 2021, Vol.21, 2624. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Wenyan Bi, Aalap D. Shah, Kimberly W. Wong, Brian Scholl, Ilker Yildirim; Perception of soft materials relies on physics-based object representations: Behavioral and computational evidence. Journal of Vision 2021;21(9):2624. doi:

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

When encountering objects, we readily perceive not only low-level properties (e.g., color and orientation), but also seemingly higher-level ones -- some of which seem to involve aspects of physics (e.g., mass). Perhaps nowhere is this contrast more salient than in the perception of soft materials such as cloths: the dynamics of these objects (including how their three-dimensional forms vary) are determined by their physical properties such as stiffness, elasticity, and mass. Here we argue that the perception of cloths and their physical properties must involve not only image statistics, but also abstract object representations that incorporate "intuitive physics". We do so by exploring the ability to *generalize* across very different image statistics in both visual matching and computational modeling. Behaviorally, observers had to visually match the stiffness of animated cloths reacting to external forces and undergoing natural transformations (e.g. flapping in the wind, or falling onto the floor). Matching performance was robust despite massive variability in the lower-level image statistics (including those due to location and orientation perturbations) and the higher-level variability in both extrinsic scene forces (e.g., wind vs. rigid-body collision) and intrinsic cloth properties (e.g., mass). We then confirmed that this type of generalization can be explained by a computational model in which, given an input animation, cloth perception amounts to inverting a probabilistic physics-based simulation process. Only this model -- and neither the alternatives relying exclusively on simpler representations (e.g., dynamic image features such as velocity coherence) nor alternatives based on deep learning approaches -- was able to explain observed behavioral patterns. These behavioral and computational results suggest the perception of soft materials is governed by a form of "intuitive physics" -- an abstract, physics-based representation of approximate cloth mechanics that explains observed shape variations in terms of how unobservable properties determine cloth reaction to external forces.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.