August 2012
Volume 12, Issue 9
Free
Vision Sciences Society Annual Meeting Abstract  |   August 2012
Estimating material by estimating shape
Author Affiliations
  • Edward Adelson
    Computer Science and AI Lab, Massachusetts Institute of Technology\nDept. of Brain and Cognitive Sciences, Massachusetts Institute of Technology
  • Forrester Cole
    Computer Science and AI Lab, Massachusetts Institute of Technology
  • Phillip Isola
    Dept. of Brain and Cognitive Sciences, Massachusetts Institute of Technology
  • William Freeman
    Computer Science and AI Lab, Massachusetts Institute of Technology
  • Fredo Durand
    Computer Science and AI Lab, Massachusetts Institute of Technology
Journal of Vision August 2012, Vol.12, 946. doi:10.1167/12.9.946
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Edward Adelson, Forrester Cole, Phillip Isola, William Freeman, Fredo Durand; Estimating material by estimating shape. Journal of Vision 2012;12(9):946. doi: 10.1167/12.9.946.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

An object’s appearance depends on a complex interaction of shape, illumination, and material. Despite decades of work, it has proven difficult to build computational systems that estimate these components individually. For example, shape from shading systems are still mediocre even when the illumination and BRDF are precisely specified. At the same time, techniques for material classification in the absence of shape remain limited. We have found that it is possible to make material estimates and shape estimates as part of the same process. Our learning based system is trained on multiple shapes and materials. We have a collection of "blobby" 3D shapes and we render them with multiple "styles," where a style captures the combined effects of illumination and BRDF. For a given patch of surface normals (our choice of shape descriptor) there will be multiple renderings that have different appearances. We build a library of pairings between shape patches and image patches, and then try to solve the inverse problem: estimating the surface normals given the image. This works poorly for an isolated patch, because many things in the world could give rise to similar image patches. Things get much better when we impose consistency across the image, using belief propagation. It is possible to make estimates of material by pooling the votes of multiple patches taken individually. However, the performance improves when the voting is done with the patches that were selected to consistently estimate the shape. Thus, while the system is primarily designed to estimate shape, it automatically gives a good estimate of material as well.

Meeting abstract presented at VSS 2012

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×