August 2010
Volume 10, Issue 7
Vision Sciences Society Annual Meeting Abstract  |   August 2010
A computational model for material recognition
Author Affiliations
  • Lavanya Sharan
    Disney Research Pittsburgh
  • Ce Liu
    Microsoft Research New England
  • Ruth Rosenholtz
    Brain and Cognitive Sciences, Massachusetts Institute of Technology
  • Edward Adelson
    Brain and Cognitive Sciences, Massachusetts Institute of Technology
Journal of Vision August 2010, Vol.10, 987. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Lavanya Sharan, Ce Liu, Ruth Rosenholtz, Edward Adelson; A computational model for material recognition. Journal of Vision 2010;10(7):987.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

We have previously shown that observers can recognize high-level material categories (e.g. paper, fabric, plastic etc.) in complex, real world images even in 40 millisecond exposures (Sharan et al., VSS 2009). This rapid perception of materials is different from object or texture recognition, and is fairly robust to low-level image degradations such as blurring or contrast inversion. We now turn to computational models and ask if machines can mimic this human performance. Recent work has shown that simple image features based on luminance statistics (Sharan et al.. 2008), or based on 5x5 pixel patches (Varma and Zisserman, 2009) are sufficient for some texture and material recognition tasks. We tested state-of-art models based on these features on the stimuli that our observers viewed. The performance was poor (Categorization rate: Varma-Zisserman = 20%, observers = 90%, chance = 11%). Our stimuli, a diverse collection of photographs derived from, are undoubtedly more challenging than state-of-art benchmarks (Dana et al., 1999). We have developed a model that combines low and mid-level image features, based on color, texture, micro-geometry, outline shape and reflectance properties, in a Bayesian framework. This model achieves significant improvement over state-of-art on our stimuli (Categorization rate: 41%) though it lags human performance by a large margin. Individual features such as color (28%) or texture (37%) or outline shape (28%) are also useful. Interestingly, when we ask human observers to categorize materials based on these features alone (e.g. by converting our stimuli to line drawings that convey shape information, or scrambling them to emphasize textures), observer performance is similar to that of the model (20-35%). Taken together, our findings suggest that isolated cues (e.g. color or texture) or simple image features based on these cues, are not sufficient for real world material recognition.

Sharan, L. Liu, C. Rosenholtz, R. Adelson, E. (2010). A computational model for material recognition [Abstract]. Journal of Vision, 10(7):987, 987a,, doi:10.1167/10.7.987. [CrossRef]
 Disney, Microsoft, NTT Japan, NSF.

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.