Abstract
Our ability to visually discriminate, categorize, recognize and compare materials is crucial for many tasks. Given the enormous variety of materials and variability in their appearance, what visual information do we rely on for distinguishing and comparing them? Here, we sought to uncover the major dimensions in our mental representation of materials, using a large-scale data-driven approach. First, we identified 200 diverse material classes sampled systematically from nouns in the American English language and collected three high-quality photographs for each class. Next, we used crowdsourcing to collect >1.5 million judgments asking which of two materials randomly chosen from the set was more similar to a third reference material, where the unchosen material acted as a context to highlight the relevant dimensions shared by the other two. We described each material image as a sparse, non-negative vector in a multidimensional representational space, modeled the assumed cognitive process for predicting choices, and iteratively used the difference between predicted choice probability and actual choice to adapt the dimensions’ weights. The resulting model predicted material similarity judgments in an independent test set with >90% accuracy relative to the human noise ceiling and allowed us to accurately construct the similarity matrix of all 600 images. Similar to recent findings in the visual perception of objects, individual material images can be described as a combination of a small number of 36 material dimensions. These dimensions are highly reproducible and interpretable, comprising material categories, color and texture attributes, and more complex (e.g., mechanical) properties. Our computational model and resulting dimensions have broad application for studying material perception and its natural dimensions, such as predicting context-specific similarities, providing perceptual scales along which to rate new materials, testing the validity of ratings of material properties commonly used in the visual perception literature, and comparing behavioral representations to cortical representations.