October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
A Database of Painterly Material Depictions
Author Affiliations & Notes
  • Mitchell Van Zuijlen
    Delft University of Technology, Perceptual Intelligence lab
  • Hubert Lin
    Cornell University, Computer Science Department
  • Kavita Bala
    Cornell University, Computer Science Department
  • Sylvia C. Pont
    Delft University of Technology, Perceptual Intelligence lab
  • Maarten W.A. Wijntjes
    Delft University of Technology, Perceptual Intelligence lab
  • Footnotes
    Acknowledgements  This work is part of a VIDI program with project number 276-54-001, which is financed by the Netherlands Organization for Scientific Research (NWO). This work is also supported by an NSERC PGS-D.
Journal of Vision October 2020, Vol.20, 1127. doi:https://doi.org/10.1167/jov.20.11.1127
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Mitchell Van Zuijlen, Hubert Lin, Kavita Bala, Sylvia C. Pont, Maarten W.A. Wijntjes; A Database of Painterly Material Depictions. Journal of Vision 2020;20(11):1127. https://doi.org/10.1167/jov.20.11.1127.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Painters depict materials by utilizing an implicit knowledge of human material perception. To enable studies of this implicit knowledge, we created a database of paintings annotated with bounding boxes of various depicted materials (e.g., fabric, stone, wood, etc.). First, we collected a set of nearly 20K paintings (primarily Old Masters) from online open-access galleries of nine internationally renowned art institutions. For 14 material categories, we asked human annotators from Amazon Mechanical Turk (AMT) to indicate the presence of each material within each painting. On average, we find 6.4 unique materials per painting. Skin and fabric were the most frequently depicted materials, while ceramic and food were the least frequent. Additionally, we calculated material co-occurrence correlations which show that, for example, when skin is present within a painting, food is less likely to be depicted within the same painting and vice-versa (r = -.59). Next, we selected 15 skilled AMT annotators to annotate more than 100k bounding boxes of the 14 materials. This allowed us to identify the spatial location within paintings where materials are depicted. For example, we find that glass is more often depicted within the top-left quadrant of the painting, which could be related to art-historical literature noting that light is often depicted as originating from the top-left (perhaps through a glass window). In the next step, more detailed material labels will be obtained, for example fabrics will be refined as silk or cotton or wool, etc. This in-depth dataset of material depictions can enable various perceptual, computational and historical analyses that could enable a deeper understanding of material perception and depiction.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.