September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Predicting Human Perception of Glossy Highlights using Neural Networks
Author Affiliations & Notes
  • Konrad E Prokott
    Justus-Liebig-Universität Giessen
  • Roland W Fleming
    Justus-Liebig-Universität Giessen
Journal of Vision September 2019, Vol.19, 297b. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Konrad E Prokott, Roland W Fleming; Predicting Human Perception of Glossy Highlights using Neural Networks. Journal of Vision 2019;19(10):297b.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Human observers easily distinguish glossy from matte materials. Glossy materials reflect their surroundings, and exhibit distinctive specular highlights. The importance of highlights for gloss perception has been demonstrated by their use for centuries in the visual arts, and by the observation that removing highlights from photographs leads to a matte surface appearance. However, the visual computations underlying gloss perception remain largely unsolved. Here, we investigated how the visual system identifies specular highlights in images. This is challenging, because a given bright spot in the image could be a surface texture marking, light source, caustic, or many other physical events. Somehow the visual system has to identify that the bright spot is due to specular reflection, and then propagate this interpretation to surface regions where there is no local evidence that the surface is glossy. To test participants’ ability of identifying highlights we showed them computer renderings of glossy textured surfaces. Participants were asked to judge whether a given location in the image was a highlight or a texture marking. The results indicate that participants are excellent at this task, but that there are occasional consistent errors. We then compared the observers’ judgements to several models, ranging from a simple intensity threshold to more complex neural networks trained to give pixel-wise output maps of the specular reflectance component of an image. Our results show that human responses can be well matched by a relatively shallow feed-forward convolution neural network. We then compared model predictions to human responses on more challenging images in which the highlights are shown in the wrong locations and orientations relative to the matte components. Investigating the internal representations of the best models reveals a number of image measurements that could be the basis of human judgments.

Acknowledgement: This work was funded by the DFG-funded Collaborative Research Center on “Cardinal Mechanisms of Perception” (SFB-TRR-135) and an ERC Consolidator Award (ERC-2015-CoG-682859: “SHAPE”) 

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.