August 2014
Volume 14, Issue 10
Vision Sciences Society Annual Meeting Abstract  |   August 2014
Which pieces anchor the Shape-from-Shading puzzle and how they fit together
Author Affiliations
  • Benjamin Kunsberg
    Applied Mathematics Program, Yale University
  • Roland Fleming
    Department of Psychology, Justus Liebig University Giessen
  • Steven Zucker
    Department of Computer Science, Yale University
Journal of Vision August 2014, Vol.14, 722. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Benjamin Kunsberg, Roland Fleming, Steven Zucker; Which pieces anchor the Shape-from-Shading puzzle and how they fit together. Journal of Vision 2014;14(10):722.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Although Shape-from-Shading (SFS) has been solved in unrealistic cases, there is a schism between computer vision techniques and neurobiological mechanisms. There is no neurobiological evidence for light-source representation in the early visual system, yet every SFS algorithm requires or estimates the light source position(s) prior to reconstruction of the surface. This is also contrary to psychophysical evidence, which shows accurate shape perception even with certain conflicting light sources or "weird shading," provided the shading flow remains stable. Thus, we investigate global SFS reconstruction using the shading flow and without knowledge of the light source. We derive mathematical results proving that the map relating shading to surface reduces in complexity at critical points of intensity, and these define pieces on which a global solution can be anchored. Psychophysical evidence illustrates this fact: shading is more important along the intensity critical points than on generic points. By applying light source invariant shading equations (presented previously at VSS) to critical points of image intensity, we are able to calculate the surface curvatures in select open neighborhoods (pieces) of the surface. However, the 3D orientation of each of these patches remains unconstrained. We thus derive compatibility equations based on the transport of the surface normal using the calculated curvatures. Under a relaxation-labeling scheme, we are able to select the correct orientation for each of the patches; thus we obtain a 3D proto-shape. This approximates the global solution and exhibits appropriate light-source invariance. Finally, we return to neuroscience. Our model is realized with a network of neurons tuned to different 3D orientations and curvatures. Recent work from Ed Connor's lab has found evidence for neurons of this kind in V4/IT. It is pleasing that the mathematics of shading inference mirrors the biology of connections.

Meeting abstract presented at VSS 2014


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.