Purchase this article with an account.
Benjamin Kunsberg, Roland Fleming, Steven Zucker; Which pieces anchor the Shape-from-Shading puzzle and how they fit together. Journal of Vision 2014;14(10):722. doi: https://doi.org/10.1167/14.10.722.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Although Shape-from-Shading (SFS) has been solved in unrealistic cases, there is a schism between computer vision techniques and neurobiological mechanisms. There is no neurobiological evidence for light-source representation in the early visual system, yet every SFS algorithm requires or estimates the light source position(s) prior to reconstruction of the surface. This is also contrary to psychophysical evidence, which shows accurate shape perception even with certain conflicting light sources or "weird shading," provided the shading flow remains stable. Thus, we investigate global SFS reconstruction using the shading flow and without knowledge of the light source. We derive mathematical results proving that the map relating shading to surface reduces in complexity at critical points of intensity, and these define pieces on which a global solution can be anchored. Psychophysical evidence illustrates this fact: shading is more important along the intensity critical points than on generic points. By applying light source invariant shading equations (presented previously at VSS) to critical points of image intensity, we are able to calculate the surface curvatures in select open neighborhoods (pieces) of the surface. However, the 3D orientation of each of these patches remains unconstrained. We thus derive compatibility equations based on the transport of the surface normal using the calculated curvatures. Under a relaxation-labeling scheme, we are able to select the correct orientation for each of the patches; thus we obtain a 3D proto-shape. This approximates the global solution and exhibits appropriate light-source invariance. Finally, we return to neuroscience. Our model is realized with a network of neurons tuned to different 3D orientations and curvatures. Recent work from Ed Connor's lab has found evidence for neurons of this kind in V4/IT. It is pleasing that the mathematics of shading inference mirrors the biology of connections.
Meeting abstract presented at VSS 2014
This PDF is available to Subscribers Only