Abstract
Although humans exhibit a substantial ability to interpret shape from shading variations, there exists no biologically plausible computation to explain it. Instead, in computer vision, shading analysis is normally formulated as a first-order non-linear partial differential equation that relates surface normal distributions to image intensities. The equation is difficult to solve in natural scenes, initial conditions and light source locations are difficult to obtain, and there is no connection to neurobiology.
In contrast, we are developing a new approach to shape from shading that is built directly upon the information available in visual cortex. Furthermore we solve for the surface and light sources together. We start with the shading flow–the cortical representation of projected surface isophotes– and supplement this with contrast variation.
We then introduce a novel mathematical formulation for calculating local surface shape in a manner that could be implemented by cortical computations. It is based on covariant derivatives, a shape operator based on curvatures, and yields a differential form that plays an analogous role to co-circularity in curve inference. Although far more complex, it could therefore be implemented by feedback or long-range horizontal connections in cortical columns.
The heart of the computation is the second fundamental form and how this relates to the shading flow. Assuming constant albedo, we are able to solve exactly for the light source/surface pairs needed for a local image patch to have a given shading flow. The magnitude of the brightness gradient restricts this family to a single light source and surface estimate for that image patch.
We observe properties regarding the interplay between the shading, light source, and surface parameters. In ambiguous cases, the lighting-from-above assumption corresponds to a surface convexity assumption. Also, for unknown ellipsoids, one can relate the angular changes in the flow to the ratio of principal axes.
Meeting abstract presented at VSS 2012