Abstract
The luminance and color of surfaces in natural scenes are relatively independent under certain linear transformations, with the luminance of a surface providing little information about the color of that surface, and vice versa. However, differences in luminance between two locations in a natural scene remain strongly associated with differences in color. These statistics for color and luminance differences are not easily captured by correlation statistics or by independent components analysis, but Bayesian statistics capture these dependencies well.
We used these spatio-chromatic statistics within natural scenes as the priors for a Bayesian model, with no free parameters, that decides whether or not two points within an image fall on the same surface. This model assumes that two pixels that fall on the same surface are likely to have the same luminance and color and two pixels that fall on different surfaces are likely to differ in both luminance and color. The performance of the model matched the segmentations made by observers, both using an image set based on natural scenes, and using a very different novel image set containing both natural and man-made environments. The generalization of our results to novel scenes is consistent with the notion that observers may use a fixed set of priors as a basis for image segmentation across different visual environments.
One common difficulty with using Bayesian models to predict behavior is that estimating observers' priors often requires ad hoc assumptions, or choosing those priors that best predict observers' performance. In this case the appropriate prior could be estimated from the spatio-chromatic structure of natural scenes without reference to human performance, allowing a less ambiguous test of whether the algorithm is a good model for human judgments.
Fine, I., MacLeod, D.I.A. & Boynton, G.M. (In press) Visual segmentation based on the luminance and chromaticity statistics of natural scenes. Bayesian and Statistical Approaches to Vision, JOSA.