Abstract
Visual segmentation is a core function of biological vision, key to adaptive behavior in
complex environments. Early models inspired by the feedforward processing in the
visual system described texture-based human segmentation as a comparison of the
summary statistics of low-level image features across space. Here we consider the
alternative view that, due to image ambiguity and sensory noise, perceptual
segmentation requires probabilistic inference.
To test this hypothesis, we develop a novel paradigm to measure perceptual
segmentation maps and their variability. We use composite textures: each segment
is characterized by a different distribution of oriented features. Participants briefly
view an image followed by two spatial cues and report whether the cued locations
belong to the same segment. We repeat the sequence with different locations and
reconstruct the full segmentation map from the binary choices, solving a system of
equations. In a second set of experiments, we manipulate uncertainty by controlling
the overlap between feature distributions and smoothing the texture boundary and
measure texture discrimination performance.
We find that segmentation maps are similar across observers but variable:
perceptual variability correlates with intrinsic image uncertainty, and both are higher
near segment boundaries. We then test the inference model that consists in
assigning pixels to segments by evaluating which distribution explains best the
observed features. Quantitative model comparison shows that perceptual variability
reflects image uncertainty beyond sensory noise and that human segmentation is
better explained by optimal probabilistic inference than by comparing summary
statistics. Lastly, we find an interaction between the effects of contour uncertainty
and feature distribution overlap.
These results support the probabilistic inference hypothesis and suggest extending
the model with contour specific components. Our work provides a normative
explanation of human perceptual segmentation as probabilistic inference and
demonstrates a novel framework to study perceptual segmentation, which could be
extended to natural images.