Abstract
Stereo occlusion boundaries separate foreground surfaces that are visible to both eyes from background surfaces that are visible to only one eye. Anderson and Nakayama (1994) hypothesize the existence of local units in early vision that are specialized to detect these boundaries. They observe that these units could sense where regions with binocular correlation are adjacent to regions without correlation, and they suggest that the units should cover a variety of boundary orientations and a variety of disparity changes across the boundary. In this work, we propose a computational implementation for these units. We introduce a taxonomy of local response patterns that can occur within a population of spatially-distributed, disparity-tuned neurons near an occlusion event. The items in the taxonomy are differentiated by the textures that exist on the surfaces adjacent to the occlusion boundary. We observe that there are distinctive local patterns that make stereo occlusion boundaries uniquely detectable in most cases, but that they can be confused with occlusion-less texture boundaries when the background surface has uniform intensity. This implies that any neurological or computational local detector of one must also detect the other. We argue that the local units should detect both types of boundaries since both cases occur at the correct spatial locations and disparities. We design a computational local detector, using a multi-scale feedforward neural network, that exploits the patterns of our taxonomy while also providing enough capacity to account for natural textural and orientation variations of boundaries. We find that our detector provides accurate boundaries for a variety of stereo images, including many well-known perceptual stimuli, realistically rendered images, and captured photographs. In many cases, it outperforms state-of-the-art computer vision stereo algorithms in finding stereo occlusion boundaries.
Acknowledgement: National Science Foundation Award IIS-1618227