Abstract
A fundamental question in visual neuroscience is how to represent image structure. The most commonly used representation schemes rely on spatially localized differential operators, approximated as Gabor filters with a set of excitatory and inhibitory lobes, which compare adjacent regions of an image. While well-suited to encoding local relationships, such operators have significant drawbacks. Specifically, they confound a filter's inter-lobe distance with the size of the lobes themselves. Thus, to make comparisons across large image distances, one must use filters with larger lobes, implying spatial averaging over extremely broad areas. This makes it difficult to directly compare small regions across large distances. In order to address this problem, we introduce the “dissociated dipole” or “sticks” operator, for performing non-local comparisons within an image. This operator de-couples lobe size from inter-lobe distance and enables parametric movement between edge-based and region-based representation modes.
Here we report on two aspects of sticks. First, we assess perceptual plausibility of the operator via psychophysical experiments that test observers' ability to compare the brightness of small target regions across large image distances. Our results suggest that subjects' thresholds are remarkably robust even over large separations (∼15 degrees of visual angle) of the targets along horizontal and vertical axes. Second, to evaluate the effectiveness of this approach for image encoding, we have implemented a system that uses sticks operators for content-based image retrieval. We have obtained good results across a diverse set of domains including outdoor scenes, faces and letters. Furthermore, performance appears to be robust against significant image degradations such as poor spatial resolution and quantization of contrast levels. Based on these results, we believe that the sticks operator can serve as an effective scheme for representing image structure.