Abstract
Both in perception and computer vision, there is a sometimes heated debate about view-based vs. model-based object recognition. From a computational point of view, both strands seem to have advantages and disadvantages. We explore the potential of a new type of object features, which self-adapt to the view at hand, in order to support the recognition of 3D objects from single views. The use of such features combines some of the advantages of both ‘schools’. As with model-based techniques, recognition is based on the consistent layout of the features. On the other hand, the features directly characterise the observed intensity or colour patterns, rather than referring to 3D parts of some kind. The features characterise local surface patches, the shape of which is determined on the basis of the local colour pattern itself. The shape adapts in such a way that the patch each time covers the same physical part of a surface, assuming that it is locally more or less planar. The crux of the matter is that these self-adaptive patches are extracted from a single image, without knowledge about other views. In order to match them, they are characterised by jointly geometrically and photometrically invariant characteristics. The extraction of the patch shape is also based on invariance theory, a basis they share with several of the model-based strategies. The object model consists of a set of reference views, in line with view-based strategies. The use of these features allows to cover the view sphere more sparsely with reference views. Each reference view can be matched to novel views under larger changes of viewpoint than any direct image comparison would allow. The viability of this approach will be shown on the basis of an implementation that takes real-world scenes as input. The method can withstand reasonable changes in illumination conditions, occlusion, and scene clutter.