Viewpoint adaptation reveals potential representational differences between 2D images and 3D objects

Abstract

For convenience and experimental control, most cognitive research has studied visual processing using 2D images instead of real (tangible) stimuli or 3D simulations. Recent evidence has shown that real objects differ from images, particularly when stimuli are relevant for actions. An open question is whether 3D features affect perceptual and semantic processing. By one view, people and artificial neural networks can accurately perform perceptual tasks such as object recognition well from purely 2D images using only monocular cues, so 3D is non-essential. By another view, if recognition relies on 3D representations of object form, it may be affected by richer cues to 3D structure, particularly from binocular disparity. In two experiments, we examined whether viewpoint representations of a face and a symmetric manipulable object (a kettle) were affected by the nature of the stimulus. In the first experiment, we compared 2D stimuli (images presented on a computer monitor) to real stimuli (a mannequin head or a physical kettle). In the second experiment, we compared 2D stimuli to 3D stimuli (rendered stereoscopically with binocular disparity). Participants adapted to a stimulus oriented leftward or rightward and then made psychophysical judgements about test stimuli with different orientations. Across both experiments, for both faces and kettles, adaptation to real or 3D stimuli induced stronger aftereffects than adaptation to 2D images, but only when the adapting orientation was rightward. A computational model found that broader tuning curves for 3D than 2D stimuli predicted larger aftereffects for 3D adaptation. Taken together, 3D stimuli (both simulated 3D and tangibly 3D) appear to evoke broader orientation tuning than 2D pictures, resulting in better viewpoint invariance. The specificity of the effect for rightward adaptation may be explained by known asymmetries in face (and perhaps object) processing.