Purchase this article with an account.
Yaniv Morgenstern, Filipp Schmidt, Frieder Hartmann, Henning Tiedemann, Eugen Prokott, Guido Maiello, Roland Fleming; An image computable model of visual shape similarity. Journal of Vision 2019;19(10):37c. doi: https://doi.org/10.1167/19.10.37c.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Shape is one of the most important sources of visual information about objects. When presented with sets of unfamiliar objects, we can usually judge how similar or different they are from one another. Yet there are many possible features that could provide the basis for such judgments. How does the visual system compute similarity relationships between shapes? Here, we developed and tested a model of human visual shape similarity (‘Shape-Comp’), based on 94 image-computable shape features (e.g., area, compactness, shape context, Fourier descriptors). To test the model, we trained Generalized Adversarial Networks (GANs) on thousands of silhouettes of animals. Drawing samples from the latent space learned by the network allows us to synthesize novel 2D shapes that are related to one another in systematic ways. We created 8 sets of 24 shapes of novel naturalistic shapes. Using a multi-arrangement method (Kreigeskorte and Mur, 2012), observers dragged shapes into spatial configurations in which distances represent perceived shape similarity within each set. Representational Similarity Analysis revealed that human shape similarity was highly correlated with feature distance in the GAN’s latent space. We then compared human shape similarity judgements to the predictions of ShapeComp, where the weights of the model’s features were established by fitting on a training set of human judgements. In untrained test sets, the results reveal that ShapeComp accounts for most of the variance in human shape similarity. Using carefully selected stimulus sets, we can tease apart the relative importance of different features in the model, and evaluate the successes and failures of ShapeComp in predicting a host of phenomena related to shape, materials and visual appearance more generally. Together, these findings show that ShapeComp is a powerful tool for investigating the representation of shape and object categories in the human visual system.
This PDF is available to Subscribers Only