September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
An image computable model of visual shape similarity
Author Affiliations & Notes
  • Yaniv Morgenstern
    Psychology, Justus-Liebig-Universität Gießen
  • Filipp Schmidt
    Psychology, Justus-Liebig-Universität Gießen
  • Frieder Hartmann
    Psychology, Justus-Liebig-Universität Gießen
  • Henning Tiedemann
    Psychology, Justus-Liebig-Universität Gießen
  • Eugen Prokott
    Psychology, Justus-Liebig-Universität Gießen
  • Guido Maiello
    Psychology, Justus-Liebig-Universität Gießen
  • Roland Fleming
    Psychology, Justus-Liebig-Universität Gießen
Journal of Vision September 2019, Vol.19, 37c. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Yaniv Morgenstern, Filipp Schmidt, Frieder Hartmann, Henning Tiedemann, Eugen Prokott, Guido Maiello, Roland Fleming; An image computable model of visual shape similarity. Journal of Vision 2019;19(10):37c.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Shape is one of the most important sources of visual information about objects. When presented with sets of unfamiliar objects, we can usually judge how similar or different they are from one another. Yet there are many possible features that could provide the basis for such judgments. How does the visual system compute similarity relationships between shapes? Here, we developed and tested a model of human visual shape similarity (‘Shape-Comp’), based on 94 image-computable shape features (e.g., area, compactness, shape context, Fourier descriptors). To test the model, we trained Generalized Adversarial Networks (GANs) on thousands of silhouettes of animals. Drawing samples from the latent space learned by the network allows us to synthesize novel 2D shapes that are related to one another in systematic ways. We created 8 sets of 24 shapes of novel naturalistic shapes. Using a multi-arrangement method (Kreigeskorte and Mur, 2012), observers dragged shapes into spatial configurations in which distances represent perceived shape similarity within each set. Representational Similarity Analysis revealed that human shape similarity was highly correlated with feature distance in the GAN’s latent space. We then compared human shape similarity judgements to the predictions of ShapeComp, where the weights of the model’s features were established by fitting on a training set of human judgements. In untrained test sets, the results reveal that ShapeComp accounts for most of the variance in human shape similarity. Using carefully selected stimulus sets, we can tease apart the relative importance of different features in the model, and evaluate the successes and failures of ShapeComp in predicting a host of phenomena related to shape, materials and visual appearance more generally. Together, these findings show that ShapeComp is a powerful tool for investigating the representation of shape and object categories in the human visual system.

Acknowledgement: This research was supported by an ERC Consolidator Award (ERC-2015-CoG-682859: ‘SHAPE’) and by the DFG (SFB-TRR-135: ‘Cardinal Mechanisms of Perception’). Guido Maiello was supported by a Marie-Sklodowska-Curie Actions Individual Fellowship (H2020-MS-CA-IF-2017: ‘VisualGrasping’ Project ID: 793660). 

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.