December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Object-classifying neural networks have animate and inanimate feature subspaces with partially distinct representational geometries
Author Affiliations
  • Daniel Janini
    Harvard University
Journal of Vision December 2022, Vol.22, 4459. doi:https://doi.org/10.1167/jov.22.14.4459
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Daniel Janini; Object-classifying neural networks have animate and inanimate feature subspaces with partially distinct representational geometries. Journal of Vision 2022;22(14):4459. https://doi.org/10.1167/jov.22.14.4459.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The human visual system has different features preferentially responsive to animate versus inanimate objects. A range of possibilities exist for how these features represent entities within each domain. At one extreme, animate vs inanimate features have completely unique representational geometries specialized for entities within their domain. At the other extreme, these two sets of features have the exact same representational geometries. Here, we investigated these possibilities in object-classifying neural networks. We found that a large proportion of the features in each layer of AlexNet selectively respond to either animate (17-45%) or inanimate objects (5-21%). Next, we compared the within-domain representational geometries of the animate vs inanimate feature subspaces. Across the layers of AlexNet, the two feature subspaces differed widely in the correlation of their representational geometries (r = -0.11-0.68). However, in every layer, the two subspaces were less correlated than randomly selected subspaces of the same size. This was true at the category-level (e.g., cow vs donkey) and the exemplar-level (e.g., one cow vs another). To compare these subspaces to human perception, we measured the pairwise visual similarity of 72 animal categories in a visual search experiment (n = 248). Instead of finding a unique correspondence between the animate subspace and animal perception, the animate- and inanimate-selective subspaces both exhibited moderate correlations with the visual similarity of animals (Spearman rho ranges: 0.43-0.59 and 0.27-0.65, noise ceiling: rho = 0.91). Thus, AlexNet has animate- and inanimate-selective subspaces with partially distinct representational geometries. However, AlexNet’s animate subspace at best exhibited a moderate and non-specific correspondence to animal perception in humans. This pattern of results may arise because AlexNet learns different visual features than humans and/or because the human visual system does not contain unique representational geometries for animate and inanimate objects. Future neuroimaging work will attempt to tease apart these possibilities.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×