September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Inducing a human-like shape bias leads to emergent human-level distortion robustness in CNNs
Author Affiliations & Notes
  • Robert Geirhos
    University of Tübingen
    International Max Planck Research School for Intelligent Systems
  • Patricia Rubisch
    University of Tübingen
    University of Edinburgh
  • Jonas Rauber
    University of Tübingen
    International Max Planck Research School for Intelligent Systems
  • Carlos R Medina Temme
    University of Tübingen
  • Claudio Michaelis
    University of Tübingen
    International Max Planck Research School for Intelligent Systems
  • Wieland Brendel
    University of Tübingen
  • Matthias Bethge
    University of Tübingen
    Bernstein Center for Computational Neuroscience Tübingen
    Max Planck Institute for Biological Cybernetics
  • Felix A Wichmann
    University of Tübingen
    Bernstein Center for Computational Neuroscience Tübingen
Journal of Vision September 2019, Vol.19, 209c. doi:https://doi.org/10.1167/19.10.209c
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Robert Geirhos, Patricia Rubisch, Jonas Rauber, Carlos R Medina Temme, Claudio Michaelis, Wieland Brendel, Matthias Bethge, Felix A Wichmann; Inducing a human-like shape bias leads to emergent human-level distortion robustness in CNNs. Journal of Vision 2019;19(10):209c. doi: https://doi.org/10.1167/19.10.209c.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Convolutional neural networks (CNNs) have been proposed as computational models for (rapid) human object recognition and the (feedforward-component) of the primate ventral stream. The usefulness of CNNs as such models obviously depends on the degree of similarity they share with human visual processing. Here we investigate two major differences between human vision and CNNs, first distortion robustness---CNNs fail to cope with novel, previously unseen distortions---and second texture bias---unlike humans, standard CNNs seem to primarily recognise objects by texture rather than shape. During our investigations we discovered an intriguing connection between the two: inducing a human-like shape bias in CNNs makes them inherently robust against many distortions. First we show that CNNs cope with novel distortions worse than humans even if many distortion-types are included in the training data. We hypothesised that the lack of generalisation in CNNs may lie in fundamentally different classification strategies: Humans primarily use object shape, whereas CNNs may rely more on (easily distorted) object texture. Thus in a second set of experiments we investigated the importance of texture vs. shape cues for human and CNN object recognition using a novel method to create texture-shape cue conflict stimuli. Our results, based on 49K human psychophysical trials and eight widely used CNNs, reveal that CNNs trained with typical “natural” images indeed depend much more on texture than on shape, a result in contrast to the recent literature claiming human-like object recognition in CNNs. However, both differences between humans and CNNs can be overcome: training CNNs on a suitable dataset induces a human-like shape bias. This resulted in an emerging human-level distortion robustness in CNNs. Taken together, our experiments highlight how key differences between human and machine vision can be harnessed to improve CNN robustness---and thus make them more similar to the human visual system---by inducing a human-like bias.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×