August 2016
Volume 16, Issue 12
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2016
Mapping human visual representations by deep neural networks
Author Affiliations
  • Kandan Ramakrishnan
    Intelligent Sensory Information Systems, UvA, Netherlands
  • H. Steven Scholte
    Department of Psychology, Brain and Cognition, UvA, Netherlands
  • Arnold Smeulders
    Intelligent Sensory Information Systems, UvA, Netherlands
  • Sennay Ghebreab
    Intelligent Sensory Information Systems, UvA, Netherlands
Journal of Vision September 2016, Vol.16, 373. doi:10.1167/16.12.373
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Kandan Ramakrishnan, H. Steven Scholte, Arnold Smeulders, Sennay Ghebreab; Mapping human visual representations by deep neural networks. Journal of Vision 2016;16(12):373. doi: 10.1167/16.12.373.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

A number of recent studies have shown that deep neural networks (DNN) map to the human visual hierarchy. However, based on a large number of subjects and accounting for the correlations between DNN layers, we show that there is no one-to-one mapping of DNN layers to the human visual system. This suggests that the depth of DNN, which is also critical to its impressive performance in object recognition, has to be investigated for its role in explaining brain responses. On the basis of EEG data collected from a large set of natural images we analyzed different DNN architectures a 7 layer, 16 layer and a 22 layer DNN network using weibull distribution for the representations at each layer. We find that the DNN architectures reveal temporal dynamics of object recognition, with early layers driving responses earlier in time and higher layers driving the responses later in time. Surprisingly the layers from the different architectures explain brain responses to a similar degree. However, by combining the representations of the DNN layers we observe that in the higher brain areas we explain more brain activity. This suggests that the higher areas in the brain are composed of multiple non-linearities that are not captured by the individual DNN layers. Overall, while DNNs form a highly promising model to map the human visual hierarchy, the representations in the human brain go beyond the simple one-to-one mapping of the DNN layers to the human visual hierarchy.

Meeting abstract presented at VSS 2016


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.