September 2015
Volume 15, Issue 12
Free
Vision Sciences Society Annual Meeting Abstract  |   September 2015
Convolutional Neural Networks in the Brain: an fMRI study
Author Affiliations
  • Kandan Ramakrishnan
    Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam, Amsterdam, Netherlands
  • Steven Scholte
    Department of Psychology, Cognitive Neuroscience Group, University of Amsterdam, Amsterdam, Netherlands
  • Victor Lamme
    Department of Psychology, Cognitive Neuroscience Group, University of Amsterdam, Amsterdam, Netherlands
  • Arnold Smeulders
    Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam, Amsterdam, Netherlands
  • Sennay Ghebreab
    Intelligent Systems Lab Amsterdam, Institute of Informatics, University of Amsterdam, Amsterdam, Netherlands Department of Psychology, Cognitive Neuroscience Group, University of Amsterdam, Amsterdam, Netherlands
Journal of Vision September 2015, Vol.15, 371. doi:10.1167/15.12.371
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Kandan Ramakrishnan, Steven Scholte, Victor Lamme, Arnold Smeulders, Sennay Ghebreab; Convolutional Neural Networks in the Brain: an fMRI study. Journal of Vision 2015;15(12):371. doi: 10.1167/15.12.371.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Biologically inspired computational models replicate the hierarchical visual processing in the human ventral stream. One such recent model, Convolutional Neural Network (CNN) has achieved state of the art performance on automatic visual recognition tasks. The CNN architecture contains successive layers of convolution and pooling, and resembles the simple and complex cell hierarchy as proposed by Hubel and Wiesel. This makes it a candidate model to test against the human brain. In this study we look at 1) where in the brain different layers of the CNN account for brain responses, and 2) how the CNN network compares against existing and widely used hierarchical vision models such as Bag-of-Words (BoW) and HMAX. fMRI brain activity of 20 subjects obtained while viewing a short video clip was analyzed voxel-wise using a distance-based variation partitioning method. Variation partitioning was done on successive CNN layers to determine the unique contribution of each layers in explaining fMRI brain activity. We observe that each of the 7 different layers of CNN accounts for brain activity consistently across subjects in areas known to be involved in visual processing. In addition, we find a relation between the visual processing hierarchy in the brain and the 7 CNN layers: visual areas such as V1, V2 and V3 are sensitive to lower layers of the CNN while areas such as LO, TO and PPA are sensitive to higher layers. The comparison of CNN with HMAX and BoW furthermore shows that while all three models explain brain activity in early visual areas, the CNN additionally explains brain activity deeper in the brain. Overall, our results suggest that Convolutional Neural Networks provide a suitable computational basis for visual processing in the brain, allowing to decode feed-forward representations in the visual brain.

Meeting abstract presented at VSS 2015

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×