August 2016
Volume 16, Issue 12
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2016
Deep Neural Networks explain spatio-temporal dynamics of visual scene and object processing
Author Affiliations
  • Radoslaw Martin Cichy
    Department of Psychology and Education, Free University Berlin, Berlin, Germany
  • Aditya Khosla
    CSAIL, MIT, MA, USA
  • Dimitrios Pantazis
    McGovern Institute of Brain and Cognitive Sciences, MIT, MA, USA
  • Antonio Torralba
    CSAIL, MIT, MA, USA
  • Aude Oliva
    CSAIL, MIT, MA, USA
Journal of Vision September 2016, Vol.16, 371. doi:10.1167/16.12.371
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, Aude Oliva; Deep Neural Networks explain spatio-temporal dynamics of visual scene and object processing. Journal of Vision 2016;16(12):371. doi: 10.1167/16.12.371.

      Download citation file:


      © 2017 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements
Abstract

Understanding visual cognition means knowing where and when what is happening in the brain when we see. To address these questions in a common framework we combined deep neural networks (DNNs) with fMRI and MEG by representational similarity analysis. We will present results from two studies. The first study investigated the spatio-temporal neural dynamics during visual object recognition. Combining DNNs with fMRI, we showed that DNNs predicted a spatial hierarchy of visual representations in both the ventral, and the dorsal visual stream. Combining DNNs with MEG, we showed that DNNs predicted a temporal hierarchy with which visual representations emerged. This indicates that 1) DNNs predict the hierarchy of visual brain dynamics in space and time, and 2) provide novel evidence for object representations in parietal cortex. The second study investigated how abstract visual properties, such as scene size, emerge in the human brain in time. First, we identified an electrophysiological marker of scene size processing using MEG. Then, to explain how scene size representations might emerge in the brain, we trained a DNN on scene categorization. Representations of scene size emerged naturally in the DNN without it ever being trained to do so, and DNN accounted for scene size representations in the human brain. This indicates 1) that DNNs are a promising model for the emergence of abstract visual properties representations in the human brain, and 2) gives rise to the idea that the cortical architecture in human visual cortex is the result of task constraints imposed by visual tasks.

Meeting abstract presented at VSS 2016

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×