September 2018
Volume 18, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2018
Using multiple optimization tasks to improve deep neural network models of higher ventral cortex
Author Affiliations
  • Chengxu Zhuang
    Department of Psychology, Stanford University
  • Daniel Yamins
    Department of Psychology, Stanford UniversityComputer Science Department, Stanford Neurosciences Institute, Stanford University
Journal of Vision September 2018, Vol.18, 905. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Chengxu Zhuang, Daniel Yamins; Using multiple optimization tasks to improve deep neural network models of higher ventral cortex. Journal of Vision 2018;18(10):905.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Recent goal-driven deep neural network (DNN) models of higher ventral visual cortex have leveraged the rich behavioral task of object recognition to impose powerful top-down constraints on network parameters. DNNs optimized to solve the multi-way object categorization in challenging real-world images have been shown to provide state-of-the-art predictions of neural responses in visual areas throughout the primate ventral pathway. Here, we show that such models can be improved by using a combination of multiple behaviorally realistic tasks as network optimization targets. Specifically, we optimized a DNN to simultaneously solve high level tasks including object categorization and scene classification, as well as intermediate visual tasks including depth estimation, normal map estimation and semantic segmentation. Task optimization was synergistic, in that performance levels for each task in the combined training were higher at a given number of training examples than for models trained on each task separately. Moreover, the model trained on the combined tasks provided improved ability to fit response patterns in neurons from both cortical areas V4 and IT. These results suggest that identifying a richer and more ecologically relevant variety of visual behaviors as network "goals" may lead to substantially improved understanding of the neural computations in the visual system.

Meeting abstract presented at VSS 2018


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.