September 2018
Volume 18, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2018
Modeling the scientist in the crib
Author Affiliations
  • Nick Haber
    Stanford University
  • Damian Mrowca
    Stanford University
  • Li Fei-Fei
    Stanford University
  • Daniel Yamins
    Stanford University
Journal of Vision September 2018, Vol.18, 10. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Nick Haber, Damian Mrowca, Li Fei-Fei, Daniel Yamins; Modeling the scientist in the crib. Journal of Vision 2018;18(10):10.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Deep convolutional neural networks, when trained on difficult supervised tasks such as object classification on large datasets, have shown the remarkable property of learning general representations useful for many other tasks and predictive of responses in the ventral visual stream. These successes have led to the pursuit of tasks that are able to generate a visual backbone in developmentally-realistic ways. Yet large gaps remain to be filled. -Lack of supervision Training must happen without large amounts of manually-labeled data. -Interaction and agency The developmental behavioral literature intertwines such developments with interaction with the world, explaining rich behaviors through this training. We built a simulated environment based on a game engine that provides 3D visual stimuli and allows for agent interactions with different types of objects. In this, we have an artificial agent gather experience, and with this experience, train a world model of self-supervised problems involving future, ego motion, and force predictions. We simultaneously train a model of future loss of the dynamical model. We execute an action policy determined by this predictive model that learns to focus its attention on what is interesting --- interaction with an object --- without explicitly encoding the notion of an object in the input. The policy adopts behaviors to put an object in view, approach it, and keep it in view. This leads to data collection that is more adversarial to the world model, allowing it to reach performance that a model trained on data through a random policy does not reach. The backbone developed is able to generalize to object localization through the training of a simple readout model without the backbone having been exposed to the true values of this problem. This represents first steps towards ecologically-realistic training of a vision system through an interactive, embodied process.

Meeting abstract presented at VSS 2018


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.