December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
A Beta-Variational Auto-Encoder Model of Human Visual Representation Formation in Utility-Based Learning
Author Affiliations & Notes
  • Tyler Malloy
    Rensselaer Polytechnic Institute
  • Chris R. Sims
    Rensselaer Polytechnic Institute
  • Footnotes
    Acknowledgements  This work was supported by NSF research grant DRL-1915874 to CRS and an IBM AIRC scholarship to TJM.
Journal of Vision December 2022, Vol.22, 3747. doi:https://doi.org/10.1167/jov.22.14.3747
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Tyler Malloy, Chris R. Sims; A Beta-Variational Auto-Encoder Model of Human Visual Representation Formation in Utility-Based Learning. Journal of Vision 2022;22(14):3747. https://doi.org/10.1167/jov.22.14.3747.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The human brain is capable of forming informationally constrained representations of complex visual stimuli in order to achieve its behavioural goals, such as utility-based learning. Recently, methods borrowed from machine learning have demonstrated a close connection between the mechanisms of visual representation formation in primate brains with the latent representations formed by Beta-Variational Auto-Encoders (Beta-VAEs). While auto-encoder models capture some aspects of visual representations, they fail to explain how visual representations are adapted in a task-directed manner. We developed a model of visual representation formation in learning environments based on a modified Beta-VAE model that simultaneously learns the task-specific utility of visual information. We hypothesized that humans update their visual representations as they learn which visual features are associated with utility in learning tasks. To test this hypothesis, we applied the proposed model onto the data from a visual contextual bandit learning task [Niv et al. 2015; J. Neuroscience]. The experiment involved humans (N=22) learning the utility associated with 9 possible visual features (3 colors, shapes or textures). Critically, our model takes in as input the same visual information that is presented to participants, instead of the hand-crafted features typically used to model human learning. A comparison of predictive accuracy between our proposed model and models using hand-crafted features demonstrated a similar correlation to human learning. These results show that representations formed by our Beta-VAE based model can predict human learning from complex visual information. Additionally, our proposed model makes predictions of how visual representations adapt during human learning in a utility-based task. Further, we performed a comparison of our proposed model across a range of parameters such as information-constraint, utility-weight, and number of training steps between predictions. Results from this comparison give insight into how the human brain adjusts its visual representation formation during learning.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×