September 2018
Volume 18, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2018
The effect of task on categorization behavior and its relationship to brain and deep neural networks
Author Affiliations
  • Martin Hebart
    Laboratory of Brain and Cognition, National Institute of Mental Health
  • Charles Zheng
    SFIM Machine Learning Core, National Institute of Mental Health
  • Chris Baker
    Laboratory of Brain and Cognition, National Institute of Mental Health
Journal of Vision September 2018, Vol.18, 395. doi:10.1167/18.10.395
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Martin Hebart, Charles Zheng, Chris Baker; The effect of task on categorization behavior and its relationship to brain and deep neural networks. Journal of Vision 2018;18(10):395. doi: 10.1167/18.10.395.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

A key objective in neuroscience is to understand how brains produce behavior. For the visual processing of objects, one approach is to relate categorization behavior to object representations at different cortical processing stages. A popular method to assess behavior is the object arrangement task, in which participants arrange objects in a 2D "arena" based on their relative similarity. While this method is efficient in producing representational dissimilarity matrices (RDMs) and is well-suited for uncovering low-dimensional representations or clearly defined clusters, it is prone to contextual biases and may be suboptimal for higher-dimensional or more continuous representations. Here we investigate the triplet task as an alternative approach for studying behavioral similarity. In this task, on each trial a participant has to choose an "odd-one out" from a set of three stimuli, yielding three binary similarity measures per response. For small to intermediate RDMs, this approach is efficient and cheap when carried out online in a distributed fashion, quickly yielding responses from hundreds of participants, while minimizing contextual effects. We compared RDMs obtained from both tasks using a set of 48 representative objects and tested their correspondence to deep neural networks (DNNs) that exhibit human-like categorization performance. While the RDMs between both tasks were highly correlated, the triplet RDM correlated much more strongly with DNN layers than the arrangement task. Using a commonly studied, more structured set of 92 objects, our results revealed comparable relationship to DNNs. However, the triplet task outperformed the arrangement task in explaining responses in human IT and later MEG responses, while earliest MEG responses were dominated by the arrangement task. Together, these results reveal the importance of the behavioral metric in teasing apart the relationship between brain, DNNs, and behavior. Further, our method provides a novel, efficient approach for obtaining behavioral measures of similarity.

Meeting abstract presented at VSS 2018

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×