Purchase this article with an account.
Martin Hebart, Charles Zheng, Chris Baker; The effect of task on categorization behavior and its relationship to brain and deep neural networks. Journal of Vision 2018;18(10):395. doi: 10.1167/18.10.395.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
A key objective in neuroscience is to understand how brains produce behavior. For the visual processing of objects, one approach is to relate categorization behavior to object representations at different cortical processing stages. A popular method to assess behavior is the object arrangement task, in which participants arrange objects in a 2D "arena" based on their relative similarity. While this method is efficient in producing representational dissimilarity matrices (RDMs) and is well-suited for uncovering low-dimensional representations or clearly defined clusters, it is prone to contextual biases and may be suboptimal for higher-dimensional or more continuous representations. Here we investigate the triplet task as an alternative approach for studying behavioral similarity. In this task, on each trial a participant has to choose an "odd-one out" from a set of three stimuli, yielding three binary similarity measures per response. For small to intermediate RDMs, this approach is efficient and cheap when carried out online in a distributed fashion, quickly yielding responses from hundreds of participants, while minimizing contextual effects. We compared RDMs obtained from both tasks using a set of 48 representative objects and tested their correspondence to deep neural networks (DNNs) that exhibit human-like categorization performance. While the RDMs between both tasks were highly correlated, the triplet RDM correlated much more strongly with DNN layers than the arrangement task. Using a commonly studied, more structured set of 92 objects, our results revealed comparable relationship to DNNs. However, the triplet task outperformed the arrangement task in explaining responses in human IT and later MEG responses, while earliest MEG responses were dominated by the arrangement task. Together, these results reveal the importance of the behavioral metric in teasing apart the relationship between brain, DNNs, and behavior. Further, our method provides a novel, efficient approach for obtaining behavioral measures of similarity.
Meeting abstract presented at VSS 2018
This PDF is available to Subscribers Only