Abstract
With the rise of convolutional neural networks (CNNs), computer vision models of object recognition have improved dramatically in recent years. Most recent progress in computer vision has been spurred by increasing the number of layers within CNN models (so called 'very-deep' learning models). Just like the ventral cortex in the human brain, CNNs show an increase in receptive field size and an increase in neuronal tuning when moving up the neural or computational hierarchy (DiCarlo et al., 2012).However, from neuroscience we know that the brain processes information not only hierarchically but also in parallel (Kravitz et al., 2013). In the current study, we trained a CNN with an Alexnet type architecture (5 convolutional layers, 2 fully connected layers, 1 softmax layer) using two different image sets (animacy or places). Additionally, we evaluated human brain responses towards 120 images (not used for training the CNNs), containing places and animate and inanimate images using BOLD-MRI. For this, we calculated summary statistics, per image, per layer of the CNN and evaluate to what degree we can explain the between image variance. We observe, using the same images, distinctly different patterns of explained variance for the animate trained networks versus the place and inanimate network. The animate trained network explains variance in the middle and inferior temporal gyrus using information from the top two convolutional layers. The summary statistics from the places-trained network explains variance in a range of visual areas using the second fully connected layers and surprisingly, in the parahippocampal complex using the softmax layer. These results suggest, in congruence with our current understanding of the functional achitecture of the brain, that the brain consists of multiple CNNs but also demonstrate that the mapping of CNN vs brain is complex.
Meeting abstract presented at VSS 2016