RT Journal Article
A1 Ramakrishnan, Kandan
A1 Scholte, H. Steven
A1 Smeulders, Arnold
A1 Ghebreab, Sennay
T1 Mapping human visual representations by deep neural networks
JF Journal of Vision
JO Journal of Vision
YR 2016
DO 10.1167/16.12.373
VO 16
IS 12
SP 373
OP 373
SN 1534-7362
AB  A number of recent studies have shown that deep neural networks (DNN) map to the human visual hierarchy. However, based on a large number of subjects and accounting for the correlations between DNN layers, we show that there is no one-to-one mapping of DNN layers to the human visual system. This suggests that the depth of DNN, which is also critical to its impressive performance in object recognition, has to be investigated for its role in explaining brain responses. On the basis of EEG data collected from a large set of natural images we analyzed different DNN architectures  a 7 layer, 16 layer and a 22 layer DNN network using weibull distribution for the representations at each layer. We find that the DNN architectures reveal temporal dynamics of object recognition, with early layers driving responses earlier in time and higher layers driving the responses later in time. Surprisingly the layers from the different architectures explain brain responses to a similar degree. However, by combining the representations of the DNN layers we observe that in the higher brain areas we explain more brain activity. This suggests that the higher areas in the brain are composed of multiple non-linearities that are not captured by the individual DNN layers. Overall, while DNNs form a highly promising model to map the human visual hierarchy, the representations in the human brain go beyond the simple one-to-one mapping of the DNN layers to the human visual hierarchy. Meeting abstract presented at VSS 2016 
RD 4/16/2021
UL https://doi.org/10.1167/16.12.373