Abstract
The human brain achieves visual object recognition through multiple stages of transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a massive millisecond resolution electroencephalography (EEG) dataset of human brain responses to images of objects on a natural background from the THINGS database. We used a time-efficient rapid serial visual presentation paradigm to extensively sample 10 participants, each with 16,740 image conditions repeated over 82,160 trials. We then leveraged the unprecedented size and richness of our dataset to train and evaluate deep neural network (DNN) based encoding models. The results showcase the quality of the dataset and its potential for computational modeling in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the number of conditions and trial repetitions of the EEG dataset contribute to the trained models’ prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release the dataset as a tool to foster research in computational neuroscience and computer vision. We believe it will be of great use to further understanding of visual object recognition through the development of high-temporal resolution computational models of the visual brain, and to optimize artificial intelligence models through biological intelligence data.