August 2023
Volume 23, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2023
A large and rich EEG dataset for modeling human visual object recognition
Author Affiliations & Notes
  • Alessandro T. Gifford
    Freie Universität Berlin
  • Kshitij Dwivedi
    Goethe Universität Frankfurt am Main
  • Gemma Roig
    Goethe Universität Frankfurt am Main
  • Radoslaw M. Cichy
    Freie Universität Berlin
  • Footnotes
    Acknowledgements  A.T.G. is supported by a PhD fellowship of the Einstein Center for Neurosciences. R.M.C. is supported by German Research Council (DFG) grants (CI 241/1-1, CI 241/3-1, CI 241/1-7) and the European Research Council (ERC) starting grant (ERC-StG-2018-803370).
Journal of Vision August 2023, Vol.23, 4579. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Alessandro T. Gifford, Kshitij Dwivedi, Gemma Roig, Radoslaw M. Cichy; A large and rich EEG dataset for modeling human visual object recognition. Journal of Vision 2023;23(9):4579.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The human brain achieves visual object recognition through multiple stages of transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a massive millisecond resolution electroencephalography (EEG) dataset of human brain responses to images of objects on a natural background from the THINGS database. We used a time-efficient rapid serial visual presentation paradigm to extensively sample 10 participants, each with 16,740 image conditions repeated over 82,160 trials. We then leveraged the unprecedented size and richness of our dataset to train and evaluate deep neural network (DNN) based encoding models. The results showcase the quality of the dataset and its potential for computational modeling in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the number of conditions and trial repetitions of the EEG dataset contribute to the trained models’ prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release the dataset as a tool to foster research in computational neuroscience and computer vision. We believe it will be of great use to further understanding of visual object recognition through the development of high-temporal resolution computational models of the visual brain, and to optimize artificial intelligence models through biological intelligence data.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.