December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Predicting Gaze Position with Deep Learning of Electroencephalography Data
Author Affiliations
  • Martyna Plomecka
    University of Zurich
  • Ard Kastrati
    ETH Zurich
  • Lukas Wolf
    University of Zurich
  • Roger Wattenhofer
    ETH Zurich
  • Nicolas Langer
    University of Zurich
Journal of Vision December 2022, Vol.22, 4010. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Martyna Plomecka, Ard Kastrati, Lukas Wolf, Roger Wattenhofer, Nicolas Langer; Predicting Gaze Position with Deep Learning of Electroencephalography Data. Journal of Vision 2022;22(14):4010.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The collection of eye gaze information is widely used in cognitive science and psychology. Moreover, many neuroscientific studies complement neuroimaging methods with eye-tracking technology to identify variations in attention, arousal and the participant's compliance with the task demands. To address limitations with conventional eye-tracking systems, recent studies have focused on leveraging advanced machine-learning techniques to compute gaze based on images from a webcam or images from Functional Magnetic Resonance Imaging (fMRI). While using a webcam to specify the eye gaze position requires an additional system and synchronization with auxiliary measures from the actual experiment that is even more cumbersome than in traditional eye-tracking systems, fMRI data acquisition is costly and does not provide the temporal resolution at the level that cognition takes place. In contrast, Electroencephalography (EEG) is a safe and cost-friendly method that directly measures the brain's electrical activity and enables measurement in clinical settings. However, an eye-tracking approach that offers gaze position estimation based on concurrently measured EEG is lacking. We address this shortcoming and show that gaze position can be restored by combining EEG activity and state-of-the-art machine learning. We use a dataset consisting of recordings from 400 healthy participants while they engage in tasks with varying complexity levels resulting in EEG and EOG features for over 3 million gaze fixations. To address intersubject variability and different experimental setups, we introduced a calibration paradigm, allowing the trained model to represent each participant's fixation characteristics throughout the experiment efficiently. Including a standardized, time-efficient and straightforward protocol to calibrate future recorded data on the pre-trained algorithm will improve the model's sensitivity, accuracy and versatility. This work emphasizes the importance of eye-tracking for the interpretation of EEG results and provides an open-source software that is widely applicable in research and clinical settings.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.