August 2016
Volume 16, Issue 12
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2016
Co-registration of eye movements and EEG to study semantic congruency during scene perception
Author Affiliations
  • Helene Devillez
    Department of Psychology and Neuroscience, University of Colorado Boulder
  • Randall O'Reilly
    Department of Psychology and Neuroscience, University of Colorado Boulder
  • Tim Curran
    Department of Psychology and Neuroscience, University of Colorado Boulder
Journal of Vision September 2016, Vol.16, 316. doi:10.1167/16.12.316
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Helene Devillez, Randall O'Reilly, Tim Curran; Co-registration of eye movements and EEG to study semantic congruency during scene perception. Journal of Vision 2016;16(12):316. doi: 10.1167/16.12.316.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Visual selection processes in real-world scenes are guided by exogenous and endogenous factors. Endogenous factors correspond to top-down influences like the cognitive tasks or prior knowledge. Object processing is affected by the gist of the scene within which it is embedded and the prior knowledge about the objects. On one hand, it has been shown that incongruent objects result in prolonged and more frequent eye-fixations than congruent objects (Underwood et al, 2008; Võ & Henderson, 2009). On the other hand, previous event-related potential (ERP) research has suggested that manipulating the semantic congruency between an object and the surrounding scene affects the high level representation of that object (Ganis & Kutas, 2003). The congruency effect is reflected by a late ERP resembling the N300/N400 effect previously associated with semantic integration (Mudrik, Lamy, & Deouell, 2010). The present study investigates the effect of semantic congruency on scene processing using eye-fixation related potentials (EFRPs). We simultaneously registered electroencephalographic (EEG) and eye-tracking signals of participants exploring natural scenes during 4 sec in preparation for a recognition memory test. We compared EFRPs evoked by congruent vs. incongruent eye-fixations (e.g., a fork in a kitchen vs. the same fork in a bathroom). First, we replicated previous eye movement results, showing that incongruent objects were more fixated and for a longer duration than congruent objects. Second, the EFRP analysis revealed that both early and late EFRPs were influenced by the congruency effect. The P1 EFRP and a late EFRP emerging around 260 ms after the fixation onset were modulated by semantic congruency. The top-down encoding of the scene was built during the first eye fixations; a mismatch between the semantic knowledge of objects and the features of the scene affected the scene exploration. These results suggest that top-down information influences early object processing during natural viewing.

Meeting abstract presented at VSS 2016

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×