September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Saliency and the population receptive field model to identify images from brain activity
Author Affiliations & Notes
  • Alex Hernandez-Garcia
    Institute of Cognitive Science, University of Osnabrück, Germany
  • Wietske Zuiderbaan
    Spinoza Centre for Neuroimaging, 1105 BK Amsterdam, Netherlands
  • Akhil Edadan
    Spinoza Centre for Neuroimaging, 1105 BK Amsterdam, Netherlands
  • Serge O. Dumoulin
    Spinoza Centre for Neuroimaging, 1105 BK Amsterdam, Netherlands
    Department of Experimental and Applied Psychology, VU University Amsterdam, Netherlands
    Department of Experimental Psychology, Helmholtz Institute, Utrecht University, Netherlands
  • Peter König
    Institute of Cognitive Science, University of Osnabrück, Germany
    Institute of Neurophysiology und Pathophysiology, University Medical Center Hamburg-Eppendorf, Germany
Journal of Vision September 2019, Vol.19, 44. doi:https://doi.org/10.1167/19.10.44
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Alex Hernandez-Garcia, Wietske Zuiderbaan, Akhil Edadan, Serge O. Dumoulin, Peter König; Saliency and the population receptive field model to identify images from brain activity. Journal of Vision 2019;19(10):44. doi: https://doi.org/10.1167/19.10.44.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

One of the goals of visual neuroscience is to develop predictive models of brain activity to better understand the underlying mechanisms of the visual cortex. One example is the identification of presented images from fMRI recordings. Recently, Zuiderbaan et al. (2017) used the population receptive field (pRF) model and image contrast information to identify the presented stimulus from a set of natural images. Here, we extend the study by analyzing the predictive power of saliency-related information. Ultimately, we seek answers to these questions: What fraction of the responses is driven by saliency? Where in the visual cortex is saliency most represented? We test the Deep Gaze II and ICF (Kümmerer et al., 2017) saliency models. Deep Gaze II predicts the fixation density by using high-level features computed from a deep artificial neural network, while ICF uses only intensity and contrast features. We calculate the prediction response profile of every image as the summed overlap of its saliency map with the pRF at each cortical location. Then, we compute the correlation between the fMRI recordings and the prediction profiles of all images. An image is correctly identified if its predicted response elicits the highest correlation of the set. We find that ICF achieves the highest performance with a median correlation of .44 and 46.7 % accuracy (baseline 2.2 %) on V1, while Deep Gaze and the contrast model achieve .34 and .26 median correlation and 34.4 and 33.3 accuracy, respectively. On higher visual areas the performance gradually decreases. Both saliency models seem to be therefore better predictors of the presented stimuli than contrast information. The better performance of ICF might be explained because, while the pRF model only captures contrast energy information, the ICF maps may contain other information present in the measured responses, hence increasing the correlation of the predictions.

Acknowledgement: This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 641805 
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×