December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Evaluation of DeepGaze-Generated Saliency Maps for Affective Scenes
Author Affiliations
  • Yujun Chen
  • Peng Liu
  • Ruogu Fang
  • Mingzhou Ding
Journal of Vision December 2022, Vol.22, 3967. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Yujun Chen, Peng Liu, Ruogu Fang, Mingzhou Ding; Evaluation of DeepGaze-Generated Saliency Maps for Affective Scenes. Journal of Vision 2022;22(14):3967.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

DeepGaze (K├╝mmerer et al., 2017) is an AI platform that is trained on and designed to generate saliency maps for natural images that are emotionally neutral. To what extent it can be used to generate saliency maps for natural images varying in emotional content remains not known. We addressed the question (1) by comparing the DeepGaze-generated saliency maps with eye-tracking data from human observers viewing affective scenes from the International Affective Picture System (IAPS) (Fan et al., 2018) and (2) by analyzing the impact of occluding the DeepGaze-generated saliency map on emotion recognition by a recently proposed VisualCortex-Amygdala (VCA) deep learning system for valence analysis. We found that (1) the saliency maps generated by DeepGaze matched the eye-tracking data better than that generated by other methods including the Grad-CAM (Selvaraju et al., 2017) and (2) occluding the saliency map generated by DeepGaze led to significantly more deteriorated valence recognition by the VCA system compared to occluding random patches of voxels. These results suggest that DeepGaze, despite being trained on emotionally neutral images, can generate saliency maps that convey the emotional significance of the images.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.