September 2011
Volume 11, Issue 11
Free
Vision Sciences Society Annual Meeting Abstract  |   September 2011
Assessing models of visual saliency against explicit saliency judgments from one hundred humans viewing eight hundred real scenes
Author Affiliations
  • Kathryn Koehler
    University of California-Santa Barbara, USA
  • Fei Guo
    University of California-Santa Barbara, USA
  • Sheng Zhang
    University of California-Santa Barbara, USA
  • Miguel Eckstein
    University of California-Santa Barbara, USA
Journal of Vision September 2011, Vol.11, 165. doi:10.1167/11.11.165
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Kathryn Koehler, Fei Guo, Sheng Zhang, Miguel Eckstein; Assessing models of visual saliency against explicit saliency judgments from one hundred humans viewing eight hundred real scenes. Journal of Vision 2011;11(11):165. doi: 10.1167/11.11.165.

      Download citation file:


      © 2017 Association for Research in Vision and Ophthalmology.

      ×
  • Supplements
Abstract

Researchers have proposed computational models of visual saliency, which have been assessed against observers' saccadic eye movements during free viewing (Parkhurst, Law, & Niebur, 2002; Itti & Koch, 2001; Peters et al., 2005; Schumann et al., 2008) and visual search (Itti & Koch, 2000; Torralba et al., 2006). Surprisingly no studies have evaluated saliency models against explicit observer judgments of visual saliency. Here, we compare saliency model predictions (Walther & Koch, 2006) to saliency judgments of one hundred observers. Methods: We used 800 natural images displaying common indoor and outdoor scenes with varying numbers of objects. One hundred human observers were instructed to view each image and click on the most salient location of each image. We evaluated the saliency model by quantifying the proportion of human-reported most salient locations falling within circular regions (with radii varying between 21.06 and 107.44 pixels) containing the top five salient model predictions. As a baseline condition, we evaluated the agreement between the model and one hundred randomly generated saliency selections. Results: Across all observers and images the proportions of explicit saliency reports within the top five model predictions were: 0.102, 0.068, 0.057, 0.047, 0.044 (0.31 ± 0.21 across all top five saliency predictions). In contrast, the agreement of model predictions with the randomly generated saliency selections was 0.08 ± 0.03. Thus, our findings show a modest agreement between model saliency predictions and human explicit judgments, but significantly larger (t = 13.86, p < 0.001) than expected by chance. The methods presented can serve as a protocol for evaluation of other saliency models (Bruce & Tsotsos, 2006; Zhang et al., 2008) and comparisons of models' ability to predict explicit saliency judgments vs. saccadic eye movements.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×