October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
What’s salient in a graphic design anyway?
Author Affiliations
  • Zoya Bylinskii
    Adobe Inc.
  • Camilo Fosco
  • Vincent Casser
  • Amish Kumar Bedi
    Adobe Inc.
  • Aaron Hertzmann
    Adobe Inc.
Journal of Vision October 2020, Vol.20, 414. doi:https://doi.org/10.1167/jov.20.11.414
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Zoya Bylinskii, Camilo Fosco, Vincent Casser, Amish Kumar Bedi, Aaron Hertzmann; What’s salient in a graphic design anyway?. Journal of Vision 2020;20(11):414. doi: https://doi.org/10.1167/jov.20.11.414.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Movie posters, social graphics, and product advertisements are designed to capture the viewer’s attention and efficiently redirect it to relevant information. For natural images, computational models of saliency have been successful at predicting where the average observer is likely to look. However, such models have been lacking for graphic designs. Knowing what holds an observer’s attention for a longer period, rather than what pops out in a bottom-up manner, is more relevant for graphic designs, which are composed of image and text elements. Analogous to saliency, we define an ‘importance map’ as a heatmap providing a real value at each image pixel indicating the probability that observers would find that image region important. In contrast to saliency, instead of collecting ground truth data using an eye tracker, we ask human participants to annotate regions of graphic designs they think are important (using methodology from O’Donovan 2014). Averaging the annotations of 25-30 observers generates smooth, ground truth importance maps. We collected importance maps for 1000 designs across 5 different classes: webpages, movie posters, mobile UIs, infographics, and advertisements, and present this as the Imp1k dataset. We also introduce a computational model of importance for graphic designs, trained using Imp1k. Our model is a deep neural network that can simultaneously predict the class of a graphic design with 95% accuracy, and can predict the importance maps with a Pearson’s Cross Correlation of 0.827 (KL score of 0.159) compared to ground truth. We extend our model by training it with natural images as a 6th class, and demonstrate that the same model can be used to predict saliency maps on natural images and importance maps on graphic designs. Finally, we show how our Unified Model of Saliency and Importance (UMSI), can be used to generate automated suggestions within interactive design applications.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.