September 2017
Volume 17, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2017
What eye movement and memory experiments can tell us about the human perception of visualizations
Author Affiliations
  • Zoya Bylinskii
    Computer Science and Artificial Intelligence Lab, MIT
  • Michelle Borkin
    College of Computer and Information Science, Northeastern
  • Nam Kim
    School of Engineering and Applied Science, Harvard University
  • Hanspeter Pfister
    School of Engineering and Applied Science, Harvard University
  • Aude Oliva
    Computer Science and Artificial Intelligence Lab, MIT
Journal of Vision August 2017, Vol.17, 532. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Zoya Bylinskii, Michelle Borkin, Nam Kim, Hanspeter Pfister, Aude Oliva; What eye movement and memory experiments can tell us about the human perception of visualizations. Journal of Vision 2017;17(10):532.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

What makes a visualization (a graph, chart, or infographic) memorable? Where do people look on a visualization during encoding and retrieval? What aspects of the visualization can later be recalled from memory? Studying these questions in the context of visualizations allows us to expand our understanding of human perception to complex visual content beyond natural scenes. We ran a series of experiments in the lab and on the crowdsourcing platform Amazon's Mechanical Turk (MTurk). In the lab, we recorded people's eye movements as they viewed infographics for 10 seconds each, with a retrieval task at the end of 20 minutes. In the final phase, participants provided descriptions of the visualizations from memory. On MTurk, we ran a series of memorability tests, and separately, a series of description tasks where participants clicked on blurred visualizations to reveal small bubble-like regions at full resolution. We compared the consistency of human memory and the consistency of human attention on visualizations, in lab and online settings. First, we find that people are consistent in where they attend and what they remember in visualizations. Second, we discover some qualities of visualizations that tend to make them more memorable. Importantly, we show that a visualization that is memorable at-a-glance can often also be recalled and described with higher fidelity. Third, we quantify which elements of visualizations people tend to focus their attention most on. We find that text, and especially titles, are particularly important; and that visual elements are not as distracting as initially hypothesized. Finally, we make our full dataset of thousands of visualizations, eye movements, memory scores, and text descriptions available to the community ( This dataset presents the opportunity of delving into a variety of old and new perception questions using a novel image type for such investigations.

Meeting abstract presented at VSS 2017


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.