Characterizing surprise in humans and monkeys

Abstract

We investigate the role of visual surprise in guiding eye movements in humans and rhesus monkeys under free viewing conditions, for a variety of natural stimuli. Surprise differs from other models of bottom-up visual attention in that it quantifies how data affects an observer, by measuring the difference between posterior and prior beliefs of the observer. We recorded eye movements from naive observers, 4 humans and 3 monkeys, while they watched 115 video clips (47,903 frames, 27 minutes) resulting in 6,775 saccades for humans and 10,406 for monkeys. Clips ranged in semantic content, including video of natural, non-natural, building-city, indoor, and sporting-outdoor scenes both with and without main actors. A surprise model of bottom-up visual attention then predicted in real-time how surprising every location was in the display. The distribution of surprise at the endpoint (target) locations of human or monkey saccadic eye movements was then compared to the distribution of surprise at random locations using a standard information theoretic technique, Kullback-Leibler distance. Considering all clips together 59% and 56% of gaze shifts were directed towards locations more surprising than average for humans and monkeys, however, agreement with the model varied greatly across clip type (ranging from 35–77%). Humans and monkeys showed a similar pattern of agreement with the model across image type, with a significant difference only in sporting-outdoor clips. This data suggests that under free viewing humans and monkeys are employing similar bottom-up attentional mechanisms.