Abstract
We propose a comprehensive computational framework unifying previous qualitative studies of high-level cognitive influences on eye movements with quantitative studies demonstrating the influence of low-level factors such as saliency. In this framework, a top-level “governor” uses high-level task information to determine how best to combine low-level saliency and gist-based task-relevance maps into a single eye-movement priority map.
We recorded the eye movements of six trained subjects playing 18 different sessions of first-person perspective video games (car racing, flight combat, and “first-person shooter”) and simultaneously recorded the game's video frames, giving about 18 hours of recording for ∼15,000,000 eye movement samples (240Hz) and ∼1.1TB of video data (640×480 pixels at 30Hz). We then computed measures of how well the individual saliency and task-relevance maps predicted observers' eye positions in each frame, and probed for the role of the governor in relationships between high-level task information — such as altimeter and damage meter settings, or the presence/absence of a target — and the predictive strength of the maps.
One such relationship occurred in the flight combat game. In this game, observers are actively task-driven while tracking enemy planes, ignoring bottom-up saliency in favor of task-relevant items like the radar screen; then, after firing a missile, observers become passively stimulus-driven while awaiting visual confirmation of the missile hit. We confirmed this quantitatively by analyzing the correspondence between saliency and eye position across a window of ±10s relative to the time of 328 such missile hits. Around −200ms (before the hit), the saliency correspondence begins to rise, reaching a peak at +100ms (after the hit) of 10-fold above the previous baseline, then is suppressed below baseline at +800ms, and rebounds back to baseline at +2000ms. Thus, one mechanism by which high-level cognitive information can influence eye movements is through dynamically weighting competing saliency and task-relevance maps.
Intelligence Community (IC) postdoctoral fellowship program