Abstract
Viewing and recall engage overlapping neural systems in the human brain. They are further linked through gaze reinstatement, the recapitulation of encoding-related gaze patterns during recall. Here, we characterize the relationship between these phenomena for continuous narratives. Specifically, we investigate whether eye movements reflect the event structure of a movie, and whether the observed neural overlap is grounded in shared gaze patterns. We tested this by combining eye tracking and model-based gaze predictions with fMRI data acquired while participants watched and recalled an episode of the BBC show Sherlock. First, we found that gaze patterns during movie viewing were indeed event specific. Further, these patterns were consistent across participants and in-scanner and out-of-scanner eye tracking, predicted by frame-wise saliency, and predictive of how memorable each event was. Second, we reconstructed the movie event structure during recall using a hidden Markov model trained on the MR-signal of the eyeballs during viewing, suggesting that encoding related gaze patterns were partially reinstated during recall. Finally, we related the eyeball multi-voxel pattern to brain activity and found substantial overlap in gaze-dependent activity between viewing and recall. Collectively, our results suggest that gaze patterns are inherently linked with how we remember events, and that behavioral and cortical reinstatement jointly support recall of episodic memories.