Abstract
Our natural environment is dynamic. What perceptual and cognitive processes govern our continuously-changing natural experiences? A key challenge in the study of naturalistic perception is disentangling the multitude of perceptual and cognitive processes fluctuating simultaneously, without altering the ongoing natural experience. Here, we present a novel behavioral approach to track the dynamics of human experience retrospectively. This method of retrospective behavioral sampling models distinct dimensions of perception and cognition along the time-course of continuous audiovisual stimuli, without manipulating, simplifying or interfering with real-time natural perception. In the current study, 28 participants viewed short movies and listened to an auditory narrative. Audiovisual stimulation was followed by a questionnaire in which participants recalled their experience of more than 50 discrete events from the stimuli, measuring their quality of memory, magnitude of surprise, negative emotion, positive emotion, perceived importance, evoked introspection, and mental time travel. In addition, between 1 to 2 weeks after audiovisual stimulation, participants again rated their quality of memory for each of the sampled events. Results reveal highly reliable, and informative, temporal patterns of change in each of the measured cognitive states, exhibiting effective variability across events while maintaining consistency across individuals. Remarkably, memory ratings more than a week after stimulation resulted in an almost identical time-course of mean memorability across subjects, as was found immediately following stimulation. Altogether, these findings validate the effectiveness of retrospective behavioral sampling in tracking fluctuations in cognitive states throughout naturalistic audiovisual stimulation, reliably across individuals and stably across measurements. We further discuss how these behavioral data can be used to identify discrete cognitive processes in neural patterns of response to the same audiovisual stimuli.