Abstract
How does the brain encode visual context at different temporal scales? When processing familiar sensory and semantic input, cortex is sensitive to input further into the past along a posterior to anterior gradient (Hasson et al. 2015). To investigate how we learn new hierarchical temporal structure in the visual domain, we designed a novel paradigm employing statistical learning that can be used to map neural contributions to contextual representation at different time scales. Over four behavioral experiments (N=72), we demonstrate that humans are sensitive to transition points among both low- and high-level sequential units during exposure to sequences of abstract images (fractals). However, results may be attributable to low-level learning of image trigrams. Thus, we altered the paradigm to more effectively disentangle learning of nested order information at slow and fast temporal scales. One of eight context cue images is presented multiple times, and embedded in this stream are paired associate images. Critically, pairwise contingencies depend on both the identity of the context cue (fast temporal scale) as well as the time since the previous context shift (slow temporal scale). We have found that multi-layer recurrent neural networks trained to predict the upcoming image in this paradigm encode order information at shorter time scales at lower levels (closer to perceptual input). However, only neural architectures that can remember further into the past (i.e. those using long-short term memory units, rather than simple recurrent units) can learn the slow temporal structure. Planned neuroimaging work will test the idea that brain regions similarly spatially segregate these timescales. In particular, we anticipate that the hippocampus will represent these hierarchical timescales on an anterior-posterior gradient and that prefrontal cortical regions will be engaged along a lateral-medial gradient.