Abstract
Our capacity to interact with dynamic external stimuli in a timely manner (e.g., catch a ball) suggests that our brain generates predictions of unfolding external dynamics. While theories assume such an internal representation of future external states, the rich dynamics of predictive neural representations remain largely unexplored. One approach for investigating neural representations is representational similarity analysis (RSA), which typically uses models of static stimulus features at different hierarchical levels of complexity (e.g., colour, shape, category) to investigate how these features are represented in the brain. We present a novel dynamic extension to RSA that uses temporally variable models to capture the neural representation of dynamic stimuli. Here we tested this approach on source-reconstructed MEG data of 21 healthy human subjects who observed 14 unique 5sec-long ballet videos, with ~35 repetitions per video. Dynamic RSA revealed unique insights into the representation of low-level visual, body posture and kinematic features: Both low- and high-level information was represented ~ 40–300 msec after the actual visual input, but with low-level information represented most prominently in visual areas, and higher-level information also in slightly more anterior areas. Strikingly, the motion of the ballet dancer was not only represented in a lagged manner, but also in a second distinct temporal window that preceded the actual input by ~ 100–300 msec, indicating that these neural representations predicted future motion. Taken together, dynamic RSA reveals delayed bottom-up and predictive top-down processing in naturalistic dynamic stimuli. As such, it opens the door for addressing important outstanding questions on how our brain represents and predicts the dynamics of the world. More generally, it can be used to study interesting concepts such as predictive coding not only in naturalistic dynamic visual stimuli, but also across a wide range of domains such as naturalistic reading, sign language, etc.