Abstract
Understanding temporally extended natural behaviour requires studying the coordination of vision, action, and memory during free-flowing interactions with the environment. However, naturalistic experiments capturing the complexity of natural behaviour are often incompatible with the desire for tight experimental control. Therefore, sub-components of natural behaviour have been predominantly studied in isolation using desktop-based lab tasks – missing potential interconnections between these sub-components that only emerge during continuous and temporally extended behaviour. Here, we share a unique open virtual reality (VR) resource for quantifying and segmenting continuous behaviour into tractable sub-units. Within a single temporally extended natural task, participants copied a Model display by selecting realistic objects from a Resource pool and placing them into a Workspace. We track head, hand, and eye movements as well as free-flowing interactions with the environment. This allows us to derive metrics for multiple components involved in coordinating vision, memory, and action: i) sampling information from the external environment (encoding), ii) searching for task-relevant items (visual search), and iii) the trade-off between reliance on information in memory and gathering information from the external environment (memory usage). By manipulating the locomotive effort required to complete the task and the difficulty of finding task-relevant items, we demonstrate how these measures are differentially sensitive to changing task demands. Further, participants completed the task at two timepoints (T1-T2: 7-12 days), allowing us to investigate replicability and compute test-retest reliability for the derived metrics. We show replicable group effects and substantial test-retest reliability of the derived metrics. By providing a freely available task repository, a large data set of multivariate VR data, a pre-processing package, and ready-to-use analysis scripts, our resources offer a bridge between tightly controlled experimental approaches and unconstrained natural behaviour.