Abstract
The objective was to investigate the effects of neural synchrony between visual and auditory processing regions on associative memory for multisensory information. On a cellular level episodic memory formation is likely mediated by fast acting long-term potentiation which relies on the fine grained timing of neural activity. Brain oscillations, especially theta oscillations in the hippocampus, orchestrate such timing, and are a likely "gluing mechanism" for human memories. In the first experiment, in the encoding phase, all participants were shown short (3-second) videos that were luminance modified with a 4 Hz sine wave, with an accompanying audio clip that had been amplitude modulated with a 4 Hz sine wave. The phase offset (i.e. synchrony) between the audio clip and the video was 0, 90, 180, or 270 degrees. In a second experiment, the videos and sounds were modulated at 4 Hz, 1.7 Hz (delta), and 10.5 Hz (alpha). On each trial, participants rated how well the audio clip suited the contents of the video clip. Each of six blocks contained 16 audio-video pairings (four at each phase angle), and was followed by a brief distractor task and an associative recognition test. We expected that associative memory performance would vary as a function of synchrony, with better performance when the video and audio clips were both presented in synchrony compared to out-of-synchrony. Our results were in line with our expectations. We demonstrated that the formation of multisensory memories rely on the exact timing of a particular oscillatory phase that corresponds to the human hippocampal theta oscillation. This effect was observed at theta, but not at a slower (delta) or a faster (alpha) frequency. Our results therefore provide direct evidence for a causal role of theta phase synchronization for the formation of human memories.
Meeting abstract presented at VSS 2016