Purchase this article with an account.
Nicholas Hindy, Emily Avery, Nicholas Turk-Browne; Semantic knowledge and action-based visual prediction. Journal of Vision 2017;17(10):1169. doi: 10.1167/17.10.1169.
Download citation file:
© ARVO (1962-2015); The Authors (2016-present)
Goal-directed behavior requires predicting the consequences of our actions. Predictions based on specific episodes in our past may arise from pattern completion in the hippocampus, which in turn reinstates the retrieved consequences in visual cortex (Hindy et al., 2016, Nature Neurosci). However, action-based predictions are not always tied to specific episodes, but instead to general semantic knowledge learned pedagogically or aggregated from many experiences over a long timescale. For example, when cracking open an egg we expect to see a yoke. It is unknown how such visual predictions based on long-term semantic memory are generated. We hypothesized that they may also be supported by the hippocampus. To test this hypothesis, we collected high-resolution fMRI data while participants selected among familiar actions that changed the state of one object into another. For the Novel condition, participants learned actions that changed objects in arbitrary ways (e.g., "point" or "wave" to transform a dollar bill into a carrot or a newspaper), and thus predictions needed to be based on recently experienced episodes. For the Known condition, the outcomes of actions were obvious (e.g., when you "roll" or "fold" a dollar bill) because predictions could rely on pre-existing semantic knowledge. In both conditions, responses in early visual cortex were attenuated when an outcome could be predicted from an action, and this predictive attenuation was related to background connectivity with the hippocampus. Examining the hippocampus more closely, there was a dissociation over the long axis, with Novel prediction in visual cortex related to the anterior hippocampus and Known prediction related to posterior hippocampus. These findings suggest that the hippocampus may mediate visual prediction even based on long-term semantic knowledge, but possibly through different mechanisms than observed for visual prediction based on recent episodic learning.
Meeting abstract presented at VSS 2017
This PDF is available to Subscribers Only