Abstract
Humans are very efficient at categorizing natural scenes. In addition to fast visual processing, such categorization implies contact with stored representations in the brain. Accordingly, we were able to decode natural scene category from both visual areas and regions in the frontal lobe (Walther et al., 2009). Moreover, we’ve seen better detection and decoding of good exemplars of scenes than bad, suggesting an influence of past experience. Here we explored the relationship between semantics and scene decoding. Specifically, we asked if scenes (for eg. Picture of a beach) and related words (for eg. ‘Beach’) share common representations in the brain. In an fMRI experiment, subjects passively viewed blocks of either Words or Pictures of scenes, for four different categories: beaches, cities, highways and mountains. A whole brain searchlight, using multi-voxel pattern analysis, was performed on the fMRI data. In keeping with our earlier results, we were able to decode scene category from pictures in occipitotemporal cortex, and the inferior frontal/precentral gyrus. To determine whether the words and pictures share similar representations we trained a classifier on one condition (e.g. word stimuli) and then tested it on the other condition (e.g. picture stimuli). One common area in the precuneus showed weak transfer from pictures to words. This area has previously been associated with imagery, suggesting the successful cross-decoding was visual rather than semantic in nature.
Meeting abstract presented at VSS 2013