Abstract
For millennia, philosophers and scientists have pondered on the nature of our internal mental representations. In the study of short-term memory and mental imagery, a key debate has been whether representations are analogical (i.e., with a form similar to that which comes from our eyes) or abstracted into a more symbolic code. Using multi-voxel pattern analysis (MVPA) and the new real-time fMRI method of Dynamically Adaptive Imaging (DAI), we were able to characterize the information content of our internal representations during the maintenance of short-term memory or production of imagery. Across five experiments, we have found that abstracted semantic characteristics dominate memory and imagery, and have found little evidence for analogical representation. In two of these experiments using MVPA and complex grating stimuli, we found strong retinotopic representation of attended parts of a display in occipital, temporal and parietal regions. However, during the memory period of a change detection task, although strong univariate activity persisted in parietal and ventral visual regions, retinotopy vanished. Three further experiments used DAI and MVPA with naturalistic stimuli, to characterize feature tuning in ventral visual regions. We find that during memory or imagery, the patterns of activity evoked in ventral regions are best explained by semantic rather than perceptual features.