Neural encoding models of color working memory reveal categorical representations in sensory cortex

Abstract

Working memory is represented by distributed regions of the human neocortex which are believed to form a gradient of abstraction from detailed representations in sensory cortex to more abstract mnemonic traces in frontal areas. Variations in neural tuning found in non-human primates tentatively support this notion, but this tenet lacks rigorous investigation. Here, we used fMRI and multivariate encoding models to assess whether mnemonic representation of remembered colors can be categorized as either detailed-continuous or categorical. To this end, we asked 10 healthy participants (4 MRI sessions, each) to perform a conventional color working memory task using prolonged (10 s) delays and a retro-cue procedure in the MRI scanner. Critically, we sampled the memorized colors from a rigorously calibrated color space in a fine-grained fashion to closely capture the similarity-structure of neural activity patterns representing color across hue. We then used cvMANOVA MVPA to identify regions representing memorized colors during the working memory delay. We then estimated the variance explained by (1) van-Mises-based models (continuous models, typically used for IEM) and (2) models informed by the subjects’ individual boundaries and prototypes of typical color categories (categorical models). Using an anatomical regions of interest approach, we found robust mnemonic color representations in V1, V4 and V01. Importantly, we found that during working memory, categorical models explained color representations in V4 and V01 significantly better than conventional continuous models. In contrast, we found no such di_erences in are V1 or when subjects were engaged in an immediate recall task with little demands on working memory. Our results support a view of working memory where storage relies on distributed circuits utilizing neural tuning functions with varying granularity and abstraction. Our results further suggest that some regions might change their tuning properties in the course of the working memory delay.