Abstract
A hallmark capability of working memory (WM) is the ability to rapidly encode approximations of sensory input with a single brief exposure and then to make that stored information available to other cognitive systems. While WM is often studied with simple features that are bound into objects, a different view of this capacity is that memories are constructed from different levels of a representational hierarchy to most efficiently capture information about a stimulus. We apply this idea using a generative, neurocomputational model, Memory for Latent Representations, that describes how visual learning creates a hierarchy of latent spaces from which memories can be constructed. This model allows for rapid tuning of representations according to task demands and also illustrates how memories of novel stimuli can be created without even forewarning participants of the kind of information, or the format of the response. We demonstrate the flexibility of such a model in storing information of various types, how memory for one object can consist of several traces from different levels of abstraction, and how the format of memory can be rapidly adjusted in the face of new task demands. Empirical demonstrations in human observers support the model by showing that memory traces can be calibrated against task demands by adding or removing detail information for both shape and color attributes. Perhaps most significantly, this perspective challenges current interpretations of working memory tasks by showing that different levels of the hierarchy could be used to store memories in different trials depending on factors such as set size, or stimulus duration. For example if duration is too short to allow binding of features for each of the objects, alternative encoding strategies might encode the whole display as one object and allow for binding after stimulus offset, though at a cost of more rapid forgetting.