Abstract
Seeing involves not only the light that reaches our retinae, but also visual memories and expectations. When searching for our favourite mug, we hold its appearance in mind. When talking to a friend over the phone, we picture their facial expressions. Talented artists can even paint detailed visual scenes from memory. The content of such mental representations is of great interest in perceptual and cognitive neuroscience, yet is challenging to measure experimentally (alas, too few of our participants are talented artists!). One previously-proposed approach is reverse correlation. Participants are asked to report whether they see a signal—e.g., the letter “s”—in pure noise. Over enough trials, some random noise samples will happen to resemble the signal the participant has in mind. Averaging over all trials in which a participant spuriously detected a signal provides a “classification image” — a visualisation of their representation of the letter. One major drawback of reverse correlation is that it requires tens of thousands of trials to obtain even coarse impressions of mental images. Perhaps more importantly, reverse correlation cannot recover multiple concurrent templates. If a participant imagined both print and cursive instances of the letter “s”, the recovered classification image would be a jumble of both. We address these issues using an evolutionary algorithm approach. We generate image populations by crossbreeding noise. On each trial, participants are shown multiple alternative images drawn from these populations. Across generations, only those images in which participants detect a signal are kept for further breeding. In both simulation and experiments with human participants, we demonstrate that this method converges faster than standard reverse correlation, can recover multiple internal representations of a signal, and can even provide access to the mental representation of illusory visual percepts. Our approach thus provides an efficient, data-driven way to access complex mental representations.