Abstract
Cortical stimulation in high-level visual areas causes complex perturbations in visual perception. Understanding the nature of stimulation-induced visual perception is necessary for characterizing visual hallucinations in psychiatric diseases and developing visual prosthetics. Most evidence is derived from anecdotal observations of human patients, but systematic studies have been severely limited due to the lack of language faculty in nonhuman primates. We developed a novel method, perceptography, to “take pictures” of the complex visual percepts induced by optogenetic stimulation of the inferior temporal (IT) cortex in macaque monkeys. Each trial started with a fixation on a computer-generated image. Halfway through the image presentation (1s), we perturbed the image features for 200ms. At the same time, IT cortex was optogenetically stimulated via an implanted LED array in half of the trials at random. The animals were rewarded for detecting cortical stimulation by looking at one of the two subsequently presented targets. Under the hood, two deep learning systems, DaVinci (GAN) and Ahab (Deep-learning feature extraction pipeline), controlled image alterations and tracked the animals’ behavioral responses, respectively. We hypothesized that false alarms (FA) are more likely to happen when an image alteration shares common features with the percept induced by cortical stimulation. In a functional closed loop with the animal, Ahab guided DaVinci to make image alterations that reduce the discriminability between stimulated and non-stimulated trials and increase the chances of FA. This closed-loop paradigm increased the FA rate from 3-4% to up to 85%. These images are called Perceptograms because seeing them is difficult for the animal to discern from the state of being cortically stimulated. We discovered that the structure of stimulation-induced percepts depends more on the concurrent visual input than the choice of cortical position. Although perceptograms obtained from anterior, IT follows the natural image manifold more than the posterior ones.