August 2023
Volume 23, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2023
Perceptography: Revealing the causal contribution of the inferior temporal cortex to visual perception.
Author Affiliations
  • Elia Shahbazi
    National Institutes of Health
  • Timothy Ma
    Center for Neural Science, New York University
  • Arash Afraz
    National Institutes of Health
Journal of Vision August 2023, Vol.23, 5871. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Elia Shahbazi, Timothy Ma, Arash Afraz; Perceptography: Revealing the causal contribution of the inferior temporal cortex to visual perception.. Journal of Vision 2023;23(9):5871.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Cortical stimulation in high-level visual areas causes complex perturbations in visual perception. Understanding the nature of stimulation-induced visual perception is necessary for characterizing visual hallucinations in psychiatric diseases and developing visual prosthetics. Most evidence is derived from anecdotal observations of human patients, but systematic studies have been severely limited due to the lack of language faculty in nonhuman primates. We developed a novel method, perceptography, to “take pictures” of the complex visual percepts induced by optogenetic stimulation of the inferior temporal (IT) cortex in macaque monkeys. Each trial started with a fixation on a computer-generated image. Halfway through the image presentation (1s), we perturbed the image features for 200ms. At the same time, IT cortex was optogenetically stimulated via an implanted LED array in half of the trials at random. The animals were rewarded for detecting cortical stimulation by looking at one of the two subsequently presented targets. Under the hood, two deep learning systems, DaVinci (GAN) and Ahab (Deep-learning feature extraction pipeline), controlled image alterations and tracked the animals’ behavioral responses, respectively. We hypothesized that false alarms (FA) are more likely to happen when an image alteration shares common features with the percept induced by cortical stimulation. In a functional closed loop with the animal, Ahab guided DaVinci to make image alterations that reduce the discriminability between stimulated and non-stimulated trials and increase the chances of FA. This closed-loop paradigm increased the FA rate from 3-4% to up to 85%. These images are called Perceptograms because seeing them is difficult for the animal to discern from the state of being cortically stimulated. We discovered that the structure of stimulation-induced percepts depends more on the concurrent visual input than the choice of cortical position. Although perceptograms obtained from anterior, IT follows the natural image manifold more than the posterior ones.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.