August 2023
Volume 23, Issue 9
Open Access
Vision Sciences Society Annual Meeting Abstract  |   August 2023
Lesioning category-selective units in silico yields functionally specialized deficits
Author Affiliations & Notes
  • Jacob S. Prince
    Harvard University
  • George A. Alvarez
    Harvard University
  • Talia Konkle
    Harvard University
  • Footnotes
    Acknowledgements  This research was supported by NSF CAREER BCS-1942438.
Journal of Vision August 2023, Vol.23, 5657. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Jacob S. Prince, George A. Alvarez, Talia Konkle; Lesioning category-selective units in silico yields functionally specialized deficits. Journal of Vision 2023;23(9):5657.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Hallmark neuropsychological findings show that lesions within the ventral visual stream yield dissociable visual recognition deficits for different categories. It has remained unclear how to reconcile these observations with distributed population coding theories, which suggest that neurons decode object category information from multivariate (high, low, and medium) activity patterns. Here, we clarify the relationship between these prominent theoretical frameworks by simulating the functional impact of lesions applied to category-selective units that emerge within self-supervised deep neural networks. Using fMRI-inspired localizer techniques, we identified groups of units with selectivity for faces, scenes, bodies, and words along the hierarchy of AlexNet trained with a domain-general instance discrimination objective (Barlow Twins). After fitting a linear classifier to assess the model’s ImageNet performance, we measured the category-level deficits that arose from lesioning each group of units. We found that the degree to which individual categories drove a strong response before lesioning predicted their graded drop in accuracy after lesioning (face-selective r=0.58; scene-selective r=0.71; body-selective r=0.52; word-selective r=0.55), suggesting that these units effectively form dissociable “content channels” that are involved in processing some categories more than others (e.g. deficits were anticorrelated when lesioning face vs. scene units: r=–0.23). Theoretically, these results demonstrate that high activations are indicative of functional role, and that the tuning directions of a distributed population code are not rotationally arbitrary. Based on this “positive routing” theory, we developed a novel sparse positive-weighted readout scheme (in contrast to standard fully connected readout), and found the same recognition deficits due to lesions held under this more efficient procedure. Broadly, this work helps reconcile existing perspectives on the functional characteristics of object-responsive cortex, proposing that information is processed via positive activation through content-specific channels along the visual hierarchy, with sparse positive readout as a potential mode of efficient information transmission to downstream areas.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.