September 2024
Volume 24, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2024
Mechanisms of a Convolutional Neural Network that Learns to Covertly Attend
Author Affiliations
  • Sudhanshu Srivastava
    University of California, Santa Barbara
  • Miguel P Eckstein
    University of California, Santa Barbara
Journal of Vision September 2024, Vol.24, 1312. doi:https://doi.org/10.1167/jov.24.10.1312
  • Views
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Sudhanshu Srivastava, Miguel P Eckstein; Mechanisms of a Convolutional Neural Network that Learns to Covertly Attend. Journal of Vision 2024;24(10):1312. https://doi.org/10.1167/jov.24.10.1312.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

The cueing task is one of the most prominent paradigms to study covert attention. Observers are quicker and more accurate in detecting a target when it appears with a cue (valid trials) than when it appears opposite to the cue (invalid trials). How neuronal populations across the visual hierarchy progressively represent and integrate visual information across the target, cues, and locations to give rise to the behavioral cueing effect is not well-understood. To gain a theoretical understanding of the plausible system-wide neuronal computations and mechanisms mediating the cueing effect, we analyze the response properties of 180k neurons per network across layers of ten feedforward Convolutional Neural Networks (CNN). The CNNs are trained on noisy images without any explicit attention mechanism and show human-like benefits of cues on detection accuracy. Early layers show retinotopic neurons separately tuned to target or cue with excitatory or inhibitory responses. Later layers show neurons jointly tuned to both target and cue and integrate information across locations. Consistent with physiological findings, we find increasing influence of the cue on target responses in deeper layers in the network and computational stages similar to those of a Bayesian ideal observer, but with more gradual transitions. The cue influences the mean neuronal response to the target and distractor, and changes target sensitivity with two mechanisms: integration of information across locations at the dense layer, and interaction with the thresholding Rectified Linear Unit (ReLU) in the last convolution layer. We find novel neuronal properties not yet reported in physiological studies: cue-inhibitory neurons, inhibitory cue influences on target neurons, and location-opponent cells, which are target-inhibitory at one location and target-excitatory at the other. Together, our analyses illustrate a system-wide analysis of the neuronal computations that might give rise to behavioral cueing effects and provide a theoretical framework to inform physiological studies.

×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×