September 2019
Volume 19, Issue 10
Open Access
Vision Sciences Society Annual Meeting Abstract  |   September 2019
Deep neural network features predict perceptual sensitivity and cortical responses to visual textures
Author Affiliations & Notes
  • Akshay V Jagadeesh
    Department of Psychology, Stanford University
  • Justin L Gardner
    Department of Psychology, Stanford University
    Wu-Tsai Neuroscience Institute, Stanford University
Journal of Vision September 2019, Vol.19, 124b. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Akshay V Jagadeesh, Justin L Gardner; Deep neural network features predict perceptual sensitivity and cortical responses to visual textures. Journal of Vision 2019;19(10):124b.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

Textures with similar visual features, scrambled locally, can be obviously distinguishable when viewed directly, but metameric (i.e. perceptually indistinguishable) when viewed in the periphery (Freeman & Simoncelli, 2011). This suggests that the visual system pools complex features over small regions of space. Prior studies of texture perception have utilized textures synthesized by iteratively updating random noise images to match handcrafted features derived from a linear filter bank (Portilla & Simoncelli, 2001). Extending this approach, we generated textures by matching complex features extracted from various layers of the VGG-19 convolutional neural network (Gatys et al., 2011), pooled over uniform-sized subregions of a naturalistic image. We asked five human observers to distinguish original images from feature-matched textures, presented at 10 degrees eccentricity. Feature-matched textures were less distinguishable from original images when features from later layers (e.g. pool4) (F=346.42, p< 0.001) or within smaller pooling regions (F=68.59, p< 0.001) were matched. We modeled behavioral performance as a function of the distance between the features of the original image and texture on each trial. Comparing 12 different observer models, we found that a model using pool4 features computed within 2-degree pooling regions best predicts human performance on held-out trials. Furthermore, to assess the neural basis of texture perception, we measured BOLD activity in the visual cortex as five human subjects viewed texture images during ten 6 minute runs. Following published procedures (Freeman et al., 2013), images were flashed at 5Hz within 9 second blocks alternating between textures and phase-scrambles. We found that sensitivity to pool2 and pool4 textures relative to phase-scrambled images emerges in V2 and increases from V2 to V3 to V4. In sum, these results suggest that both feature complexity and pooling region size contribute to visual metamerism and that cortical representations in areas V2–V4 may support these perceptual effects.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.