December 2022
Volume 22, Issue 14
Open Access
Vision Sciences Society Annual Meeting Abstract  |   December 2022
Task-dependent contribution of higher-order statistics to natural texture processing
Author Affiliations & Notes
  • Daniel Herrera
    Faculty of Sciences, Universidad de la República
  • Ruben Coen-Cagli
    Department of Systems and Computational Biology, and Dominick Purpura Department of Neuroscience, Albert Einstein College of Medicine
  • Footnotes
    Acknowledgements  NIH grant EY031166
Journal of Vision December 2022, Vol.22, 3542. doi:
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Daniel Herrera, Ruben Coen-Cagli; Task-dependent contribution of higher-order statistics to natural texture processing. Journal of Vision 2022;22(14):3542.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

During natural visual behavior our visual system extracts multiple features from its inputs and uses them to solve different tasks. Each feature conveys relevant information to different tasks, and for a given task our visual system relies on the relevant features while ignoring others. However, this hypothesis remains largely untested for complex tasks with natural stimuli. Here we compare the role of spectral and higher order (HOS) texture statistics of the Portilla-Simoncelli model across tasks using natural images, to explain their task-dependent use by humans. Portilla-Simoncelli HOS are important for human texture perception, peripheral vision, and physiology, but they play a much smaller role for texture segmentation. Modeling work suggests this could reflect the redundancy between HOS and spectral statistics (a strong segmentation cue in humans) for natural image segmentation. But the importance of HOS for texture perception suggests that these statistics may be informative for other texture-related tasks. In this work we test the hypothesis that, in contrast to segmentation, HOS are superior to spectral statistics for natural texture classification. To test this, we trained linear classifiers to solve 4 different natural image classification tasks (classification of physical texture instances, materials, perceptual descriptions and scenes) across 11 datasets, and compared the performance afforded by HOS and spectral statistics. We find that HOS improved task performance considerably over spectral statistics, unlike what was reported for segmentation. This is compatible with an account of the task-dependent use of these features by humans based on their task-dependent relevance in natural images. Interestingly, we find that the contribution of HOS varies between classification tasks, with larger improvements for instance classification. Future work should test whether use of HOS by humans follows this finer pattern within classification, and explore the computational underpinnings of the varying HOS contributions.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.