October 2020
Volume 20, Issue 11
Open Access
Vision Sciences Society Annual Meeting Abstract  |   October 2020
Bayesian Image Reconstruction from Retinal Cone Signals
Author Affiliations & Notes
  • Ling-Qi Zhang
    Department of Psychology, University of Pennsylvania
  • Nicolas P. Cottaris
    Department of Psychology, University of Pennsylvania
  • David H. Brainard
    Department of Psychology, University of Pennsylvania
  • Footnotes
    Acknowledgements  Funding: Facebook Reality Labs
Journal of Vision October 2020, Vol.20, 842. doi:https://doi.org/10.1167/jov.20.11.842
  • Views
  • Share
  • Tools
    • Alerts
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Ling-Qi Zhang, Nicolas P. Cottaris, David H. Brainard; Bayesian Image Reconstruction from Retinal Cone Signals. Journal of Vision 2020;20(11):842. doi: https://doi.org/10.1167/jov.20.11.842.

      Download citation file:

      © ARVO (1962-2015); The Authors (2016-present)

  • Supplements

The visual system takes sensory measurements of the light incident at the eyes and uses these to make perceptual inferences about external world. The sensory measurements do not preserve all of the information available in the light signal. One approach to understanding the implications of the first stages of visual processing is ideal observer analysis, which evaluates the information available to support psychophysical discriminations at various stages of the early visual representation. We are interested in extending this type of analysis to take into account the statistical structure of natural images. To do so, we developed a Bayesian method for reconstructing image stimuli from the signals available in the retinal cone mosaic. We evaluated the likelihood function using the open-source software package ISETBio (isetbio.org). ISETBio simulates the wavelength-dependent optical blur of the human eye as well as the interleaved sampling of the retinal image by the L, M and S cones. Noise in the cone signals is characterized by a Poisson process. To model the statistical structure of natural images, we applied independent components analysis to an image dataset, and fit an exponential prior to the individual component weights. We obtain reconstructions of the image stimulus using maximum a posteriori probability (MAP) estimates. Our method enables us to visualize and quantify the information loss due to optics and cone mosaic, while taking the spatial and spectral correlations of natural image statistics into account. To illustrate, we reconstructed images using a series of retinal mosaics with decreasing proportion of M cones. We quantified reconstructed image quality using the S-CIELAB difference between original and reconstructed images. We found that the reduction of M cones has little impact until the proportion of M cones drops below 5%, at which point the reconstruction error increases dramatically as the modeled visual system eventually becomes dichromatic.


This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.