Abstract
The visual system takes sensory measurements of the light incident at the eyes and uses these to make perceptual inferences about external world. The sensory measurements do not preserve all of the information available in the light signal. One approach to understanding the implications of the first stages of visual processing is ideal observer analysis, which evaluates the information available to support psychophysical discriminations at various stages of the early visual representation. We are interested in extending this type of analysis to take into account the statistical structure of natural images.
To do so, we developed a Bayesian method for reconstructing image stimuli from the signals available in the retinal cone mosaic. We evaluated the likelihood function using the open-source software package ISETBio (isetbio.org). ISETBio simulates the wavelength-dependent optical blur of the human eye as well as the interleaved sampling of the retinal image by the L, M and S cones. Noise in the cone signals is characterized by a Poisson process. To model the statistical structure of natural images, we applied independent components analysis to an image dataset, and fit an exponential prior to the individual component weights. We obtain reconstructions of the image stimulus using maximum a posteriori probability (MAP) estimates.
Our method enables us to visualize and quantify the information loss due to optics and cone mosaic, while taking the spatial and spectral correlations of natural image statistics into account. To illustrate, we reconstructed images using a series of retinal mosaics with decreasing proportion of M cones. We quantified reconstructed image quality using the S-CIELAB difference between original and reconstructed images. We found that the reduction of M cones has little impact until the proportion of M cones drops below 5%, at which point the reconstruction error increases dramatically as the modeled visual system eventually becomes dichromatic.