Abstract
Evolution and learning guarantee that visual systems will exploit the statistical structure of natural images when performing visual tasks. Thus, understanding which aspects of this statistical structure are incorporated into the human nervous system is a fundamental goal in vision science. We consider the task of estimating missing gray scale or color (R, G or B) values at arbitrary pixel locations in natural images, given the context of available pixel values. First, we measured the relevant statistical information for these tasks in calibrated natural images [1]. We find that the statistical structure is sufficient for surprisingly accurate estimation of missing pixel values. Second, we measured human accuracy for estimating missing gray scale values and compared human accuracy with various simple heuristics (e.g., local average, median, and mode), and with optimal observers that have complete knowledge of the local statistical structure in natural images. We find that human estimates are more accurate than simple heuristics, and they match the performance of an optimal observer that knows the local statistical structure of relative intensities (contrasts). This optimal observer predicts the detailed pattern of human estimation errors and hence places strong constraints on the possible underlying neural mechanisms. However, humans do not reach the performance of an optimal observer that knows statistical structure of absolute intensities, which reflect both local relative intensities and local mean intensity. Also, as predicted from our analyses of natural images, human estimation accuracy is negligibly improved by expanding the context from a local patch to the whole image.
Meeting abstract presented at OSA Fall Vision 2012