Abstract
The light reflected from scenes under the sun and sky changes over the course of the day, yet the reflecting properties of individual surfaces appear unchanged. The phenomenon of color constancy is often attributed to operations applied to cone photoreceptor signals. These operations include cone-specific adaptation such as von Kries scaling, typically by average scene color or the brightest color; transformations of combinations of cone signals; and transformations of the whole color gamut, e.g., for optimum color discrimination. But are any of these or similar operations sufficient for constancy in the real world, where both spectral and geometric changes in illumination occur, including changes in shadows and mutual illumination? To address this question, cone signals were calculated from time-lapse hyperspectral radiance images of five different outdoor scenes containing mixtures of herbaceous vegetation, woodland, barren land, rock, and rural and urban buildings. Shannon's mutual information between cone signals was estimated across successive time intervals. Combined with the data processing inequality from information theory ("functions of data cannot increase information"), these estimates set an upper limit on the performance of any color constancy operation using cone signals alone. For all five scenes, the information limit declined markedly with increasing time interval, though not always monotonically. This pattern was little altered by changing the way that cone signals were initially sampled before information was estimated, e.g., taking spatial ratios of signals, omitting signals from dark regions of scenes, and using local statistical features (local mean, maximum, and standard deviation of cone signals). Moreover, dividing scenes into a mosaic of smaller patches for independent processing did not improve performance. It seems that operations on color signals alone are insufficient to uniquely identify the reflecting properties of individual surfaces. Reliable color constancy in the real world depends on more than just color.
Meeting abstract presented at VSS 2017