A fundamental problem in image analysis is to understand the nature of the computations and mechanisms that provide information about the material properties of surfaces. Information about a surface's 3D shape, optics, illumination field, and atmospheric conditions are conflated in the image, which must somehow be disentangled to derive the properties of surfaces. It was recently suggested that the visual system exploits some simple image statistics—histogram or sub-band skew—to infer the lightness and gloss of surfaces (I. Motoyoshi, S. Nishida, L. Sharan, & E. H. Adelson, 2007). Here, we show that the correlations Motoyoshi et al. (2007) observed between skew, lightness, and gloss only arose because of the limited space of surface geometries, reflectance properties, and illumination fields they evaluated. We argue that the lightness effects they reported were a statistical artifact of equating the means of images with skewed histograms, and that the perception of gloss requires an analysis of the consistency between the estimate of a surface's 3D shape and the positions and orientations of highlights on a surface. We argue that the derivation of surface and material properties requires a photo-geometric analysis, and that purely photometric statistics such as skew fail to capture any diagnostic information about surfaces because they are devoid of the structural information needed to distinguish different types of surface attributes.

“The image of a surface arises from the combination of the surface geometry, the surrounding illumination, and the surface optics. Each of these components can be complex (for example, the reflectance at each point is characterized by a four-dimensional function known as the bidirectional reflectance distribution function). Each is typically unknown, and estimating any one using “inverse optics” requires knowing the others. To bypass this problem, we have looked for simple statistical image measurements that can provide information that is useful even if not complete (p. 206).”

*purely photometric*. Thus, the claim that pixel histograms provide information diagnostic of surface properties is tantamount to asserting that surface properties can be deduced from purely photometric information, i.e., that the geometric

*distribution*of pixel intensities is irrelevant. Although some geometric information is measured by sub-band filter outputs, Motoyoshi et al.'s experiments showed that such measures do not provide information about gloss and lightness much beyond that contained in luminance histograms (see Skew and adaptation section). Hence, the structural properties measured by sub-band skew do not appear to capture any relevant geometric constraints beyond those that might exist implicitly in histogram skew of the images that they studied. In what follows, we will therefore focus our discussion on the skew of luminance histograms and return to the role of sub-band skew when we consider non-uniform albedo surfaces.

*r*

^{2}= .76 and .79, respectively). Importantly, however, when a range of different uniform albedo materials was evaluated, the proportion of variance accounted for dropped significantly (

*r*

^{2}= .64 and .37 for lightness and gloss, respectively). What accounts for these correlations, and why did the consideration of more varied surfaces lead to such a sharp reduction in the correlation of skew with gloss?

*r*

^{2}values reported above). Although no theoretical justification is offered for the decision to equate the means of the images, it appears that the motivation for this decision lies in the attempt to equate the average image intensity across the images that observers compared, so that lightness judgments could not be based on this first order statistic. The question is whether equating means is a reasonable method of equating “average” image intensity for the images that they studied. When considered from a purely statistical perspective, it is well known that means are a poor estimate of central tendency for skewed distributions. Means are strongly affected by outliers, and their use as a measure of central tendency becomes increasingly inappropriate as the asymmetry of the underlying distribution—the skew—increases. Means are “pulled” in the direction of the outliers and hence provide a poor measure of where the central mass of a distribution is located. This simple statistical fact alone could fully explain the observed negative correlation between skew and perceived lightness reported by Motoyoshi et al. As the skew of the histogram increases, the majority of the surfaces that are rated as “darker” contain a larger proportion of dark pixels and a few sparse bright pixels; the converse is true for negative skew and light surfaces. In other words, the majority of pixels in images that are rated as darker

*are*darker than the images that are rated as lighter. The fact that observers' judgments may be determined by the mere preponderance of pixels in their images is supported by their finding that the lightness effects they reported were also observed in phase scrambled versions of their stimuli, which did not even appear as uniform albedo surfaces, or even very clear surfaces of any kind. Thus, the correlation between skew and lightness they report is likely to be a statistical artifact of equating an inappropriate measure of central tendency of image luminance for skewed distributions and hence does not provide any evidence that skew computations play a role in perceived lightness.

*amplitudes*of the intensities are affected, not their geometric distribution. In other words, manipulating the shape of the luminance histogram alters the strength of local contrasts (luminance gradients) in the image, but it does not alter their spatial distribution. This means that, although manipulating the luminance histogram is itself a purely photometric transformation, its efficacy in shaping perception of surface attributes may have arisen because such photometric transformations work in concert with the geometric structure present in the image to modulate perceived gloss, not because skew per se is computed to infer gloss.

*consistency*between the 3D shape of a surface, and the positions and orientations of highlights, then the perception of gloss should be strongly dependent on the locations and orientations of the highlights relative to the geometry of the surface.

^{2}and a white point of 82.4 cd/m

^{2}.

^{2}). Subjects viewed images at a distance of approximately 50 cm from the display (producing stimulus images subtending an approximate visual angle of 15 degrees). The participant could toggle between the two images in a 2AFC pairing by depressing the spacebar on a standard 101 PC keyboard at which time the screen was cleared to neutral gray for 1.0 s before the alternative image was displayed. When both images had been viewed at least once, the subject could select the glossier of the two images by making it visible and pressing the up arrow key to indicate their preference, where the screen was again cleared for 2.0 s before advancing to the next trial. For both rotational and translational stimuli, the order of presentation was randomly permutated and counterbalanced with up to 72 images (

*n*

^{2}−

*n,*where the number of images

*n*= 9) being presented in one block of stimulus trials. The experimental session was completed after participants completed four blocks of trials in each condition or when they exceeded the duration of their time slot.

*P*

_{i}is the probability of selecting image

*i*with an orientation field correlation of

*O*

_{i}and error parameter

*E*

_{0}(estimate of false positives) was taken as the inverse of the probability the image with the smallest (here, zero) angular/linear transformation of the gloss map was selected as glossier. Error parameter

*E*

_{1}(estimate of false negatives) was taken as the average probability the image(s) with the largest angular/linear transformation of their respective gloss maps was selected as glossier.

*appeared*as uniform albedo surfaces. We will return to this issue in the General discussion, where we will argue that claims of this form are logically circular. At this juncture, we were interested in determining whether it is possible to elicit a percept of gloss in images that appear as uniform albedo surfaces with some highlights, even if the image has a negative skew.

*a priori*that the surface viewed has a uniform albedo, and the illumination direction is within a fixed range of possible angles relative to the surface. But this is precisely the kind of analysis that image statistics like skew were supposed to circumvent. Thus, all of the same ambiguities in the 3D shape, surface optics, and illumination field still exist for any given value of skew and cannot be resolved by the kind of model proposed.

*photo-geometric*analysis. We contend that surfaces with different material and reflectance properties possess characteristic patterns of

*correlations*between the intensities in the image that allow them to be distinguished. Histogram skew is, by definition, a purely photometric quantity that does not capture any of the spatial correlations that convey the structure of a surface. The correlation between skew and surface properties like gloss and/or lightness only occurs when the other variables that contribute to image structure are held constant. In the Motoyoshi et al. (2007) studies using images of hand-made surfaces, gloss was the only parameter that was systematically varied; the illumination field, albedo, and surface geometry were all held constant. In this restricted context, there is a strong correlation between the physical gloss of surfaces and histogram (or sub-band) skew. Given that observers were able to discriminate the different degrees of surface gloss of these surfaces, their gloss ratings would have to correlate strongly with skew simply because skew is correlated with gloss in these particular images, even if skew is not explicitly computed by the visual system. Such data are a statistical inevitability that follows from the correlation between skew and physical gloss used in their studies and hence cannot provide any evidence for the role of skew in the perception of surface properties. Critically, the correlation between skew and gloss drops off precipitously when a broader range of surfaces and illumination environments is evaluated. We have shown that any histogram skew can be generated by non-uniform albedo surfaces containing a distribution of different surface pigments, and that the kinds of skew computations advocated by Motoyoshi et al. are incapable of distinguishing non-uniform albedo surfaces from surfaces that vary in gloss.

*r*

^{2}values falling from .79 to .37), even when other variables (like the illumination field) are constrained.

“While skewness is predictive of perceived surface qualities, it can of course be computed on arbitrary images, whether or not they look like surfaces. A picture of fireworks against the night sky will be positively skewed, but one cannot meaningfully judge its albedo or gloss; the same is true of the adapting stimulus of Figure 4a. Our findings were made in the case where the image is perceived as a surface of uniform albedo with some highlights. We do not know what aspects of image structure determine “surfaceness” or “highlights”. When our images are phase-scrambled so as to retain sub-band power, but not phase structure, they are typically seen as plausible but not convincing surfaces. The lightness effects are retained, but glossiness is lost. When the images are pixel-scrambled they are seen as two-dimensional noise patterns without a unitary albedo or gloss (p. 209).”

*perceived*as uniform albedo surfaces with some highlights. The problem with a statement of this kind is that statistics like skew are supposed to provide the information needed to

*determine*the nature of the surface being viewed. Their statement suggests that their theory only applies to certain kinds of perceptual

*outcomes,*the very thing that skew computations are supposed to help explain. In the above passage, Motoyoshi et al. acknowledge that statistics like skew cannot determine whether the image contains a surface of any kind, so it is difficult to understand how skew could provide a source of information about a specific kind of surface (i.e., glossy or matte, light or dark). Clearly, whatever additional geometric structure—i.e., spatial correlations in photometric information—is required to define a surface, uniform albedo or otherwise, cannot be derived from statistics like image skew. Their claim reduces to one of the form: histogram skew predicts the perceived albedo and gloss of a uniform albedo surface with some highlights if and only if a uniform albedo surface with some highlights is perceived. Even if the circular logic of this claim was accepted, the claim itself is demonstrably false. Skew does not predict the perception of gloss for surfaces that are perceived as uniform albedo surfaces when the illuminant is free to vary. As can be seen in Figure 9, the skew of a matte surface can be strongly positive by merely varying the position of the light source, without producing any appearance of surface gloss.

*requires*the analysis of image structures beyond that captured by histogram or sub-band skew, then there is no basis by which it can be asserted that sub-band skew plays any direct role in the perception of gloss. We have shown that skew fails to predict the perceived gloss of surfaces when the luminance maxima are not appropriately aligned with the underlying diffuse shading profile; when the illumination direction is systematically varied; or when non-uniform albedo surfaces are viewed. Any value of skew contains all of the same ambiguities in image interpretation that it was supposed to help resolve, namely, determining whether the luminance variations in an image arise from the illuminant, surface optics, or surface geometry.